True, but 2012 might be long enough ago that many of the concerns he had then may now be irrelevant. In addition, based on my understanding of MIRI’s current approach and their arguments for that approach, I feel that many of his concerns either represent fundamental misunderstandings or are based on viewpoints that have significantly changed within MIRI since that time. For example, I have a hard time wrapping my head around this objection:
Objection 1: it seems to me that any AGI that was set to maximize a “Friendly” utility function would be extraordinarily dangerous.
This seems to be precisely the same concern expressed by MIRI and one of the fundamental arguments that their Agent Foundations approach is based on, in particular, what they deem the Value Specification problem. And I believe Yudkowsky has used this as a primary argument for AI safety in general for quite a while, very likely before 2012.
There is also the “tool/agent” distinction cited as objection 2 that I think is well addressed in MIRI’s publications as well as Bostrom’s Superintelligence, where it’s made pretty clear that the distinction is not quite that clear cut (and gets even blurrier the more intelligent the “tool AI” gets).
Given that MIRI has had quite some time to refine their views as well as their arguments, as well as having gone through a restructuring and hiring quite a few new researchers since that time, what is the likelihood that Holden holds the same objections that were stated in the 2012 review?
True, but 2012 might be long enough ago that many of the concerns he had then may now be irrelevant. In addition, based on my understanding of MIRI’s current approach and their arguments for that approach, I feel that many of his concerns either represent fundamental misunderstandings or are based on viewpoints that have significantly changed within MIRI since that time. For example, I have a hard time wrapping my head around this objection:
This seems to be precisely the same concern expressed by MIRI and one of the fundamental arguments that their Agent Foundations approach is based on, in particular, what they deem the Value Specification problem. And I believe Yudkowsky has used this as a primary argument for AI safety in general for quite a while, very likely before 2012.
There is also the “tool/agent” distinction cited as objection 2 that I think is well addressed in MIRI’s publications as well as Bostrom’s Superintelligence, where it’s made pretty clear that the distinction is not quite that clear cut (and gets even blurrier the more intelligent the “tool AI” gets).
Given that MIRI has had quite some time to refine their views as well as their arguments, as well as having gone through a restructuring and hiring quite a few new researchers since that time, what is the likelihood that Holden holds the same objections that were stated in the 2012 review?