At one point in the dialog, Scott raises what I think is a valid objection to the “nine people in the basement” picture of FAI’s development. He points out that it’s not how science progresses, and so not how he expects this novel development to happen.
I would like to hear more from Eliezer on just how likely he thinks the ‘nine people in the basement’ development scenario is.
My own impression would be that a more gradual development of GAI is more likely but that that ‘basement development’ is the only way there is even a remote possibility that the development will not lead to rapid human extinction. That would make the ‘nine people in the basement picture’ either wishful thinking or ‘my best plan of action’ depending on whether or not we are Eliezer.
I would like to hear more from Eliezer on just how likely he thinks the ‘nine people in the basement’ development scenario is.
My own impression would be that a more gradual development of GAI is more likely but that that ‘basement development’ is the only way there is even a remote possibility that the development will not lead to rapid human extinction. That would make the ‘nine people in the basement picture’ either wishful thinking or ‘my best plan of action’ depending on whether or not we are Eliezer.