We won’t know if he is right, because “no fire alarm...” etc. But a “wrong” scenario might be something like “look at this ML marvel writing all the code ever needed, being a perfect companion, proving every theorem, fixing aging, inventing interstellar propulsion, and yet indifferent to world domination, just chilling”.
Yes, that would qualify. Even if it turns out that the AI was actually treacherous and kills us all later, that would still fall under a class of world-models he disclaimed.
Likewise there was a disagreement about speed of world GDP growth before transformative AI.
We might know that Eliezer is right, for a very short time before we all die.
As far as I understand his model, there might be a few AGIs that work like this but if you have 9 AGI’s like this and then someone creates an AGI that optimizes for world domination, then that AGI is likely going to take over.
A few good AGI’s that don’t make pivotal moves don’t end the risk.
We won’t know if he is right, because “no fire alarm...” etc. But a “wrong” scenario might be something like “look at this ML marvel writing all the code ever needed, being a perfect companion, proving every theorem, fixing aging, inventing interstellar propulsion, and yet indifferent to world domination, just chilling”.
Yes, that would qualify. Even if it turns out that the AI was actually treacherous and kills us all later, that would still fall under a class of world-models he disclaimed.
Likewise there was a disagreement about speed of world GDP growth before transformative AI.
We might know that Eliezer is right, for a very short time before we all die.
As far as I understand his model, there might be a few AGIs that work like this but if you have 9 AGI’s like this and then someone creates an AGI that optimizes for world domination, then that AGI is likely going to take over.
A few good AGI’s that don’t make pivotal moves don’t end the risk.