Thanks for asking! This has been the subject of much virtual ink, much of it here (e.g. the book Superintelligence, and much of Eliezer’s writings, including the whole “Foom” debate with Robin Hanson). Rather than try to summarize that debate, I’ll encourage others to chime in with the links and explanations they think are best, since this is an important thing to get right and I’d like to see what our community currently thinks is the best way to explain this. I’m not thrilled that I don’t have a great go-to here.
If no one comes up with anything I’m happy with within the week, I’ll see if I can do better.
Thanks for asking! This has been the subject of much virtual ink, much of it here (e.g. the book Superintelligence, and much of Eliezer’s writings, including the whole “Foom” debate with Robin Hanson). Rather than try to summarize that debate, I’ll encourage others to chime in with the links and explanations they think are best, since this is an important thing to get right and I’d like to see what our community currently thinks is the best way to explain this. I’m not thrilled that I don’t have a great go-to here.
If no one comes up with anything I’m happy with within the week, I’ll see if I can do better.