I think that LWers assign a much higher probability to a FOOM scenario that most people. Most people probably wouldn’t assign much value to an AI that just seeks to maximize the number of paperclips in the universe, and continuously attempts to improve its ability to make that goal happen. Someone could build something like that expecting that its abilities would level off pretty quickly, and be badly wrong.
I think that LWers assign a much higher probability to a FOOM scenario that most people. Most people probably wouldn’t assign much value to an AI that just seeks to maximize the number of paperclips in the universe, and continuously attempts to improve its ability to make that goal happen. Someone could build something like that expecting that its abilities would level off pretty quickly, and be badly wrong.