I think the usual rejoinder on the “AI go foom” side is that we are likely to overestimate x by underestimating what really effective thinking can do
Well, yeah, and on the whole, it’s the kind of assumption that one can’t scientifically prove or disprove. It’s something that can’t be observed yet and that we’ll see play out (hopefully) this century.
I guess the main issue I see with this stance is not that it’s unfounded, but that its likely cause is something like <childhood indoctrination to hold having good grade, analytical thinking, etc as the highest value in life>, as in, it would perfectly explain why it seems to be readily believed by anyone that stumbles upon less wrong, whereas few/no other beliefs (that don’t have a real-world observation to prove/disprove them) are so widely shared here (as well as more generally in a lot of nerdy communities).
Granted, I can’t “prove” this one way or another, but I think it helps to have some frameworks of thinking that are able to persuade people that start from an “intelligence is supreme” perspective towards the centre, much like the alien story might persuade people that start from an “intelligence can’t accomplish much” perspective.
Well, yeah, and on the whole, it’s the kind of assumption that one can’t scientifically prove or disprove. It’s something that can’t be observed yet and that we’ll see play out (hopefully) this century.
I guess the main issue I see with this stance is not that it’s unfounded, but that its likely cause is something like <childhood indoctrination to hold having good grade, analytical thinking, etc as the highest value in life>, as in, it would perfectly explain why it seems to be readily believed by anyone that stumbles upon less wrong, whereas few/no other beliefs (that don’t have a real-world observation to prove/disprove them) are so widely shared here (as well as more generally in a lot of nerdy communities).
Granted, I can’t “prove” this one way or another, but I think it helps to have some frameworks of thinking that are able to persuade people that start from an “intelligence is supreme” perspective towards the centre, much like the alien story might persuade people that start from an “intelligence can’t accomplish much” perspective.