I find I strongly agree that—in case of this future happening—it is extremely important that as little people as possible give up on their attempts to perceive the world as it really is, even if that world might literally ‘we failed, humanity is going to die out, the problem is way too hard and there’s no reasonable chance we’ll make it in time’. It seems to me like especially in scenarios like this, we’d need people to keep trying, and to stay (or become) dignified in order to have any chance at still solving the problem at all.
I’m a total newcomer though, and it might be obvious for everyone more immersed in alignment, but what I really don’t understand is why ‘failure to solve the problem in time’ sounds so much like ‘we’re all going to die, and that’s so certain that some otherwise sensible people are tempted to just give in to despair and stop trying at all’. From what I’ve seen so far, this attitude is very common here, and I’d greatly appreciate if anyone could point me to some resources explaining the basics.
I’m seventeen, on the brink of having to choose a career, and interested in science (and the future of humanity, obviously) anyway, so if the problem really is as massive and under-focused on as it sounds, it seems like a good idea for me to join the battle as soon as possible—if that would even make a difference at this point.
No, until I googled it I never heard of it before. Thank you for pointing it out! I’m quite fascinated, now, I’d appreciate if you could tell me some more about it.
Have you considered applying to the AGI Safety Fundamentals Course? (They might not accept minors though b/c of legal reasons, but you’ll be an adult soon enough).
Thanks for the advice, I appreciate it. I’m one of these people who have very firm rules about not lying though. Then again, I did manage to get a vaccine despite parents objecting using the second option, so I suppose it’ll be worth a try :)
“what I really don’t understand is why ‘failure to solve the problem in time’ sounds so much like ‘we’re all going to die, and that’s so certain that some otherwise sensible people are tempted to just give in to despair and stop trying at all’ ”
I agree. In this community, most people only talk of x-risk (existential risk). Most people equate failure to align AI to our values to human extinction. I disagree. Classic literature examples of failure can be found, like With Folded Hands, where AI creates an unbreakable dictatorship, not extinction.
I think it’s for the sake of sanity (things worse than extinction are quite harder to accept), or not to scare the normies, who are already quite scared.
But it’s also true that unaligned AI could result in a kinda positive outcome, or even neutral. I just personally wouldn’t put much probability in there. Why? 2 concepts that you can look up on Lesswrong: orthogonality thesis (high intelligence isn’t necessarily correlated to high values), and basic AI drives (advanced AI would naturally develop dangerous instrumental goals like survival and resource acquisition). And also that it’s pretty hard to tell computers to do what we mean, which scaled up could turn out very dangerous.
(See Eliezer’s post “Failed Utopia 4-2”, where an unaligned AGI ends up creating a failed utopia which really doesn’t sound THAT bad, and I’d say is even much better than the current world when you weight all the good and bad.)
Fundamentally, we just shouldn’t take the gamble. The stakes are too high.
If you wanna have an impact, AI is the way to go. Definitely.
I find I strongly agree that—in case of this future happening—it is extremely important that as little people as possible give up on their attempts to perceive the world as it really is, even if that world might literally ‘we failed, humanity is going to die out, the problem is way too hard and there’s no reasonable chance we’ll make it in time’. It seems to me like especially in scenarios like this, we’d need people to keep trying, and to stay (or become) dignified in order to have any chance at still solving the problem at all.
I’m a total newcomer though, and it might be obvious for everyone more immersed in alignment, but what I really don’t understand is why ‘failure to solve the problem in time’ sounds so much like ‘we’re all going to die, and that’s so certain that some otherwise sensible people are tempted to just give in to despair and stop trying at all’. From what I’ve seen so far, this attitude is very common here, and I’d greatly appreciate if anyone could point me to some resources explaining the basics.
I’m seventeen, on the brink of having to choose a career, and interested in science (and the future of humanity, obviously) anyway, so if the problem really is as massive and under-focused on as it sounds, it seems like a good idea for me to join the battle as soon as possible—if that would even make a difference at this point.
Have you looked at the Atlas Fellowship, btw?
No, until I googled it I never heard of it before. Thank you for pointing it out! I’m quite fascinated, now, I’d appreciate if you could tell me some more about it.
This post explains more, I don’t have any other info: https://www.lesswrong.com/posts/F7RgpHHDpZYBjZGia/high-schoolers-can-apply-to-the-atlas-fellowship-usd50k
My appliance was submitted, I mentioned you pointed me to it. Thanks a lot!
Have you considered applying to the AGI Safety Fundamentals Course? (They might not accept minors though b/c of legal reasons, but you’ll be an adult soon enough).
OP, if you’re interested, lie about your age or just don’t mention it. They’re not going to check as long as you don’t look six.
Thanks for the advice, I appreciate it. I’m one of these people who have very firm rules about not lying though. Then again, I did manage to get a vaccine despite parents objecting using the second option, so I suppose it’ll be worth a try :)
That’s the spirit!
Thanks a lot! I’ll look into applying!
“what I really don’t understand is why ‘failure to solve the problem in time’ sounds so much like ‘we’re all going to die, and that’s so certain that some otherwise sensible people are tempted to just give in to despair and stop trying at all’ ”
I agree. In this community, most people only talk of x-risk (existential risk). Most people equate failure to align AI to our values to human extinction. I disagree. Classic literature examples of failure can be found, like With Folded Hands, where AI creates an unbreakable dictatorship, not extinction.
I think it’s for the sake of sanity (things worse than extinction are quite harder to accept), or not to scare the normies, who are already quite scared.
But it’s also true that unaligned AI could result in a kinda positive outcome, or even neutral. I just personally wouldn’t put much probability in there. Why? 2 concepts that you can look up on Lesswrong: orthogonality thesis (high intelligence isn’t necessarily correlated to high values), and basic AI drives (advanced AI would naturally develop dangerous instrumental goals like survival and resource acquisition). And also that it’s pretty hard to tell computers to do what we mean, which scaled up could turn out very dangerous.
(See Eliezer’s post “Failed Utopia 4-2”, where an unaligned AGI ends up creating a failed utopia which really doesn’t sound THAT bad, and I’d say is even much better than the current world when you weight all the good and bad.)
Fundamentally, we just shouldn’t take the gamble. The stakes are too high.
If you wanna have an impact, AI is the way to go. Definitely.