I think this story is very good, probably the most realistic story of AI takeover I’ve ever read; though, I don’t believe there’s any (edit: meaningful) chance AI will care about us enough to spare us a little sunlight.
“Don’t believe there is any chance” is very strong. If there is a viable way to bet on this I would be willing to bet at even odds that conditional on AI takeover, a few humans survive 100 years past AI takeover.
Do you think if an AI with random goals that doesn’t get acausally paid to preserve us takes over, then there’s a meaningful chance there will be some humans around in 100 years? What does it look like?
“Random goals” is a crux. Complicated goals that we can’t control well enough to prevent takeover are not necessarily uniformly random goals from whatever space you have in mind.
I think this story is very good, probably the most realistic story of AI takeover I’ve ever read; though, I don’t believe there’s any (edit: meaningful) chance AI will care about us enough to spare us a little sunlight.
“Don’t believe there is any chance” is very strong. If there is a viable way to bet on this I would be willing to bet at even odds that conditional on AI takeover, a few humans survive 100 years past AI takeover.
Do you think if an AI with random goals that doesn’t get acausally paid to preserve us takes over, then there’s a meaningful chance there will be some humans around in 100 years? What does it look like?
“Random goals” is a crux. Complicated goals that we can’t control well enough to prevent takeover are not necessarily uniformly random goals from whatever space you have in mind.