My apologies for challenging the premise, but I don’t understand how anyone could hope to be “convinced” that humanity isn’t doomed by AGI unless they’re in possession of a provably safe design that they have high confidence of being able to implement ahead of any rivals.
Put aside all of the assumptions you think the pessimists are making and simply ask whether humanity knows how to make a mind that will share our values. It it does, please tell us how. If it doesn’t, then accept that any AGI we make is, by default, alien—and building an AGI is like opening a random portal to invite an alien mind to come play with us.
What is your prior for alien intelligence playing nice with humanity—or for humanity being able to defeat it? I don’t think it’s wrong to say we’re not automatically doomed. But let’s suppose we open a portal and it turns out ok: We share tea and cookies with the alien, or we blow its brains out. Whatever. What’s to stop humanity from rolling the dice on another random portal? And another? Unless we just happen to stumble on a friendly alien that will also prevent all new portals, we should expect to eventually summon something we can’t handle.
Feel free to place wagers on whether humanity can figure out alignment before getting a bad roll. You might decide you like your odds! But don’t confuse a wager with a solution.
If doomed means about 0% chance of survival then you don’t need to know for sure a solution exists to not be convinced we are doomed.
Solutions: SuperAGI proves hard, harder then using narrow AI to solve the Programmer/ Human control problem. (That’s what I’m calling the problem of it being inevitable that someone somewhere will make dangerous AGI if they can).
Constant surveillance of all person’s and all computers made possible by narrow AI, perhaps with subhuman AGI, and some very stable political situation could make this possible. Perhaps for millions of years.
My apologies for challenging the premise, but I don’t understand how anyone could hope to be “convinced” that humanity isn’t doomed by AGI unless they’re in possession of a provably safe design that they have high confidence of being able to implement ahead of any rivals.
Put aside all of the assumptions you think the pessimists are making and simply ask whether humanity knows how to make a mind that will share our values. It it does, please tell us how. If it doesn’t, then accept that any AGI we make is, by default, alien—and building an AGI is like opening a random portal to invite an alien mind to come play with us.
What is your prior for alien intelligence playing nice with humanity—or for humanity being able to defeat it? I don’t think it’s wrong to say we’re not automatically doomed. But let’s suppose we open a portal and it turns out ok: We share tea and cookies with the alien, or we blow its brains out. Whatever. What’s to stop humanity from rolling the dice on another random portal? And another? Unless we just happen to stumble on a friendly alien that will also prevent all new portals, we should expect to eventually summon something we can’t handle.
Feel free to place wagers on whether humanity can figure out alignment before getting a bad roll. You might decide you like your odds! But don’t confuse a wager with a solution.
If doomed means about 0% chance of survival then you don’t need to know for sure a solution exists to not be convinced we are doomed.
Solutions: SuperAGI proves hard, harder then using narrow AI to solve the Programmer/ Human control problem. (That’s what I’m calling the problem of it being inevitable that someone somewhere will make dangerous AGI if they can).
Constant surveillance of all person’s and all computers made possible by narrow AI, perhaps with subhuman AGI, and some very stable political situation could make this possible. Perhaps for millions of years.