In this hypothetical, you have a superhuman AI actively trying to make the world nice. Therefore, whatever happens will be very nice.
Maybe the AI leaves some things to humans, you know, like art or something.
Isn’t the point of improving the world to make the world better. If you are working on a cancer cure, and hear the news that someone else succeeded, do you go “How dare they. I wanted to cure cancer. Me Me Me.” or do you go “yay”?
Why should it be any different with AI’s.
More to the point. People still play chess, even when the AI is totally better than them. People will find something to do. People do that. People don’t go around moping at the “hollow purposelessness of their existance”.
Then you have screwed up your alignment. Any time your woried that the AI will make the wrong decision, rather than that the AI will make the right decision, but you want to have made that decision, then your worried about an alignment failure. Now maybe it’s easier to make a hands off AI than a perfectly aligned AI. An AI that asks us it’s moral dilemmas because we can’t figure out how to make an AI choose correctly on it’s own.
But this is a “best we could do given imperfect alignment”. It isn’t the highest form of AI to aim for.
Problem is, our alignment is glitchy too. We are wired to keep running for the carrot that we will never be able to have for long. Because we will also strive for me. But AI can just teleport us to the “maximum carrot” point. Meanwhile, what we really need is not the destination, but the journey. At least, that’s what I believe into. Sadly, not much people understand/agree with it.
In this hypothetical, you have a superhuman AI actively trying to make the world nice. Therefore, whatever happens will be very nice.
Maybe the AI leaves some things to humans, you know, like art or something.
Isn’t the point of improving the world to make the world better. If you are working on a cancer cure, and hear the news that someone else succeeded, do you go “How dare they. I wanted to cure cancer. Me Me Me.” or do you go “yay”?
Why should it be any different with AI’s.
More to the point. People still play chess, even when the AI is totally better than them. People will find something to do. People do that. People don’t go around moping at the “hollow purposelessness of their existance”.
Helping solving health problems and prolonging life I can accept from AI, if we can’t solve it by ourselves.
But what if AI will decide that, say, being constantly maximally happy is nice, so it turns everyone into happy vegatables?
Then you have screwed up your alignment. Any time your woried that the AI will make the wrong decision, rather than that the AI will make the right decision, but you want to have made that decision, then your worried about an alignment failure. Now maybe it’s easier to make a hands off AI than a perfectly aligned AI. An AI that asks us it’s moral dilemmas because we can’t figure out how to make an AI choose correctly on it’s own.
But this is a “best we could do given imperfect alignment”. It isn’t the highest form of AI to aim for.
Problem is, our alignment is glitchy too. We are wired to keep running for the carrot that we will never be able to have for long. Because we will also strive for me. But AI can just teleport us to the “maximum carrot” point. Meanwhile, what we really need is not the destination, but the journey. At least, that’s what I believe into. Sadly, not much people understand/agree with it.