But we could see that human beings use e.g. money as RL stimulus, and a lot of people try to get them counterfactually and we call them criminals. The same about sex, popularity and pleasure.
So complex intelligent beings which are humans are still easily corrupted by RL mechanisms in biology and society. The same could happen with AI. It would have much more opportunities to short-circut its RL system by rewriting itself than human being (and it could result to its halt if it will be able to it quickly).
So I just proved that self-improving AI halts. )))
The appropriate analogy would be people who get monetary reward by writing a letter to themselves, which reads like it came from the Lottery and tells them they have won the biggest payout in the history of the world. Then they mail the letter to themselves, receive it, open it, jump for joy, and go around telling everyone they can find that they are rich. And they believe it.
We tend not to invite those sorts of people to dinner parties.
But we could see that human beings use e.g. money as RL stimulus, and a lot of people try to get them counterfactually and we call them criminals. The same about sex, popularity and pleasure.
So complex intelligent beings which are humans are still easily corrupted by RL mechanisms in biology and society. The same could happen with AI. It would have much more opportunities to short-circut its RL system by rewriting itself than human being (and it could result to its halt if it will be able to it quickly).
So I just proved that self-improving AI halts. )))
Ah, not quite, no.
The appropriate analogy would be people who get monetary reward by writing a letter to themselves, which reads like it came from the Lottery and tells them they have won the biggest payout in the history of the world. Then they mail the letter to themselves, receive it, open it, jump for joy, and go around telling everyone they can find that they are rich. And they believe it.
We tend not to invite those sorts of people to dinner parties.