Not really, no. Most people don’t recognize the “hidden complexity of wishes” in Far Mode, or when it’s their wishes. However, I think if I explain to them that I’ll be encoding my wishes, they’ll quickly figure out that my attempts to hardcode AI Friendliness are going to be very bad for them. Human intelligence evolved for winning arguments when status, wealth, health, and mating opportunities are at issue: thus, convince someone to treat you as an opponent, and leave the correct argument lying right where they can pick it up, and they’ll figure things out quickly.
Hmmm… I wonder if that bit of evolutionary psychology explains why many people act rude and nasty even to those close to them. Do we engage more intelligence when trying to win a fight than when trying to be nice?
Not really, no. Most people don’t recognize the “hidden complexity of wishes” in Far Mode, or when it’s their wishes. However, I think if I explain to them that I’ll be encoding my wishes, they’ll quickly figure out that my attempts to hardcode AI Friendliness are going to be very bad for them. Human intelligence evolved for winning arguments when status, wealth, health, and mating opportunities are at issue: thus, convince someone to treat you as an opponent, and leave the correct argument lying right where they can pick it up, and they’ll figure things out quickly.
Hmmm… I wonder if that bit of evolutionary psychology explains why many people act rude and nasty even to those close to them. Do we engage more intelligence when trying to win a fight than when trying to be nice?
You’ve missed the main point...it’s not that encoding wishes accurately is hard,it is that hardcoding isn’t .AI.