Yes, it seems to work for human reputation. If I precommit to cooperation with others from, say, school, it will be eventually well known to people around—even if I can’t predict how they will learn it. And if I am a scumbag, it will be also part of my reputation.
Sure, there are people, who become scumbags only when they expect that nobody will ever know, but it is a bad strategy as it requires complete knowledge about all possible futures which is computationally expensive and likely impossible. So better to be nice from the start.
Agree, though I was not referring to this strategy but a variant thereof. We as human society might want to move away from meat consumption for that reason.
For an agi that can make part of itself legible it might be a good strategy to prove that it was probably cooperating (to some degree) to other agents in it’s lightcone.
Yes, it seems to work for human reputation. If I precommit to cooperation with others from, say, school, it will be eventually well known to people around—even if I can’t predict how they will learn it. And if I am a scumbag, it will be also part of my reputation.
Sure, there are people, who become scumbags only when they expect that nobody will ever know, but it is a bad strategy as it requires complete knowledge about all possible futures which is computationally expensive and likely impossible. So better to be nice from the start.
Agree, though I was not referring to this strategy but a variant thereof. We as human society might want to move away from meat consumption for that reason.
For an agi that can make part of itself legible it might be a good strategy to prove that it was probably cooperating (to some degree) to other agents in it’s lightcone.
Interesting point about meat consumption.