Most incentive designs rely on privacy, because by keeping a person’s actions off the record, you keep the meaning of those actions limited, confined, discrete, knowable. If, on the other hand, a person’s vote, say, is put onto a permanent public record, then you can no longer know what it means to them to vote. Once they can prove how they voted to external parties, they can be paid to vote a certain way. They can worry about retribution for voting the wrong way. Things that might not even exist yet, that the incentive designer couldn’t account for, now interfere with their behaviour. It becomes so much harder to reason about systems of agents, every act affects every other act, what hope have we of designing a robust society under those conditions? (Still quite a lot of hope, IMO, but it’s a noteworthy point)
In Vitalik Buterin’s interview on 80KHours (https://80000hours.org/podcast/episodes/vitalik-buterin-new-ways-to-fund-public-goods/ I recommend it) he brought something up that evoked a pretty stern criticism of radical transparency.
Most incentive designs rely on privacy, because by keeping a person’s actions off the record, you keep the meaning of those actions limited, confined, discrete, knowable. If, on the other hand, a person’s vote, say, is put onto a permanent public record, then you can no longer know what it means to them to vote. Once they can prove how they voted to external parties, they can be paid to vote a certain way. They can worry about retribution for voting the wrong way. Things that might not even exist yet, that the incentive designer couldn’t account for, now interfere with their behaviour. It becomes so much harder to reason about systems of agents, every act affects every other act, what hope have we of designing a robust society under those conditions? (Still quite a lot of hope, IMO, but it’s a noteworthy point)