If I understood you correctly, I think that people do do this kind of thing, except it’s all nonverbal and implicit. E.g. Using hard to fake tests for the other person’s decision theory is a way to make the other person honestly reveal what’s going on inside them. Another component is use of strong emotions, which are sort of like a precommitment mechanism for people, because once activated, they are stable.
Yes, I understand the signal must be hard to fake. But if the concern is merely about optimizing signal quality, wouldn’t it be an even stronger mechanism to noticeably couple your payoff profile to a credible mechanism?
Just as a sketch, find some “punisher” that noticeably imposes disutility (like repurposing the signal faker’s means toward paperclip production, since that’s such such a terrible outcome, apparently) on you whenever you deviate from your purported decision theory. It’s rather trivial to have a publicly-viewable database of who is coupled to the punisher (and by what decision theory), and to make it verifiable that any being with which you are interacting matches a specific database entry.
This has the effect of elevating your signal quality to that of the punisher’s. Then, it’s just a problem of finding a reliable punisher.
We do. That’s one of the functions of reputation and gossip among humans, and also the purpose of having a legal system. But it doesn’t work perfectly: we have yet to find a reliable punisher, and if we did find one it would probably need to constantly monitor everyone and invade their privacy.
Attention Users: please provide me with your decision theory, and what means I should use to enforce your decision theory so that you can reliably claim to adhere to it.
For this job, I request 50,000 USD as compensation, and I ask that it be given to User:Kevin.
If I understood you correctly, I think that people do do this kind of thing, except it’s all nonverbal and implicit. E.g. Using hard to fake tests for the other person’s decision theory is a way to make the other person honestly reveal what’s going on inside them. Another component is use of strong emotions, which are sort of like a precommitment mechanism for people, because once activated, they are stable.
Yes, I understand the signal must be hard to fake. But if the concern is merely about optimizing signal quality, wouldn’t it be an even stronger mechanism to noticeably couple your payoff profile to a credible mechanism?
Just as a sketch, find some “punisher” that noticeably imposes disutility (like repurposing the signal faker’s means toward paperclip production, since that’s such such a terrible outcome, apparently) on you whenever you deviate from your purported decision theory. It’s rather trivial to have a publicly-viewable database of who is coupled to the punisher (and by what decision theory), and to make it verifiable that any being with which you are interacting matches a specific database entry.
This has the effect of elevating your signal quality to that of the punisher’s. Then, it’s just a problem of finding a reliable punisher.
Why not just do that, for example?
We do. That’s one of the functions of reputation and gossip among humans, and also the purpose of having a legal system. But it doesn’t work perfectly: we have yet to find a reliable punisher, and if we did find one it would probably need to constantly monitor everyone and invade their privacy.
Yet another reason why people invented religion...
Well it looks like you just got yourself a job ;-0
That is good!
Attention Users: please provide me with your decision theory, and what means I should use to enforce your decision theory so that you can reliably claim to adhere to it.
For this job, I request 50,000 USD as compensation, and I ask that it be given to User:Kevin.