You lost points because nothing you said even begins to address the problem. You seem to be arguing that contradicting ourselves isn’t that bad, which might be defensible if we observed that some particular type of improper prior got good results in practice. (Though Eliezer would still argue against using it unless you’ve tried and failed to find a better way.) But here we want to know:
whether or not we have a reason to act on bizarre claims like the mugger’s—which we presumably don’t if the argument for doing so is incoherent
what principle we could use to reject the mugger’s unhelpful and intuitively ridiculous demand without causing problems elsewhere.
On a side-note, we don’t care whether this seemingly crazy person is “honest”, but whether his claim is correct (or whether paying him has higher expected value than not).
You lost points because nothing you said even begins to address the problem. You seem to be arguing that contradicting ourselves isn’t that bad, which might be defensible if we observed that some particular type of improper prior got good results in practice. (Though Eliezer would still argue against using it unless you’ve tried and failed to find a better way.) But here we want to know:
whether or not we have a reason to act on bizarre claims like the mugger’s—which we presumably don’t if the argument for doing so is incoherent
what principle we could use to reject the mugger’s unhelpful and intuitively ridiculous demand without causing problems elsewhere.
On a side-note, we don’t care whether this seemingly crazy person is “honest”, but whether his claim is correct (or whether paying him has higher expected value than not).