Gwern, I happen to agree with most of what you’ve said, if this were written in regards to x-risks. It is in fact irrelevant to UFAI, but was mostly an exercise in a) practicing writing, and b) working through some intuitions in regards to betting/prediction markets. I wrote it for LW because I assumed it would be enjoyed, but not really learned from (hence Discussion, not Main). A re-write would explore more thoroughly and explicitly the difference between Orin being correct, a spy, or mistaken, and how his bet changes those probabilities.
I suppose it makes an ok-ish example of “people take their money more seriously than their beliefs, and betting helps fix that” Which I think is am important lesson in general.
b) working through some intuitions in regards to betting/prediction markets.
Why, then, did you choose an existential threat issue to base the entire story on when you know how many issues those bring up for prediction markets with regard to incentives and counterparty risk and the difficulty of ‘betting on the apocalypse’? You should have chosen an issue more clearly within prediction markets’ scope.
Maybe you should read it as a fable and not as a blueprint for dealing with the UFAI problem.
Why? As a fable it is boring and irrelevant, and clearly OP did not intend it to be taken the way you suggest taking it.
Gwern, I happen to agree with most of what you’ve said, if this were written in regards to x-risks. It is in fact irrelevant to UFAI, but was mostly an exercise in a) practicing writing, and b) working through some intuitions in regards to betting/prediction markets. I wrote it for LW because I assumed it would be enjoyed, but not really learned from (hence Discussion, not Main). A re-write would explore more thoroughly and explicitly the difference between Orin being correct, a spy, or mistaken, and how his bet changes those probabilities.
I suppose it makes an ok-ish example of “people take their money more seriously than their beliefs, and betting helps fix that” Which I think is am important lesson in general.
Why, then, did you choose an existential threat issue to base the entire story on when you know how many issues those bring up for prediction markets with regard to incentives and counterparty risk and the difficulty of ‘betting on the apocalypse’? You should have chosen an issue more clearly within prediction markets’ scope.