Publication of “Anthropic Decision Theory”
My paper “Anthropic decision theory for self-locating beliefs”, based on posts here on Less Wrong, has been published as a Future of Humanity Institute tech report. Abstract:
This paper sets out to resolve how agents ought to act in the Sleeping Beauty problem and various related anthropic (self-locating belief) problems, not through the calculation of anthropic probabilities, but through finding the correct decision to make. It creates an anthropic decision theory (ADT) that decides these problems from a small set of principles. By doing so, it demonstrates that the attitude of agents with regards to each other (selfish or altruistic) changes the decisions they reach, and that it is very important to take this into account. To illustrate ADT, it is then applied to two major anthropic problems and paradoxes, the Presumptuous Philosopher and Doomsday problems, thus resolving some issues about the probability of human extinction.
Most of these ideas are also explained in this video.
To situate Anthropic Decision Theory within the UDT/TDT family: it’s basically a piece of UDT applied to anthropic problems, where the UDT approach can be justified by using generally fewer, and more natural, assumptions than UDT does.
Congratulations!
Just a quick note on another possible way to present this idea. A few years ago I realized that the simple subset of UDT can be formulated as a certain kind of single player game. It seems like the most natural way to connect UDT to standard terminology, and it’s very crisp mathematically. Then one can graduate to the modal version which goes a little deeper and is just as crisp, and decidable to boot. That’s the path my dream paper would take, if I didn’t have a job and a million other responsibilities :-/
What would be required for UDT to be written up fully? And what is missing between FDT (in the Death in Damascus problem) and UDT?
I’m puzzled by the FDT paper, it claims to be a generalization of UDT but it seems strictly less general, the difference being this.
As to your first question, we already have several writeups that fit in the context of decision theory literature (TDT, FDT, ADT) but they omit many ideas that would fit better in a different context, the intersection of game theory and computation (like the paper on program equilibrium by Tennenholtz). Thinking back, I played a large part in developing these ideas, and writing them up was probably my responsibility which I flunked :-( Wei’s reluctance to publish also played a role though, see the thread “Writing up the UDT paper” on the workshop list in Sep 2011.
So do you think it’s worth writing up, or getting someone to do so?
Yeah. I’ve been feeling a bit guilty so I started another attempt at a writeup, in a week or two we’ll see if it goes anywhere.
I think the second thing to do is to list all the problems, and what the correct answer is/what answer the algorithms give.
My current outline of UDT is organized by levels:
1) Indexical uncertainty, which is solved by converting to single player games with imperfect information. This level is basically playing with graphs. Absent-Minded Driver, Wei’s coordination problem, Psy-Kosh’s problem. Interpreting anthropic problems as choosing the right game, like in your work.
2) Cartesian uncertainty, where your copies aren’t delineated in the world and you need to find them first, then reduce the problem to level 1. This level is where self-referential sentences come in. Symmetric PD, Newcomb’s Problem, Counterfactual Mugging. Models based on halting oracles, Peano arithmetic, modal logic.
3) Logical uncertainty, where you can’t do level 2 crisply because your power is limited. This level is about approximations and bounds. Proof searchers, spurious counterfactuals, logical inductors, logical updatelessness.
4) Full on game theory, where even level 3 isn’t enough because there are other powerful agents around. This level is pretty much warfare and chaos. Bargaining, blackmail, modal combat, agent simulates predictor.
At this point I feel that we have definitively solved levels 1 and 2, are making progress on level 3, and have a few glimpses of level 4. But even on the first two levels, writing good exposition is a challenge. I’ll send you drafts as I go.
Good decomposition, though I am sceptical that there is much clean at level 4.
It’s basically a piece of UDT applied to anthropic problems, where the UDT approach can be justified by using generally fewer, and more natural, assumptions than UDT does.
bullet force -