This vague outline is the result of Eliezer yielding to our pleas to say something—anything—about his confident solution to Newcomb’s problem. Now that it’s been posted as a not-obviously-formalizable text, and people are discussing it informally, I share a lot of your disappointment. But let’s give the topic some days and see how it crystallizes.
What’s Flare? (...looks it up...) Oh dear Cthulhu, oh no.
(Edit: I originally listed several specific users as “refusing to formalize”. That was wrong.)
These considerations lead to the following design for the decision algorithm S. S is coded with a vector of programs that it cares about, and a utility function on vectors of the form that defines its preferences on how those programs should run. When it receives an input X, it looks inside the programs P1, P2, P3, …, and uses its “mathematical intuition” to form a probability distribution P_Y over the set of vectors for each choice of output string Y. Finally, it outputs a string Y* that maximizes the expected utility Sum P_Y() U().
Which part do you find insufficiently formal? Of course I use “mathematical intuition” as a black box without explaining how it works, but that’s just like EDT using “prior” without explaining where it comes from, or CDT using “causal probability” as a black box. It’s an unsolved problem, not refusal to formalize.
Your decision theory is formal enough for me, but it seems to be different from Eliezer’s, which I was talking about. If they’re really the same, could you explain how?
In that case, I never said I understood Eliezer’s version well enough that I could formalize it if I wanted to, and I don’t think Nesov and Drescher claimed that either, so I don’t know why you mentioned our names in connection with “refuse to formalize”. Actually I explicitly said that I don’t understand Eliezer’s theory very well yet.
This vague outline is the result of Eliezer yielding to our pleas to say something—anything—about his confident solution to Newcomb’s problem. Now that it’s been posted as a not-obviously-formalizable text, and people are discussing it informally, I share a lot of your disappointment. But let’s give the topic some days and see how it crystallizes.
What’s Flare? (...looks it up...) Oh dear Cthulhu, oh no.
(Edit: I originally listed several specific users as “refusing to formalize”. That was wrong.)
A legacy of pre-2003 Eliezer, of no particular importance one way or another.
What about what I wrote?
Which part do you find insufficiently formal? Of course I use “mathematical intuition” as a black box without explaining how it works, but that’s just like EDT using “prior” without explaining where it comes from, or CDT using “causal probability” as a black box. It’s an unsolved problem, not refusal to formalize.
Your decision theory is formal enough for me, but it seems to be different from Eliezer’s, which I was talking about. If they’re really the same, could you explain how?
In that case, I never said I understood Eliezer’s version well enough that I could formalize it if I wanted to, and I don’t think Nesov and Drescher claimed that either, so I don’t know why you mentioned our names in connection with “refuse to formalize”. Actually I explicitly said that I don’t understand Eliezer’s theory very well yet.
You’re right. I apologize. Amended the comment.