Author of meaningness.com, vividness.live, and other things.
MIT AI PhD, successful biotech entrepreneur, and other things.
David_Chapman
Karma: 232
Well, regardless of the value of metaprobability, or its lack of value, in the case of the black box, it doesn’t seem to offer any help in finding a decision strategy. (I find it helpful in understanding the problem, but not in formulating an answer.)
How would you go about choosing a strategy for the black box?
Well, I hope to continue the sequence… I ended this article with a question, or puzzle, or homework problem, though. Any thoughts about it?
So, how would you analyze this problem, more specifically? What do you think the optimal strategy is?
Hi, I have a site tech question. (Sorry if this is the wrong place to post that!—I couldn’t find any other.)
I can’t find a way to get email notifications of comment replies (i.e. when my inbox icon goes red). If there is one, how do I turn it on?
If there isn’t one, is that a deliberate design feature, or a limitation of the software, or...?
Thanks (and thanks especially to whoever does the system maintenance here—it must be a big job.)
Probability and radical uncertainty
Then why use it instead of learning the standard terms and using those?
The standard term is A_p, which seemed unnecessarily obscure.
Re the figure, see the discussion here.
(Sorry to be slow to reply to this; I got busy and didn’t check my LW inbox for more than a month.)
Thank you very much—link fixed!
That’s a really funny quote!
Multi-armed bandit problems were intractable during WWII probably mainly because computers weren’t available yet. In many cases, the best approach is brute force simulation. That’s the way I would approach the “blue box” problem (because I’m lazy).
But exact approaches have also been found: “Burnetas AN and Katehakis MN (1996) also provided an explicit solution for the important case in which the distributions of outcomes follow arbitrary (i.e., nonparametric) discrete, univariate distributions.” The blue box problem is within that class.
Thanks, yes! I.e. who is this “everyone else,” and where do they treat it the same way Jaynes does? I’m not aware of any examples, but I have only a basic knowledge of probability theory.
It’s certainly possible that this approach is common, but Jaynes wasn’t ignorant, and he seemed to think it was a new and unusual and maybe controversial idea, so I kind of doubt it.
Also, I should say that I have no dog in this fight at all; I’m not advocating “Jaynes is the greatest thing since sliced bread”, for example. (Although that does seem to be the opinion of some LW writers.)
Can you point me at some other similar treatments of the same problem? Thanks!
Thanks, that’s really funny! “On the other hand” is my general approach to life, so I’m happy to argue with myself.
And yes, I’m steelmanning. I think this approach is an excellent one in some cases; it will break down in others. I’ll present a first one in the next article. It’s another box you can put coins in that (I’ll claim) can’t usefully be modeled in this way.
Here’s the quote from Jaynes, by the way:
What are we doing here? It seems almost as if we are talking about the ‘probability of a probability’. Pending a better understanding of what that means, let us adopt a cautious notation that will avoid giving possibly wrong impressions. We are not claiming that P(Ap|E) is a ‘real probability’ in the sense that we have been using that term; it is only a number which is to obey the mathematical rules of probability theory.
Yes, meta-probabilities are probabilities, although somewhat odd ones; they obey the normal rules of probability. Jaynes discusses this in his Chapter 18; his discussion there is worth a read.
The statement “probability estimates are not, by themselves, adequate to make rational decisions” was meant to describe the entire sequence, not this article.
I’ve revised the first paragraph of the article, since it seems to have misled many readers. I hope the point is clearer now!
Probability, knowledge, and meta-probability
Are you claiming there’s no prior distribution over sequences which reflects our knowledge?
No. Well, not so long as we’re allowed to take our own actions into account!
I want to emphasize—since many commenters seem to have mistaken me on this—that there’s an obvious, correct solution to this problem (which I made explicit in the OP). I deliberately made the problem as simple as possible in order to present the A_p framework clearly.
Are we talking about the Laplace vs. fair coins?
Not sure what you are asking here, sorry...
We could also try to summarize some features of such epistemic states by talking about the instability of estimates—the degree to which they are easily updated by knowledge of other events
Yes, this is Jaynes’ A_p approach.
this will be a derived feature of the probability distribution, rather than an ontologically extra feature of probability.
I’m not sure I follow this. There is no prior distribution for the per-coin payout probabilities that can accurately reflect all our knowledge.
I reject that this is a good reason for probability theorists to panic.
Yes, it’s clear from comments that my OP was somewhat misleading as to its purpose. Overall, the sequence intends to discuss cases of uncertainty in which probability theory is the wrong tool for the job, and what to do instead.
However, this particular article intended only to introduce the idea that one’s confidence in a probability estimate is independent from that estimate, and to develop the A_p (meta-probability) approach to expressing that confidence.
So, let me try again to explain why I think this is missing the point… I wrote “a single probability value fails to capture everything you know about an uncertain event.” Maybe “simple” would have been better than “single”?
The point is that you can’t solve this problem without somehow reasoning about probabilities of probabilities. You can solve it by reasoning about the expected value of different strategies. (I said so in the OP; I constructed the example to make this the obviously correct approach.) But those strategies contain reasoning about probabilities within them. So the “outer” probabilities (about strategies) are meta-probabilistic.
[Added:] Evidently, my OP was unclear and failed to communicate, since several people missed the same point in the same way. I’ll think about how to revise it to make it clearer.
Glad you liked it!
I also get “stop after two losses,” although my numbers come out slightly differently. However, I suck at this sort of problem, so it’s quite likely I’ve got it wrong.
My temptation would be to solve it numerically (by brute force), i.e. code up a simulation and run it a million times and get the answer by seeing which strategy does best. Often that’s the right approach. However, sometimes you can’t simulate, and an analytical (exact, a priori) answer is better.
I think you are right about the sportsball case! I’ve updated my meta-meta-probability curve accordingly :-)
Can you think of a better example, in which the curve ought to be dead flat?
Jaynes uses “the probability that there was once life on Mars” in his discussion of this. I’m not sure that’s such a great example either.
Thanks! Fixed.
Yup, it’s definitely wrong! I was hoping no one would notice. I thought it would be a distraction to explain why the two are different (if that’s not obvious), and also I didn’t want to figure out exactly what the right math was to feed to my plotting package for this case. (Is the correct form of the curve for the p=0 case obvious to you? It wasn’t obvious to me, but this isn’t my area of expertise...)
Decisions are made on the basis of expected value, not probability.
Yes, that’s the point here!
your analysis of the first bet ignores the value of the information gained from it in executing your options for further play thereafter.
By “the first bet” I take it that you mean “your first opportunity to put a coin in a green box” (rather than meaning “brown box”).
My analysis of that was “you should put some coins in the box”, exactly because of the information gain.
This statement indicates a lack of understanding of Jaynes, or at least an adherence to his foundations.
This post was based closely on the Chapter 18 of Jaynes’ book, where he writes:
Suppose you have a penny and you are allowed to examine it carefully, and convince yourself that it is an honest coin; i.e. accurately round, with head and tail, and a center of gravity where it ought to be. Then you’re asked to assign a probability that this coin will come up heads on the first toss. I’m sure you’ll say 1⁄2. Now, suppose you are asked to assign a probability to the proposition that there was once life on Mars. Well, I don’t know what your opinion is there, but on the basis of all the things that I have read on the subject, I would again say about 1⁄2 for the probability. But, even though I have assigned the same ‘external’ probabilities to them, I have a very different ‘internal’ state of knowledge about those propositions.
Do you think he’s saying something different from me here?
This is interesting—it seems like the project here would be to construct a universal, hierarchical ontology of every possible thing a device could do? This seems like a very big job… how would you know you hadn’t left out important possibilities? How would you go about assigning probabilities?
(The approach I have in mind is simpler...)