The Bayesian Universalist answer to this would be that there is no separate meta-probability. You have a universal prior over all possible hypotheses, and mutter a bit about Solomonoff induction and AIXI.
I am putting it this way, distancing myself from the concept, because I don’t actually believe it, but it is the standard answer to draw out from the LessWrong meme space, and it has not yet been posted in this thread. Is there anyone who can make a better fist of expounding it?
Solomonoff induction is extraordinarily unhelpful, I think… that it is uncomputable is only one reason.
Because it’s output is not human-readable being the other?
I mean, even if I’ve got a TARDIS to use as a halting oracle, an Inductive Turing Machine isn’t going to output something I can actually use to make predictions about specific events such as “The black box gives you money under X, Y, and Z circumstances.”
Well, the problem I was thinking of is “the universe is not a bit string.” And any unbiased representation we can make of the universe as a bit string is going to be extremely large—much too large to do even sane sorts of computation with, never mind Solomonoff.
Maybe that’s saying the same thing you did? I’m not sure...
Can you please give us a top level post at some point, be it in Discussion or Main, arguing that “the universe is not a bit string”? I find that very interesting, relevant, and plausible.
Going back to the basic question about the black box:
What is the probability of its giving you $2?
Too small to be worth considering. I might as well ask, what’s the probability that I’ll find $2 hidden half way up the nearest tree? Nothing has been claimed about the black box to specifically draw “it will pay you $2 for $1” out of hypothesis space.
Hmm… given that the previous several boxes have either paid $2 or done nothing, it seems like that primes the hypothesis that the next in the series also pays $2 or does nothing. (I’m not actually disagreeing, but doesn’t that argument seem reasonable?)
it seems like that primes the hypothesis that the next in the series also pays $2 or does nothing
Priming a hypothesis merely draws it to attention; it does not make it more likely. Every piece of spam, every con game, “primes the hypothesis” that it is genuine. It also “primes the hypothesis” that it is not. “Priming the hypothesis” is no more evidence than a purple giraffe is evidence of the blackness of crows.
Explicltly avoiding saying that it does pay $2, and saying instead that it is “interesting”, well, that pretty much stomps the “priming” into a stain on the sidewalk.
Well, yes. As is the mere presence of the idea of $2 for $1 terrible evidence that the black box will do any such thing.
Eliezer speaks in the Twelve Virtues of letting oneself be as light as a leaf, blown unresistingly by the wind of evidence, but evidence of this sort is on the level of the individual molecules and Brownian motion of that leaf.
You can give a meta-probability if you want. However, this makes no difference in your final result. If you are 50% certain that a box has a diamond in it with 20% probability, and you are 50% certain that it has a diamond with 30% probability, then you are 50% sure that it has an expected value of 0.2 diamonds and 50% sure that it has an expected value of 0.3 diamonds, so it has an expected expected value of 0.25 diamonds. Why not just be 25% sure from the beginning?
Supposedly, David gave an example of meta-probability being necessary in the earlier post her references. However, using conditional probabilities give you the right answer. There is a difference between a gambling machine having independent 50% chances of giving out two coins when you put in one, and one that has a 50% chance the first time, but has a 100% chance of giving out two coins the nth time given that it did the first time and a 0% chance given it did not. Since there are times where you need conditional probabilities and meta-probabilities won’t suffice, you need to have conditional probabilities anyway, so why bother with meta-probabilities?
That’s not to say that meta-probabilities can’t be useful. If the probability of A depends on B, and all you care about is A, meta-probabilities will model this perfectly, and will be much simpler to use than conditional probabilities. A good example of a successful use of meta-probabilities is Student’s t-test, which can be thought of as a distribution of normal distributions, in which the standard deviation itself has a probability distribution.
The Bayesian Universalist answer to this would be that there is no separate meta-probability. You have a universal prior over all possible hypotheses, and mutter a bit about Solomonoff induction and AIXI.
I am putting it this way, distancing myself from the concept, because I don’t actually believe it, but it is the standard answer to draw out from the LessWrong meme space, and it has not yet been posted in this thread. Is there anyone who can make a better fist of expounding it?
Yes, I’m not at all committed to the metaprobability approach. In fact, I concocted the black box example specifically to show its limitations!
Solomonoff induction is extraordinarily unhelpful, I think… that it is uncomputable is only one reason.
I think there’s a fairly simple and straightforward strategy to address the black box problem, which has not been mentioned so far...
Because it’s output is not human-readable being the other?
I mean, even if I’ve got a TARDIS to use as a halting oracle, an Inductive Turing Machine isn’t going to output something I can actually use to make predictions about specific events such as “The black box gives you money under X, Y, and Z circumstances.”
Well, the problem I was thinking of is “the universe is not a bit string.” And any unbiased representation we can make of the universe as a bit string is going to be extremely large—much too large to do even sane sorts of computation with, never mind Solomonoff.
Maybe that’s saying the same thing you did? I’m not sure...
Can you please give us a top level post at some point, be it in Discussion or Main, arguing that “the universe is not a bit string”? I find that very interesting, relevant, and plausible.
Thanks for the encouragement! I have way too many half-completed writing projects, but this does seem an important point.
Going back to the basic question about the black box:
Too small to be worth considering. I might as well ask, what’s the probability that I’ll find $2 hidden half way up the nearest tree? Nothing has been claimed about the black box to specifically draw “it will pay you $2 for $1” out of hypothesis space.
Hmm… given that the previous several boxes have either paid $2 or done nothing, it seems like that primes the hypothesis that the next in the series also pays $2 or does nothing. (I’m not actually disagreeing, but doesn’t that argument seem reasonable?)
Priming a hypothesis merely draws it to attention; it does not make it more likely. Every piece of spam, every con game, “primes the hypothesis” that it is genuine. It also “primes the hypothesis” that it is not. “Priming the hypothesis” is no more evidence than a purple giraffe is evidence of the blackness of crows.
Explicltly avoiding saying that it does pay $2, and saying instead that it is “interesting”, well, that pretty much stomps the “priming” into a stain on the sidewalk.
.… purple giraffes are evidence of the blackness of crows, though. Just, really really terrible evidence.
Well, yes. As is the mere presence of the idea of $2 for $1 terrible evidence that the black box will do any such thing.
Eliezer speaks in the Twelve Virtues of letting oneself be as light as a leaf, blown unresistingly by the wind of evidence, but evidence of this sort is on the level of the individual molecules and Brownian motion of that leaf.
It depends on your priors
You can give a meta-probability if you want. However, this makes no difference in your final result. If you are 50% certain that a box has a diamond in it with 20% probability, and you are 50% certain that it has a diamond with 30% probability, then you are 50% sure that it has an expected value of 0.2 diamonds and 50% sure that it has an expected value of 0.3 diamonds, so it has an expected expected value of 0.25 diamonds. Why not just be 25% sure from the beginning?
Supposedly, David gave an example of meta-probability being necessary in the earlier post her references. However, using conditional probabilities give you the right answer. There is a difference between a gambling machine having independent 50% chances of giving out two coins when you put in one, and one that has a 50% chance the first time, but has a 100% chance of giving out two coins the nth time given that it did the first time and a 0% chance given it did not. Since there are times where you need conditional probabilities and meta-probabilities won’t suffice, you need to have conditional probabilities anyway, so why bother with meta-probabilities?
That’s not to say that meta-probabilities can’t be useful. If the probability of A depends on B, and all you care about is A, meta-probabilities will model this perfectly, and will be much simpler to use than conditional probabilities. A good example of a successful use of meta-probabilities is Student’s t-test, which can be thought of as a distribution of normal distributions, in which the standard deviation itself has a probability distribution.