this post seems incredibly dense and convoluted. I literally do not know what you’re talking about
That was not my experience. I understood everything in the first five paragraphs without having to reflect or even read a second time except that I did have to reflect for a few minutes on the last sentence of paragraph four. Although I am still less confident that I know what Justin intended there than I am with the other sentences, I am 72% confident I know. I think he meant that even if we are not religious, society tends to pull us into moral realism even though of course moral realism is an illusion. (Time constraints prevent me from reading the rest now.)
Defining “supergoal uncertainty” would be a necessary step
Oh, he did that. And the definition was quite clear to me on first reading, but then I have done a lot of math, and a lot of math in which I attempt my own definitions.
Well, I am relatively new at assigning my beliefs numerical probabilities, so if Eliezer or E.T. Jaynes says different, believe them, but here is my reply.
72% confident
Two sig figs? Really?
Note that if I had said .7 that does not mean that my probability will not go to.4 or .9 tomorrow. On the other hand, if I say the doo-dad is .7 meters long, I am implying that if I re-measure the doo-dad tomorrow, the result will be somewhere in the range .65 to .75 (or to .8). In summary, significant figures does not seem a worthwhile way to communicate how much evidence is required to move a probability by a certain amount. What I suggest people do instead is communicate somehow the nature of the evidence used to arrive at the number. In this case, I left implied that my evidence comes from squishy introspective considerations. Also, note that the fact that Justin will be checking frequently for comments (because it is his post) and Justin can very easily drive my probability to close to 1 or close to 0 with a reply that takes him only 10 seconds to make means that it does not serve the “vericidal” interests of the community for me to spend more than a few seconds in arriving at my numerical probability. I could have mentioned these considerations of the cost of updating my probability and the implications that cost structure has for how much effort I put into my number, but I considered them so obvious that the reader would take them into consideration without my having to say anything.
Look: there is a cost to the experimentalist’s tradition by which .7 means that tomorrow the number will not change to anything lower than .65 and higher than .75 or .7999 and that cost is that the only numbers available to the writer are .1 .2 .3 .4 .5 .6 .7 .8 .9. The previous paragraph explains why I consider that cost not worth paying for subjective probabilities.
Jaynes does have something to say on this, which I will summarize thus: you get to (ought to, even) put credible-interval-type bounds on a stated probability (that is, you could have said, e.g., “between 50% and 90%”). The central location of the interval tells us what you now think of your probability (~70%), and the width of the interval tells us how apt your estimate is to move in the face of new evidence.
The above is an approximation; there are lots of refinements. One I will mention right off is that the scheme will break down for probabilities near 0 or 1, because the implied distribution is no longer symmetric around the center of the interval.
Can you give a reference? Because that strikes me as rather un-Jaynesian.
You say that the interval tells us something about how apt the estimate is to move in the face of new evidence. What does it tell us about that? Doesn’t it depend on which piece of evidence we’re talking about? Do you have to specify a prior over which variables you are likely to observe next?
The material I have in mind is Chapter 18 of PT:LOS. You can see the section headings on page 8 (numbered vii because the title page is unnumbered) here. One of the section titles is “Outer and Inner Robots”; when rhollerith says 72%, he’s giving the outer robot answer. To give an account of how unstable your probability estimates are, you need to give the inner robot answer.
What does it tell us about that? Doesn’t it depend on which piece of evidence we’re talking about?
When we receive new evidence, we assign a likelihood function for the probability. (We take the perspective of the inner robot reasoning about what the outer robot will say.) The width of the interval for the probability tells us how narrow the likelihood function has to be to shift the center of that interval by a non-neglible amount.
Do you have to specify a prior over which variables you are likely to observe next?
That is a strange little chapter, but I should note that if you talk about the probability that you will make some future probability estimate, then the distribution of a future probability estimate does make a good way of talking about the instability of a state of knowledge. As opposed to the notion of talking about the probability of a current probability estimate, which sounds much more like you’re doing something wrong.
Second the question, it doesn’t sound Jaynesian to me either.
I’m relieved that I’m not the only one who thought that. I was somewhat aghast to hear Jaynes recommend something that is so, well, obviously a bull@# hack.
It’s curious to me that you’d write this even after I cited chapter and verse. Do you have a copy of PT:LOS?
I do have a copy but I will take your word for it. I am shocked and amazed that Jayenes would give such a poor recommendation. It doesn’t sound Jaynesian to me either and I rather hope he presents a variant that is sufficiently altered as to not be this suggestion at all. You yourself gave the reason why it doesn’t work and I am sure there is a better approach than just hacking the scale when it is near 1 or 0. (I am hoping your paraphrase sounds worse than the original.)
It is good to indicate the strength of your priors. Perhaps one could indicate how much you think your opinion is likely to change over some specified timescale—or in response to the next set of pertinent data points.
For significant figures to be at all applicable you would need to express confidence with a completely different kind of scale. I am not going to round off 96% to “not even a probability”.
For my part I find it irritating. But it would certainly work better for 1 significant figure expressions. Although I suppose you could say it kind of relies on two significant figures (one on either side) to work at all.
I think he meant that even if we are not religious, society tends to pull us into moral realism even though of course moral realism is an illusion.
You are correct, though I don’t go as far as calling moral realism an illusion because of unknown unknowns (though I would be very surprised to find it isn’t illusionary).
That was not my experience. I understood everything in the first five paragraphs without having to reflect or even read a second time except that I did have to reflect for a few minutes on the last sentence of paragraph four. Although I am still less confident that I know what Justin intended there than I am with the other sentences, I am 72% confident I know. I think he meant that even if we are not religious, society tends to pull us into moral realism even though of course moral realism is an illusion. (Time constraints prevent me from reading the rest now.)
Oh, he did that. And the definition was quite clear to me on first reading, but then I have done a lot of math, and a lot of math in which I attempt my own definitions.
Two sig figs? Really?
Well, I am relatively new at assigning my beliefs numerical probabilities, so if Eliezer or E.T. Jaynes says different, believe them, but here is my reply.
Note that if I had said .7 that does not mean that my probability will not go to.4 or .9 tomorrow. On the other hand, if I say the doo-dad is .7 meters long, I am implying that if I re-measure the doo-dad tomorrow, the result will be somewhere in the range .65 to .75 (or to .8). In summary, significant figures does not seem a worthwhile way to communicate how much evidence is required to move a probability by a certain amount. What I suggest people do instead is communicate somehow the nature of the evidence used to arrive at the number. In this case, I left implied that my evidence comes from squishy introspective considerations. Also, note that the fact that Justin will be checking frequently for comments (because it is his post) and Justin can very easily drive my probability to close to 1 or close to 0 with a reply that takes him only 10 seconds to make means that it does not serve the “vericidal” interests of the community for me to spend more than a few seconds in arriving at my numerical probability. I could have mentioned these considerations of the cost of updating my probability and the implications that cost structure has for how much effort I put into my number, but I considered them so obvious that the reader would take them into consideration without my having to say anything.
Look: there is a cost to the experimentalist’s tradition by which .7 means that tomorrow the number will not change to anything lower than .65 and higher than .75 or .7999 and that cost is that the only numbers available to the writer are .1 .2 .3 .4 .5 .6 .7 .8 .9. The previous paragraph explains why I consider that cost not worth paying for subjective probabilities.
Jaynes does have something to say on this, which I will summarize thus: you get to (ought to, even) put credible-interval-type bounds on a stated probability (that is, you could have said, e.g., “between 50% and 90%”). The central location of the interval tells us what you now think of your probability (~70%), and the width of the interval tells us how apt your estimate is to move in the face of new evidence.
The above is an approximation; there are lots of refinements. One I will mention right off is that the scheme will break down for probabilities near 0 or 1, because the implied distribution is no longer symmetric around the center of the interval.
Can you give a reference? Because that strikes me as rather un-Jaynesian.
You say that the interval tells us something about how apt the estimate is to move in the face of new evidence. What does it tell us about that? Doesn’t it depend on which piece of evidence we’re talking about? Do you have to specify a prior over which variables you are likely to observe next?
The material I have in mind is Chapter 18 of PT:LOS. You can see the section headings on page 8 (numbered vii because the title page is unnumbered) here. One of the section titles is “Outer and Inner Robots”; when rhollerith says 72%, he’s giving the outer robot answer. To give an account of how unstable your probability estimates are, you need to give the inner robot answer.
When we receive new evidence, we assign a likelihood function for the probability. (We take the perspective of the inner robot reasoning about what the outer robot will say.) The width of the interval for the probability tells us how narrow the likelihood function has to be to shift the center of that interval by a non-neglible amount.
No.
That is a strange little chapter, but I should note that if you talk about the probability that you will make some future probability estimate, then the distribution of a future probability estimate does make a good way of talking about the instability of a state of knowledge. As opposed to the notion of talking about the probability of a current probability estimate, which sounds much more like you’re doing something wrong.
Second the question, it doesn’t sound Jaynesian to me either.
I’m relieved that I’m not the only one who thought that. I was somewhat aghast to hear Jaynes recommend something that is so, well, obviously a bull@# hack.
It’s curious to me that you’d write this even after I cited chapter and verse. Do you have a copy of PT:LOS?
I do have a copy but I will take your word for it. I am shocked and amazed that Jayenes would give such a poor recommendation. It doesn’t sound Jaynesian to me either and I rather hope he presents a variant that is sufficiently altered as to not be this suggestion at all. You yourself gave the reason why it doesn’t work and I am sure there is a better approach than just hacking the scale when it is near 1 or 0. (I am hoping your paraphrase sounds worse than the original.)
Best to give a probabilty density function—but two 2-S-F probabilites typically gives more information than one.
It is good to indicate the strength of your priors. Perhaps one could indicate how much you think your opinion is likely to change over some specified timescale—or in response to the next set of pertinent data points.
For significant figures to be at all applicable you would need to express confidence with a completely different kind of scale. I am not going to round off 96% to “not even a probability”.
I like the odds scale, myself.
For my part I find it irritating. But it would certainly work better for 1 significant figure expressions. Although I suppose you could say it kind of relies on two significant figures (one on either side) to work at all.
You are correct, though I don’t go as far as calling moral realism an illusion because of unknown unknowns (though I would be very surprised to find it isn’t illusionary).
Uh, oh …
http://www.youtube.com/watch?v=jKdYlSrXn_4