One difficulty is that such models aren’t mutually exclusive, there’s a lot of overlap. Another is that “correct” is very hard to define for elements that don’t have reportable experiential predictions. And even if we do suppose some resolution to the wager (which is what gives “probability” it’s meaning), many models could be partly-correct (either in some of their elements, or in some contexts but not universal).
Great points, there will be a vast cloud of models that aren’t mutually exclusive, and this base rate currently fails to capture them. We would have to somehow expand it.
Also definitely true on how models can be slightly correct, mostly-correct, etc.
I would say the wording that the resolution to the wager is “what gives the probability its meaning” is not entirely correct, though I am highly sympathetic to that sentiment. Suppose you have a shoebox in your room, and in the shoebox is a dice. (One dice, I’m pushing for the word “die” to die). You shake the shoebox to roll it, carefully take a picture of what it rolled, and preserve that somewhere, but I never see the dice, box, nor the picture, and you never tell me the result.
I can still have a forecast for the dice roll. Though, there are a lot more uncertainties than normal. For example I can’t be sure how many sides it even had, could’ve been one of those Dungeons and Dragons dice for all I know. In fact it could be a 3D-printed dice with a near-arbitrary amount of sides. I’d have to have a forecast distribution for the number of sides the dice had. And that distribution would be much wider than most people would guess. Though, I can confidently say it’s got fewer sides than the number TREE(3). I’d also have to forecast the odds you’re lying about how many dice are in there, or lying some other way I haven’t thought of.
In the end there is some remnant of a meaningful forecast process to be made. Just as if I was participating in a forecast on AI or on COVID. My true prior could be that n is somewhere from one to TREE(3) or whatever, and I slim it down somewhat. But there are two major distinctions:
1) I’ll be deprived the information of the resolution.
2) I’m not able to whittle the estimate down nearly as much as other domains. I will end up with a cloud of estimates whose 25th%ile and 75th%ile span many orders of magnitude. This is uncomfortable to work with, but I don’t see why I wouldn’t have some sort of base rate for the next metaphysical idea I hear. (If I felt like bothering to, of course.)
If such an estimate is meaningless, then the way I hear many smart people talk about the subject is hyper-meaningless. They’re debating the merits of specific numbers that could land in the shoebox. They’ll even get very invested in some of them, building whole religions (or in EA perhaps “proto-religion” to be more fair) around them. They do this without even bothering to estimate the number of sides it could have, number of dice, or anything else. Smart people don’t agree with me yet that the forecasting process is of some relevance to this domain, but they probably will some day, except for the ones that have a religion.
You can forecast all you want, but there is no “correctness” to that forecast. It requires some informational path from event to experience for a probability to have meaning. That does include telling the forecast to someone who observed (even indirectly) the outcome and seeing if they laugh at you, but does not include making up a number and then forgetting about it forever.
So to help me understand your position, how do you feel comparatively when someone like Bostrom says there’s, for example, maybe a 50% chance we’re in a simulation? (More egregiously, Elon saying there’s a one in a billion chance we’re not in a simulation!).
I think they are both perfectly reasonable statements about the models they prefer to imagine. They’re using probability terminology to give a sense of how much they like the model, not as any prediction—there’s no experience that will differ whether it’s true or false.
Probability of being in a simulation doesn’t make sense without clarifying what that means for the same reasons as probability in Sleeping Beauty. In the decision-relevant sense you need to ask what you’d care about affecting, since your decisions affect both real and simulated instances.
Virtually all forecasting has varying degrees of a risk the prediction resolves “ambiguous”. That risk reduces the informativeness. While I can’t say what exactly does or does not count as us being “in a simulation”, there’s also no particular reason I can’t put a probability on it. In the vast semantic cloud of possible interpretations, most of which is not visible to me, I have some nonzero information about what isn’t a simulation, and I know a simulation-promoter has shifted probability away from those other things. E.g. I know they are saying it’s not just WYSIWYG. It’s not much, but it’s also nonzero.
I also have placed many predictions on things that I will never see the resolution of, even if they are well-defined. Things that could not possibly affect anything to do with me.
I would wholeheartedly endorse an economic argument that such predictions are of too little tangible value to us. I do not endorse the idea that you fundamentally can’t have a probability attached. In fact it’s remarkably difficult for that to be entirely true, once actual numbers are used and extremely small amounts of information or confidence are a thing.
While I can’t say what exactly does or does not count as us being “in a simulation”, there’s also no particular reason I can’t put a probability on it.
Well, I quoted Sleeping Beauty as a particular illustration for why you’d put different probabilities on something depending on what you require, and that must be more specific than “a probability”. This is not a situation where you “can’t have a probability attached”, but illustrates that asking for “a probability” is occasionally not specific enough a question to be meaningful.
I would agree that models are generally useful as ML demonstrates, even if it’s unclear what they are saying, but in such cases interpreting them as hypotheses that give probabilities to events can be misleading, especially when there is no way of extracting these probabilities out of the models, or no clear way of formulating the events we’d be interested in. Instead, you have an error function, and you found models that have low error for the dataset, and these models make things better than the models with greater error. That doesn’t always have to be coerced into the language of probability.
One difficulty is that such models aren’t mutually exclusive, there’s a lot of overlap. Another is that “correct” is very hard to define for elements that don’t have reportable experiential predictions. And even if we do suppose some resolution to the wager (which is what gives “probability” it’s meaning), many models could be partly-correct (either in some of their elements, or in some contexts but not universal).
Great points, there will be a vast cloud of models that aren’t mutually exclusive, and this base rate currently fails to capture them. We would have to somehow expand it.
Also definitely true on how models can be slightly correct, mostly-correct, etc.
I would say the wording that the resolution to the wager is “what gives the probability its meaning” is not entirely correct, though I am highly sympathetic to that sentiment. Suppose you have a shoebox in your room, and in the shoebox is a dice. (One dice, I’m pushing for the word “die” to die). You shake the shoebox to roll it, carefully take a picture of what it rolled, and preserve that somewhere, but I never see the dice, box, nor the picture, and you never tell me the result.
I can still have a forecast for the dice roll. Though, there are a lot more uncertainties than normal. For example I can’t be sure how many sides it even had, could’ve been one of those Dungeons and Dragons dice for all I know. In fact it could be a 3D-printed dice with a near-arbitrary amount of sides. I’d have to have a forecast distribution for the number of sides the dice had. And that distribution would be much wider than most people would guess. Though, I can confidently say it’s got fewer sides than the number TREE(3). I’d also have to forecast the odds you’re lying about how many dice are in there, or lying some other way I haven’t thought of.
In the end there is some remnant of a meaningful forecast process to be made. Just as if I was participating in a forecast on AI or on COVID. My true prior could be that n is somewhere from one to TREE(3) or whatever, and I slim it down somewhat. But there are two major distinctions:
1) I’ll be deprived the information of the resolution.
2) I’m not able to whittle the estimate down nearly as much as other domains. I will end up with a cloud of estimates whose 25th%ile and 75th%ile span many orders of magnitude. This is uncomfortable to work with, but I don’t see why I wouldn’t have some sort of base rate for the next metaphysical idea I hear. (If I felt like bothering to, of course.)
If such an estimate is meaningless, then the way I hear many smart people talk about the subject is hyper-meaningless. They’re debating the merits of specific numbers that could land in the shoebox. They’ll even get very invested in some of them, building whole religions (or in EA perhaps “proto-religion” to be more fair) around them. They do this without even bothering to estimate the number of sides it could have, number of dice, or anything else. Smart people don’t agree with me yet that the forecasting process is of some relevance to this domain, but they probably will some day, except for the ones that have a religion.
You can forecast all you want, but there is no “correctness” to that forecast. It requires some informational path from event to experience for a probability to have meaning. That does include telling the forecast to someone who observed (even indirectly) the outcome and seeing if they laugh at you, but does not include making up a number and then forgetting about it forever.
So to help me understand your position, how do you feel comparatively when someone like Bostrom says there’s, for example, maybe a 50% chance we’re in a simulation? (More egregiously, Elon saying there’s a one in a billion chance we’re not in a simulation!).
I think they are both perfectly reasonable statements about the models they prefer to imagine. They’re using probability terminology to give a sense of how much they like the model, not as any prediction—there’s no experience that will differ whether it’s true or false.
Probability of being in a simulation doesn’t make sense without clarifying what that means for the same reasons as probability in Sleeping Beauty. In the decision-relevant sense you need to ask what you’d care about affecting, since your decisions affect both real and simulated instances.
Virtually all forecasting has varying degrees of a risk the prediction resolves “ambiguous”. That risk reduces the informativeness. While I can’t say what exactly does or does not count as us being “in a simulation”, there’s also no particular reason I can’t put a probability on it. In the vast semantic cloud of possible interpretations, most of which is not visible to me, I have some nonzero information about what isn’t a simulation, and I know a simulation-promoter has shifted probability away from those other things. E.g. I know they are saying it’s not just WYSIWYG. It’s not much, but it’s also nonzero.
I also have placed many predictions on things that I will never see the resolution of, even if they are well-defined. Things that could not possibly affect anything to do with me.
I would wholeheartedly endorse an economic argument that such predictions are of too little tangible value to us. I do not endorse the idea that you fundamentally can’t have a probability attached. In fact it’s remarkably difficult for that to be entirely true, once actual numbers are used and extremely small amounts of information or confidence are a thing.
Well, I quoted Sleeping Beauty as a particular illustration for why you’d put different probabilities on something depending on what you require, and that must be more specific than “a probability”. This is not a situation where you “can’t have a probability attached”, but illustrates that asking for “a probability” is occasionally not specific enough a question to be meaningful.
I would agree that models are generally useful as ML demonstrates, even if it’s unclear what they are saying, but in such cases interpreting them as hypotheses that give probabilities to events can be misleading, especially when there is no way of extracting these probabilities out of the models, or no clear way of formulating the events we’d be interested in. Instead, you have an error function, and you found models that have low error for the dataset, and these models make things better than the models with greater error. That doesn’t always have to be coerced into the language of probability.