One way to communicate about uncertainty is to provide explicit probabilities, e.g. “I think it’s 20% likely that [...]”, or “I would put > 90% probability on [...]”. Another way to communicate about uncertainty is to use words like “plausible”, “possible”, “definitely”, “likely”, e.g. “I think it is plausible that [...]”.
People seem to treat the words as shorthands for probability statements. I don’t know why you’d do this, it’s losing information and increasing miscommunication for basically no reason—it’s maybe slightly more idiomatic English, but it’s not even much longer to just put the number into the sentence! (And you don’t have to have precise numbers, you can have ranges or inequalities if you want, if that’s what you’re using the words to mean.)
According to me, probabilities are appropriate for making decisions so you can estimate the EV of different actions. (This can also extend to the case where you aren’t making a decision, but you’re talking to someone who might use your advice to make decisions, but isn’t going to understand your reasoning process.) In contrast, words are for describing the state of your reasoning algorithm, which often doesn’t have much to do with probabilities.
A simple model here is that your reasoning algorithm has two types of objects: first, strong constraints that rule out some possible worlds (“An AGI system that is widely deployed will have a massive impact”), and second, an implicit space of imagined scenarios in which each scenario feels similarly unsurprising if you observe it (this is the human emotion of surprise, not the probability-theoretic definition of surprise; one major difference is that the human emotion of surprise often doesn’t increase with additional conjuncts). Then, we can define the meanings of the words as follows:
Plausible: “This is within my space of imagined scenarios.”
Possible: “This isn’t within my space of imagined scenarios, but it doesn’t violate any of my strong constraints.” (Though unfortunately it can also mean “this violates a strong constraint, but I recognize that my constraint may be wrong, so I don’t assign literally zero probability to it”.)
Definitely: “If this weren’t true, that would violate one of my strong constraints.”
Implausible: “This violates one of my strong constraints.” (Given other definitions, this should really be called “impossible”, but unfortunately that word is unavailable (though I think Eliezer uses it this way anyway).)
When someone gives you an example scenario, it’s much easier to label it with one of the four words above, because their meanings are much much closer to the native way in which brains reason (at least, for my brain). That’s why I end up using these words rather than always talking in probabilities. To convert these into actual probabilities, I then have to do some additional reasoning in order to take into account things like model uncertainty, epistemic deference, conjunctive fallacy, estimating the “size” of the space of imagined scenarios to figure out the average probability for any given possibility, etc.
I like this, but it feels awkward to say that something can be not inside a space of “possibilities” but still be “possible”. Maybe “possibilities” here should be “imagined scenarios”?
One way to communicate about uncertainty is to provide explicit probabilities, e.g. “I think it’s 20% likely that [...]”, or “I would put > 90% probability on [...]”. Another way to communicate about uncertainty is to use words like “plausible”, “possible”, “definitely”, “likely”, e.g. “I think it is plausible that [...]”.
People seem to treat the words as shorthands for probability statements. I don’t know why you’d do this, it’s losing information and increasing miscommunication for basically no reason—it’s maybe slightly more idiomatic English, but it’s not even much longer to just put the number into the sentence! (And you don’t have to have precise numbers, you can have ranges or inequalities if you want, if that’s what you’re using the words to mean.)
According to me, probabilities are appropriate for making decisions so you can estimate the EV of different actions. (This can also extend to the case where you aren’t making a decision, but you’re talking to someone who might use your advice to make decisions, but isn’t going to understand your reasoning process.) In contrast, words are for describing the state of your reasoning algorithm, which often doesn’t have much to do with probabilities.
A simple model here is that your reasoning algorithm has two types of objects: first, strong constraints that rule out some possible worlds (“An AGI system that is widely deployed will have a massive impact”), and second, an implicit space of imagined scenarios in which each scenario feels similarly unsurprising if you observe it (this is the human emotion of surprise, not the probability-theoretic definition of surprise; one major difference is that the human emotion of surprise often doesn’t increase with additional conjuncts). Then, we can define the meanings of the words as follows:
Plausible: “This is within my space of imagined scenarios.”
Possible: “This isn’t within my space of imagined scenarios, but it doesn’t violate any of my strong constraints.” (Though unfortunately it can also mean “this violates a strong constraint, but I recognize that my constraint may be wrong, so I don’t assign literally zero probability to it”.)
Definitely: “If this weren’t true, that would violate one of my strong constraints.”
Implausible: “This violates one of my strong constraints.” (Given other definitions, this should really be called “impossible”, but unfortunately that word is unavailable (though I think Eliezer uses it this way anyway).)
When someone gives you an example scenario, it’s much easier to label it with one of the four words above, because their meanings are much much closer to the native way in which brains reason (at least, for my brain). That’s why I end up using these words rather than always talking in probabilities. To convert these into actual probabilities, I then have to do some additional reasoning in order to take into account things like model uncertainty, epistemic deference, conjunctive fallacy, estimating the “size” of the space of imagined scenarios to figure out the average probability for any given possibility, etc.
I like this, but it feels awkward to say that something can be not inside a space of “possibilities” but still be “possible”. Maybe “possibilities” here should be “imagined scenarios”?
That does seem like better terminology! I’ll go change it now.