“The beginning of wisdom is the definition of terms.” - (attributed to) Socrates
I’d argue that for any conversation between people to make progress, they have to have some agreeance on what they’re talking about. A counter example, if I were to order a 1/4“ bolt from someone and we have different measurement standards of what 1/4” is, I won’t be able to build on what I’ve received from them. Consistency is the key characteristic that allows interoperability between multiple parties.
I appreciate greatly that Gwern uses confidence tags to convey this aspect, as it gives me a relative sense of how much I can trust a piece of evidence.
Maybe most important, I’d also argue that even if standardized usage is only used throughout the AIS community, that would be worthwhile in itself. Living in the Bay and participating in a lot of doom conversations, I believe a lot of nuance is missed and unjustly has people updating priors when having a standardized set of terms would allow people to notice when something seems off about another person’s claim.
That said, even if this wouldn’t help with informal discussions within the community, having more discrete terms to share would allow for higher quality information transfer between researchers, similar to the 1/4″ bolt example above.
Oh, I hadn’t seen Gwern’s confidence tags. There’s an interesting difference between the Kesselman list that Gwern draws from (screenshotted below) and my proposal above. The Kesselman terms refer to non-overlapping, roughly equal ranges, whereas the climatology usage I give above describe overlapping ranges of different sizes, with one end of the range at 0 or 100%.
Both are certainly useful. I think I’d personally lean toward the climatology version as the better one to adopt, especially for talking about risk; I think in general researchers are less likely to want to say that something is 56-70% likely than that it is 66-100% likely, ie that it’s at least 66% likely. If I were trying to identify a relatively narrow specific range, I think I’d be more inclined to just give an actual approximate number. The climatology version, it seems to me is more primarily for indicating a level of uncertainty. When saying something is ‘likely’ (66-100%) vs ‘very likely’, in both cases you’re saying that you think something will probably happen, but you’re saying different things about your confidence level.
(@gwern I’d love to get your take on that question as well, since you settled on the Kesselman list)
I’ve just posted a quick take getting a bit further into the range of possible things that probabilities can mean; our ways of communicating about probabilities and probability ranges seem hideously inadequate to me.
“The beginning of wisdom is the definition of terms.” - (attributed to) Socrates
I’d argue that for any conversation between people to make progress, they have to have some agreeance on what they’re talking about. A counter example, if I were to order a 1/4“ bolt from someone and we have different measurement standards of what 1/4” is, I won’t be able to build on what I’ve received from them. Consistency is the key characteristic that allows interoperability between multiple parties.
I appreciate greatly that Gwern uses confidence tags to convey this aspect, as it gives me a relative sense of how much I can trust a piece of evidence.
Maybe most important, I’d also argue that even if standardized usage is only used throughout the AIS community, that would be worthwhile in itself. Living in the Bay and participating in a lot of doom conversations, I believe a lot of nuance is missed and unjustly has people updating priors when having a standardized set of terms would allow people to notice when something seems off about another person’s claim.
That said, even if this wouldn’t help with informal discussions within the community, having more discrete terms to share would allow for higher quality information transfer between researchers, similar to the 1/4″ bolt example above.
Oh, I hadn’t seen Gwern’s confidence tags. There’s an interesting difference between the Kesselman list that Gwern draws from (screenshotted below) and my proposal above. The Kesselman terms refer to non-overlapping, roughly equal ranges, whereas the climatology usage I give above describe overlapping ranges of different sizes, with one end of the range at 0 or 100%.
Both are certainly useful. I think I’d personally lean toward the climatology version as the better one to adopt, especially for talking about risk; I think in general researchers are less likely to want to say that something is 56-70% likely than that it is 66-100% likely, ie that it’s at least 66% likely. If I were trying to identify a relatively narrow specific range, I think I’d be more inclined to just give an actual approximate number. The climatology version, it seems to me is more primarily for indicating a level of uncertainty. When saying something is ‘likely’ (66-100%) vs ‘very likely’, in both cases you’re saying that you think something will probably happen, but you’re saying different things about your confidence level.
(@gwern I’d love to get your take on that question as well, since you settled on the Kesselman list)
I’ve just posted a quick take getting a bit further into the range of possible things that probabilities can mean; our ways of communicating about probabilities and probability ranges seem hideously inadequate to me.