(a comment I made in another forum while discussing my recent post proposing more consistent terminology for probability ranges)
I think there’s a ton of work still to be done across the sciences (and to some extent other disciplines) in figuring out how to communicate evidence and certainty and agreement. My go-to example is: when your weather app says there’s a 30% chance of rain tomorrow, it’s really non-obvious to most people what that means. Some things it could mean:
We have 30% confidence that it will rain on you tomorrow.
We are entirely confident that there is an irreducible 30% chance that it will rain tomorrow.
30% of this area will get rain tomorrow.
It will be raining 30% of the day tomorrow.
30% of our models say it will rain tomorrow.
30% of the separate runs of our model say it will rain tomorrow [this is actually the typical meaning IIRC, but wow is that non-obvious].
Our model says it will definitely rain tomorrow, and it has been accurate on 70% of previous days.
Our new model says it will definitely rain tomorrow, and 70% of the meteorologists in our office think it’s right.
Our latest model says it will definitely rain tomorrow but we have Knightian uncertainty about the validity of the new model which we’ve chosen to represent by giving the model 70% credence.
Probably quite a few others that I’m not thinking of at the moment? And of course these aren’t all independent; in most real-world cases many of these sources of uncertainty are simultaneously in play.
And that’s not even starting to touch on communicating variance / standard deviation / confidence intervals.I used to work as a software engineer in climatology, and got really interested in data visualization, and spent a lot of time struggling with how to try to convey all this without swamping people who may really just want a one-bit answer about whether they should bring their umbrella to work tomorrow.
Is there an existing body of work on this? If so I’d love to know about it!
(a comment I made in another forum while discussing my recent post proposing more consistent terminology for probability ranges)
I think there’s a ton of work still to be done across the sciences (and to some extent other disciplines) in figuring out how to communicate evidence and certainty and agreement. My go-to example is: when your weather app says there’s a 30% chance of rain tomorrow, it’s really non-obvious to most people what that means. Some things it could mean:
We have 30% confidence that it will rain on you tomorrow.
We are entirely confident that there is an irreducible 30% chance that it will rain tomorrow.
30% of this area will get rain tomorrow.
It will be raining 30% of the day tomorrow.
30% of our models say it will rain tomorrow.
30% of the separate runs of our model say it will rain tomorrow [this is actually the typical meaning IIRC, but wow is that non-obvious].
Our model says it will definitely rain tomorrow, and it has been accurate on 70% of previous days.
Our new model says it will definitely rain tomorrow, and 70% of the meteorologists in our office think it’s right.
Our latest model says it will definitely rain tomorrow but we have Knightian uncertainty about the validity of the new model which we’ve chosen to represent by giving the model 70% credence.
Probably quite a few others that I’m not thinking of at the moment? And of course these aren’t all independent; in most real-world cases many of these sources of uncertainty are simultaneously in play.
And that’s not even starting to touch on communicating variance / standard deviation / confidence intervals.I used to work as a software engineer in climatology, and got really interested in data visualization, and spent a lot of time struggling with how to try to convey all this without swamping people who may really just want a one-bit answer about whether they should bring their umbrella to work tomorrow.
Is there an existing body of work on this? If so I’d love to know about it!