Any time you get a data point about X, you get to update both on X and on the process that generated the data point. If you get several data points in a row, then as your view of the data-generating process changes you have re-evaluate all of the data it gave it you earlier. Examples:
If somebody gives me a strong-sounding argument for X and several weak-sounding arguments for X, I’m usually less persuaded than if I just heard a strong-sounding argument for X. The weak-sounding arguments are evidence that the person I’m talking to can’t evaluate arguments well, so it’s relatively more likely that the strong-sounding argument has a flaw that I just haven’t spotted.
If somebody gives me a strong-sounding argument for X and several reasonable-but-not-as-strong arguments against X, I’m more persuaded than just by the strong argument for X. This is because the arguments against X are evidence that the data-generating process isn’t filtered (there’s an old Zack_M_Davis post about this but I can’t find it). But this only works to the extent that the arguments against X seem like real arguments and not strawmen: weak-enough arguments against X make me less persuaded again, because they’re evidence of a deceptive data-generating process.
If I know someone wants to persuade me of X, I mostly update less on their arguments than I would if they were indifferent, because I expect them to filter and misrepresent the data (but this one is tricky: sometimes the strong arguments are hard to find, and only the enthusiasts will bother).
If I hear many arguments for X that seem very similar I don’t update very much after the first one, since I suspect that all the arguments are secretly correlated.
On social media the strongest evidence is often false, because false claims can be better optimized for virality. If I hear lots of different data points of similar strength, I’ll update more strongly on each individual data point.
None of this is cheap to compute; there are a bunch of subtle, clashing considerations. So if we don’t have a lot of time, should we use the sum, or the average, or what? Equivalently: what prior should we have over data-generating processes? Here’s how I think about it:
Sum: Use this when you think your data points are independent, and not filtered in any particular way—or if you think you can precisely account for conditional dependence, selection, and so on. Ideal, but sometimes impractical and too expensive to use all the time.
Max: Useful when your main concern is noise. Probably what I use the most in my ordinary life. The idea is that most of the data I get doesn’t pertain to X at all, and the data that is about X is both subject to large random distortions and probably secretly correlated in a way that I can’t quantify very well. Nevertheless, if X is true you should expect to see signs of it, here and there, and tracking the max leaves you open to that evidence without having to worry about double-updating. As a bonus, it’s very memory efficient: you only have to remember the strongest data favoring X and the strongest data disfavoring it, and can forget all the rest.
Average: What I use when I’m evaluating an attempt at persuasion from someone I don’t know well. Averaging is a lousy way to evaluate arguments but a pretty-good-for-how-cheap-it-is way to evaluate argument-generating processes. Data points that aren’t arguments probably shouldn’t ever be averaged.
Min: I don’t think this one has any legitimate use at all. Lots of data points are only very weakly about X, even when X is true.
All of these heuristics have cases where they abjectly fail, and none of them work well when your adversary is smarter than you are.
Any time you get a data point about X, you get to update both on X and on the process that generated the data point. If you get several data points in a row, then as your view of the data-generating process changes you have re-evaluate all of the data it gave it you earlier. Examples:
If somebody gives me a strong-sounding argument for X and several weak-sounding arguments for X, I’m usually less persuaded than if I just heard a strong-sounding argument for X. The weak-sounding arguments are evidence that the person I’m talking to can’t evaluate arguments well, so it’s relatively more likely that the strong-sounding argument has a flaw that I just haven’t spotted.
If somebody gives me a strong-sounding argument for X and several reasonable-but-not-as-strong arguments against X, I’m more persuaded than just by the strong argument for X. This is because the arguments against X are evidence that the data-generating process isn’t filtered (there’s an old Zack_M_Davis post about this but I can’t find it). But this only works to the extent that the arguments against X seem like real arguments and not strawmen: weak-enough arguments against X make me less persuaded again, because they’re evidence of a deceptive data-generating process.
If I know someone wants to persuade me of X, I mostly update less on their arguments than I would if they were indifferent, because I expect them to filter and misrepresent the data (but this one is tricky: sometimes the strong arguments are hard to find, and only the enthusiasts will bother).
If I hear many arguments for X that seem very similar I don’t update very much after the first one, since I suspect that all the arguments are secretly correlated.
On social media the strongest evidence is often false, because false claims can be better optimized for virality. If I hear lots of different data points of similar strength, I’ll update more strongly on each individual data point.
None of this is cheap to compute; there are a bunch of subtle, clashing considerations. So if we don’t have a lot of time, should we use the sum, or the average, or what? Equivalently: what prior should we have over data-generating processes? Here’s how I think about it:
Sum: Use this when you think your data points are independent, and not filtered in any particular way—or if you think you can precisely account for conditional dependence, selection, and so on. Ideal, but sometimes impractical and too expensive to use all the time.
Max: Useful when your main concern is noise. Probably what I use the most in my ordinary life. The idea is that most of the data I get doesn’t pertain to X at all, and the data that is about X is both subject to large random distortions and probably secretly correlated in a way that I can’t quantify very well. Nevertheless, if X is true you should expect to see signs of it, here and there, and tracking the max leaves you open to that evidence without having to worry about double-updating. As a bonus, it’s very memory efficient: you only have to remember the strongest data favoring X and the strongest data disfavoring it, and can forget all the rest.
Average: What I use when I’m evaluating an attempt at persuasion from someone I don’t know well. Averaging is a lousy way to evaluate arguments but a pretty-good-for-how-cheap-it-is way to evaluate argument-generating processes. Data points that aren’t arguments probably shouldn’t ever be averaged.
Min: I don’t think this one has any legitimate use at all. Lots of data points are only very weakly about X, even when X is true.
All of these heuristics have cases where they abjectly fail, and none of them work well when your adversary is smarter than you are.