This post seems to me to be misunderstanding a major piece of Paul’s “sluggish updating” post, and clashing with Paul’s post in ways that aren’t explicit.
The core of Paul’s post, as I understood it, is that incentive landscapes often reward people for changing their stated views too gradually in response to new arguments/evidence, and Paul thinks he has often observed this behavioral pattern which he called “sluggish updating.” Paul illustrated this incentive landscape through a story involving Alice and Bob, where Bob is thinking through his optimal strategy, since that’s a convenient way to describe incentive landscapes. But that kind of intentional strategic thinking isn’t how the incentives typically manifest themselves in behavior, in Paul’s view (e.g., “I expect this to result in unconscious bias rather than conscious misrepresentation. I suspect this incentive significantly distorts the beliefs of many reasonable people on important questions”). This post by Zvi misunderstands this as Paul describing the processes that go on inside the heads of actual Bobs. This loses track of the important distinction (which is the subject of multiple other LW Review nominees) between the rewards that shape an agent’s behavior and the agent’s intentions. It also sweeps much of the disagreement between Paul & Zvi’s posts under the rug.
A few related ways the views in the two posts clash:
This post by Zvi focuses on dishonesty, while Paul suggests that unconsciously distorted beliefs are the typical case. This could be because Zvi disagrees with Paul and thinks that dishonesty is the typical case. Or it could be that Zvi is using the word “dishonest” broadly—he mostly agrees with Paul about what happens in people’s heads, but applies the “dishonesty” frame in places where Paul wouldn’t. Or maybe Zvi is just choosing to focus on the dishonest subset of cases. Or some combination of these.
Zvi focuses on cases where Bob is going to the extreme in following these incentives, optimizing heavily for it and propagating it into his thinking. “This is a world where all one cares about is how one is evaluated, and lying and deceiving others is free as long as you’re not caught.” “Bob’s optimal strategy is full anti-epistemology.” Paul seems especially interested in cases where pretty reasonable people (with some pretty good features in their epistemics, motivations, and incentives) still sometimes succumb to these incentives for sluggishness. Again, it’s unclear how much of this is due to Zvi & Paul having different beliefs about the typical case and how much is about choosing to focus on different subsets of cases (or which cases to treats as central for model-building).
Paul’s post is written from a perspective of ‘Good epistemics don’t happen by default’, where thinking well as an individual involves noticing places where your mental processes haven’t been aimed towards accurate beliefs and trying to do better, and social epistemics are an extension of that at the group level. Zvi’s post is written from a perspective of ‘catching cheaters’, where good social epistemics is about noticing ways that people are something-like-lying to you, and trying to stop that from happening.
Zvi treats Bob as an adversary. Paul treats him as a potential ally (or as a state that you or I or anyone could find oneself in), and mentions “gaining awareness” of the sluggishness as one way for an individual to counter it.
Related to all of this, the terminology clashes (as I mentioned in a comment). I’d like to say a simple sentence like “Paul sees [?sluggishness?] as mainly due to [?unconscious processes?], Zvi as mainly due to [?dishonest update reporting?]” but I’m not sure what terms go in the blanks.
The “fire Bob” recommendation depends a lot on how you’re looking at the problem space / which part of the problem space you’re looking at. If it’s just a recommendation for a narrow set of cases then I think it wouldn’t apply to most of the cases that Paul was talking about in his “Observations in the wild”, but if it’s meant to apply more widely then that could get messy in ways that interact with the clashes I’ve described.
The other proposed solutions seem less central to these two posts, and to the clash between Paul & Zvi’s perspectives.
I think there is something interesting in the contrast between Paul & Zvi’s perspectives, but this post didn’t work as a way to shine light on that contrast. It focuses on a different part of the problem space, while bringing in bits from Paul’s post in ways that make it seem like it’s engaging with Paul’s perspective more than it actually does and make it confusing to look at both perspectives side by side.
This post seems to me to be misunderstanding a major piece of Paul’s “sluggish updating” post, and clashing with Paul’s post in ways that aren’t explicit.
The core of Paul’s post, as I understood it, is that incentive landscapes often reward people for changing their stated views too gradually in response to new arguments/evidence, and Paul thinks he has often observed this behavioral pattern which he called “sluggish updating.” Paul illustrated this incentive landscape through a story involving Alice and Bob, where Bob is thinking through his optimal strategy, since that’s a convenient way to describe incentive landscapes. But that kind of intentional strategic thinking isn’t how the incentives typically manifest themselves in behavior, in Paul’s view (e.g., “I expect this to result in unconscious bias rather than conscious misrepresentation. I suspect this incentive significantly distorts the beliefs of many reasonable people on important questions”). This post by Zvi misunderstands this as Paul describing the processes that go on inside the heads of actual Bobs. This loses track of the important distinction (which is the subject of multiple other LW Review nominees) between the rewards that shape an agent’s behavior and the agent’s intentions. It also sweeps much of the disagreement between Paul & Zvi’s posts under the rug.
A few related ways the views in the two posts clash:
This post by Zvi focuses on dishonesty, while Paul suggests that unconsciously distorted beliefs are the typical case. This could be because Zvi disagrees with Paul and thinks that dishonesty is the typical case. Or it could be that Zvi is using the word “dishonest” broadly—he mostly agrees with Paul about what happens in people’s heads, but applies the “dishonesty” frame in places where Paul wouldn’t. Or maybe Zvi is just choosing to focus on the dishonest subset of cases. Or some combination of these.
Zvi focuses on cases where Bob is going to the extreme in following these incentives, optimizing heavily for it and propagating it into his thinking. “This is a world where all one cares about is how one is evaluated, and lying and deceiving others is free as long as you’re not caught.” “Bob’s optimal strategy is full anti-epistemology.” Paul seems especially interested in cases where pretty reasonable people (with some pretty good features in their epistemics, motivations, and incentives) still sometimes succumb to these incentives for sluggishness. Again, it’s unclear how much of this is due to Zvi & Paul having different beliefs about the typical case and how much is about choosing to focus on different subsets of cases (or which cases to treats as central for model-building).
Paul’s post is written from a perspective of ‘Good epistemics don’t happen by default’, where thinking well as an individual involves noticing places where your mental processes haven’t been aimed towards accurate beliefs and trying to do better, and social epistemics are an extension of that at the group level. Zvi’s post is written from a perspective of ‘catching cheaters’, where good social epistemics is about noticing ways that people are something-like-lying to you, and trying to stop that from happening.
Zvi treats Bob as an adversary. Paul treats him as a potential ally (or as a state that you or I or anyone could find oneself in), and mentions “gaining awareness” of the sluggishness as one way for an individual to counter it.
Related to all of this, the terminology clashes (as I mentioned in a comment). I’d like to say a simple sentence like “Paul sees [?sluggishness?] as mainly due to [?unconscious processes?], Zvi as mainly due to [?dishonest update reporting?]” but I’m not sure what terms go in the blanks.
The “fire Bob” recommendation depends a lot on how you’re looking at the problem space / which part of the problem space you’re looking at. If it’s just a recommendation for a narrow set of cases then I think it wouldn’t apply to most of the cases that Paul was talking about in his “Observations in the wild”, but if it’s meant to apply more widely then that could get messy in ways that interact with the clashes I’ve described.
The other proposed solutions seem less central to these two posts, and to the clash between Paul & Zvi’s perspectives.
I think there is something interesting in the contrast between Paul & Zvi’s perspectives, but this post didn’t work as a way to shine light on that contrast. It focuses on a different part of the problem space, while bringing in bits from Paul’s post in ways that make it seem like it’s engaging with Paul’s perspective more than it actually does and make it confusing to look at both perspectives side by side.