I found this pretty useful—Zvi’s definitely reflecting a particular, pretty negative view of society and strategy here. But I disagree with some of your inferences, and I think you’re somewhat exaggerating the level of gloom-and-doom implicit in the post.
>Implication: “judge” means to use information against someone. Linguistic norms related to the word “judgment” are thoroughly corrupt enough that it’s worth ceding to these, linguistically, and using “judge” to mean (usually unjustly!) using information against people.
No, this isn’t bare repetition. I agree with Raemon that “judge” here means something closer to one of its standard usages, “to make inferences about”. Though it also fits with the colloquial “deem unworthy for baring [understandable] flaws”, which is also a thing that would happen with blackmail and could be bad.
>Implication: more generally available information about what strategies people are using helps “our” enemies more than it helps “us”. (This seems false to me, for notions of “us” that I usually use in strategy)
I can imagine a couple things going on here? One, if the world is a place where may more vulnerabilities are more known, this incentivizes more people to specialize in exploiting those vulnerabilities. Two, as a flawed human there are probably some stressors against which you can’t credibly play the “won’t negotiate with terrorists” card.
>Implication: even in the most just possible system of norms, it would be good to sometimes violate those norms and hide the fact that you violated them. (This seems incorrect to me!)
I think the assumption is these are ~baseline humans we’re talking about, and most human brains can’t hold norms of sufficient sophistication to capture true ethical law, and are also biased in ways that will sometimes strain against reflectively-endorsed ethics (e.g. they’re prone to using constrained circles of moral concern rather than universality).
>Implication: the bad guys won; we have rule by gangsters, who aren’t concerned with sustainable production, and just take as much stuff as possible in the short term. (This seems on the right track but partially false; the top marginal tax rate isn’t 100%)
This part of the post reminded me of (the SSC review of) Seeing Like a State, which makes a similar point; surveying and ‘rationalizing’ farmland, taking a census, etc. = legibility = taxability. “all of them” does seem like hyperbole here. I guess you can imagine the maximally inconvenient case where motivated people with low cost of time and few compunctions know your resources and full utility function, and can proceed to extract ~all liquid value from you.
I agree with Raemon that “judge” here means something closer to one of its standard usages, “to make inferences about”.
The post implies it is bad to be judged. I could have misinterpreted why, but that implication is there. If judge just meant “make inferences about” why would it be bad?
One, if the world is a place where may more vulnerabilities are more known, this incentivizes more people to specialize in exploiting those vulnerabilities.
But it also helps in knowing who’s exploiting them! Why does it give more advantages to the “bad” side?
Two, as a flawed human there are probably some stressors against which you can’t credibly play the “won’t negotiate with terrorists” card.
Why would you expect the terrorists to be miscalibrated about this before the reduction in privacy, to the point where they think people won’t negotiate with them when they actually will, and less privacy predictably changes this opinion?
I think the assumption is these are ~baseline humans we’re talking about, and most human brains can’t hold norms of sufficient sophistication to capture true ethical law
Perhaps the optimal set of norms for these people is “there are no rules, do what you want”. If you can improve on that, than that would constitute a norm-set that is more just than normlessness. Capturing true ethical law in the norms most people follow isn’t necessary.
I guess you can imagine the maximally inconvenient case where motivated people with low cost of time and few compunctions know your resources and full utility function, and can proceed to extract ~all liquid value from you.
The post implies it is bad to be judged. I could have misinterpreted why, but that implication is there. If judge just meant “make inferences about” why would it be bad?
As Raemon says, knowing that others are making correct inferences about your behavior means you can’t relax. No, idk, watching soap operas, because that’s an indicator of being less likely to repay your loans, and your premia go up. There’s an ethos of slack, decisionmaking-has-costs, strategizing-has-costs that Zvi’s explored in his previous posts, and that’s part of how I’m interpreting what he’s saying here.
But it also helps in knowing who’s exploiting them! Why does it give more advantages to the “bad” side?
Sure, but doesn’t it help me against them too?
You don’t want to spend your precious time on blackmailing random jerks, probably. So at best, now some of your income goes toward paying a white-hat blackmailer to fend off the black-hats. (Unclear what the market for that looks like. Also, black-hatters can afford to specialize in unblackmailability; it comes up much more often for them than the average person.) You’re right, though, that it’s possible to have an equilibrium where deterrence dominates and the black-hatting incentives are low, in which case maybe the white-hat fees are low and now you have a white-hat deterrent. So this isn’t strictly bad, though my instinct is that it’s bad in most plausible cases.
Why would you expect the terrorists to be miscalibrated about this before the reduction in privacy, to the point where they think people won’t negotiate with them when they actually will, and less privacy predictably changes this opinion?
That’s a fair point! A couple of counterpoints: I think risk-aversion of ‘terrorists’ helps. There’s also a point about second-order effects again; the easier it is to blackmail/extort/etc., the more people can afford to specialize in it and reap economies of scale.
Perhaps the optimal set of norms for these people is “there are no rules, do what you want”. If you can improve on that, than that would constitute a norm-set that is more just than normlessness. Capturing true ethical law in the norms most people follow isn’t necessary.
Eh, sure. My guess is that Zvi is making a statement about norms as they are likely to exist in human societies with some level of intuitive-similarity to our own. I think the useful question here is like “is it possible to instantiate norms s.t. norm-violations are ~all ethical-violations”. (we’re still discussing the value of less privacy/more blackmail, right?) No-rule or few-rule communities could work for this, but I expect it to be pretty hard to instantiate them at large scale. So sure, this does mean you could maybe build a small local community where blackmail is easy. That’s even kind of just what social groups are, as Zvi notes; places where you can share sensitive info because you won’t be judged much, nor attacked as a norm-violator. Having that work at super-Dunbar level seems tough.
As Raemon says, knowing that others are making correct inferences about your behavior means you can’t relax. No, idk, watching soap operas, because that’s an indicator of being less likely to repay your loans, and your premia go up.
This is really, really clearly false!
This assumes that, upon more facts being revealed, insurance companies will think I am less (not more) likely to repay my loans, by default (e.g. if I don’t change my TV viewing behavior).
More egregiously, this assumes that I have to keep putting in effort into reducing my insurance premiums until I have no slack left, because these premiums really, really, really matter. (I don’t even spend that much on insurance premiums!)
If you meant this more generally, and insurance was just a bad example, why is the situation worse in terms of slack than it was before? (I already have the ability to spend leisure time on gaining more money, signalling, etc.)
It’s true the net effect is low to first order, but you’re neglecting second-order effects. If premia are important enough, people will feel compelled to Goodhart proxies used for them until those proxies have less meaning.
Given the linked siderea post, maybe this is not very true for insurance in particular. I agree that wasn’t a great example.
Slack-wise, uh, choices are bad. really bad. Keep the sabbath. These are some intuitions I suspect are at play here. I’m not interested in a detailed argument hashing out whether we should believe that these outweigh other factors in practice across whatever range of scenarios, because it seems like it would take a lot of time/effort for me to actually build good models here, and opportunity costs are a thing. I just want to point out that these ideas seem relevant for correctly interpreting Zvi’s position.
I found this pretty useful—Zvi’s definitely reflecting a particular, pretty negative view of society and strategy here. But I disagree with some of your inferences, and I think you’re somewhat exaggerating the level of gloom-and-doom implicit in the post.
>Implication: “judge” means to use information against someone. Linguistic norms related to the word “judgment” are thoroughly corrupt enough that it’s worth ceding to these, linguistically, and using “judge” to mean (usually unjustly!) using information against people.
No, this isn’t bare repetition. I agree with Raemon that “judge” here means something closer to one of its standard usages, “to make inferences about”. Though it also fits with the colloquial “deem unworthy for baring [understandable] flaws”, which is also a thing that would happen with blackmail and could be bad.
>Implication: more generally available information about what strategies people are using helps “our” enemies more than it helps “us”. (This seems false to me, for notions of “us” that I usually use in strategy)
I can imagine a couple things going on here? One, if the world is a place where may more vulnerabilities are more known, this incentivizes more people to specialize in exploiting those vulnerabilities. Two, as a flawed human there are probably some stressors against which you can’t credibly play the “won’t negotiate with terrorists” card.
>Implication: even in the most just possible system of norms, it would be good to sometimes violate those norms and hide the fact that you violated them. (This seems incorrect to me!)
I think the assumption is these are ~baseline humans we’re talking about, and most human brains can’t hold norms of sufficient sophistication to capture true ethical law, and are also biased in ways that will sometimes strain against reflectively-endorsed ethics (e.g. they’re prone to using constrained circles of moral concern rather than universality).
>Implication: the bad guys won; we have rule by gangsters, who aren’t concerned with sustainable production, and just take as much stuff as possible in the short term. (This seems on the right track but partially false; the top marginal tax rate isn’t 100%)
This part of the post reminded me of (the SSC review of) Seeing Like a State, which makes a similar point; surveying and ‘rationalizing’ farmland, taking a census, etc. = legibility = taxability. “all of them” does seem like hyperbole here. I guess you can imagine the maximally inconvenient case where motivated people with low cost of time and few compunctions know your resources and full utility function, and can proceed to extract ~all liquid value from you.
The post implies it is bad to be judged. I could have misinterpreted why, but that implication is there. If judge just meant “make inferences about” why would it be bad?
But it also helps in knowing who’s exploiting them! Why does it give more advantages to the “bad” side?
Why would you expect the terrorists to be miscalibrated about this before the reduction in privacy, to the point where they think people won’t negotiate with them when they actually will, and less privacy predictably changes this opinion?
Perhaps the optimal set of norms for these people is “there are no rules, do what you want”. If you can improve on that, than that would constitute a norm-set that is more just than normlessness. Capturing true ethical law in the norms most people follow isn’t necessary.
Sure, but doesn’t it help me against them too?
As Raemon says, knowing that others are making correct inferences about your behavior means you can’t relax. No, idk, watching soap operas, because that’s an indicator of being less likely to repay your loans, and your premia go up. There’s an ethos of slack, decisionmaking-has-costs, strategizing-has-costs that Zvi’s explored in his previous posts, and that’s part of how I’m interpreting what he’s saying here.
You don’t want to spend your precious time on blackmailing random jerks, probably. So at best, now some of your income goes toward paying a white-hat blackmailer to fend off the black-hats. (Unclear what the market for that looks like. Also, black-hatters can afford to specialize in unblackmailability; it comes up much more often for them than the average person.) You’re right, though, that it’s possible to have an equilibrium where deterrence dominates and the black-hatting incentives are low, in which case maybe the white-hat fees are low and now you have a white-hat deterrent. So this isn’t strictly bad, though my instinct is that it’s bad in most plausible cases.
That’s a fair point! A couple of counterpoints: I think risk-aversion of ‘terrorists’ helps. There’s also a point about second-order effects again; the easier it is to blackmail/extort/etc., the more people can afford to specialize in it and reap economies of scale.
Eh, sure. My guess is that Zvi is making a statement about norms as they are likely to exist in human societies with some level of intuitive-similarity to our own. I think the useful question here is like “is it possible to instantiate norms s.t. norm-violations are ~all ethical-violations”. (we’re still discussing the value of less privacy/more blackmail, right?) No-rule or few-rule communities could work for this, but I expect it to be pretty hard to instantiate them at large scale. So sure, this does mean you could maybe build a small local community where blackmail is easy. That’s even kind of just what social groups are, as Zvi notes; places where you can share sensitive info because you won’t be judged much, nor attacked as a norm-violator. Having that work at super-Dunbar level seems tough.
This is really, really clearly false!
This assumes that, upon more facts being revealed, insurance companies will think I am less (not more) likely to repay my loans, by default (e.g. if I don’t change my TV viewing behavior).
More egregiously, this assumes that I have to keep putting in effort into reducing my insurance premiums until I have no slack left, because these premiums really, really, really matter. (I don’t even spend that much on insurance premiums!)
If you meant this more generally, and insurance was just a bad example, why is the situation worse in terms of slack than it was before? (I already have the ability to spend leisure time on gaining more money, signalling, etc.)
Relevant: https://siderea.dreamwidth.org/1486739.html
It’s true the net effect is low to first order, but you’re neglecting second-order effects. If premia are important enough, people will feel compelled to Goodhart proxies used for them until those proxies have less meaning.
Given the linked siderea post, maybe this is not very true for insurance in particular. I agree that wasn’t a great example.
Slack-wise, uh, choices are bad. really bad. Keep the sabbath. These are some intuitions I suspect are at play here. I’m not interested in a detailed argument hashing out whether we should believe that these outweigh other factors in practice across whatever range of scenarios, because it seems like it would take a lot of time/effort for me to actually build good models here, and opportunity costs are a thing. I just want to point out that these ideas seem relevant for correctly interpreting Zvi’s position.