Hey—to preface—obviously I’m a great admirer of yours Kaj and I’ve been grateful to learn a lot from you, particularly in some of the exceptional research papers you’ve shared with me.
With that said, of course your emotions are your own but in terms of group ethics and standards, I’m very much in disagreement.
The upset feels similar to what I’ve previously experienced when something that’s obviously a purely symbolic gesture is treated as a Big Important Thing That’s Actually Making A Difference.
On the one hand, you’re totally right. On the other hand, basically the entire world is made up of abstractions along these lines. What can the Supreme Court opinion in Marbury vs Madison be recognized as other than a purely symbolic gesture? Madison wasn’t going to deliver the commissions, Justice Marshall (no relation) knew that for sure, and he made a largely symbolic gesture in how he navigated the thing. It had no practical importance for a long time but now forms one of the foundations of American jurisprudence effecting, indirectly, billions of lives. But at the time, if you dig into the history, it really was largely symbolic at the time.
The world is built out of all sorts of abstract symbolism and intersubjective convention.
That by itself wouldn’t trigger the reaction; the world is full of purely symbolic gestures that are claiming to make a difference, but they mostly haven’t upset me in a long time. Some of the communication around Petrov Day has. I think it’s because of a sense that this idea is being pushed on people-that-I-care-about as something important despite not actually being in accordance to their values, and that there’s social pressure for people to be quiet about it and give in to the social pressure at a cost to their epistemics.
(“Canonical” was intentionally chosen, incidentally.)
I feel like Oliver’s comment is basically saying “people should have taken this seriously and people who treat this light-heartedly are in the wrong”. It’s spoken from a position of authority, and feels like it’s shaming people whose main sin is that they aren’t particularly persuaded by this ritual actually being significant, as no compelling reason for this ritual actually being significant has ever been presented.
In any case the light didn’t go on in my head about egalitarian instincts (instincts to prevent leaders from exercising power) killing online communities until just recently. [...] I have seen rationalist communities die because they trusted their moderators too little.
Honestly, for anything that wasn’t clearly egregiously wrong, I’d support the leadership team on here even if my feelings ran in a different direction. Like, leadership is hard. Really really really hard. If there was something I didn’t believe in, I’d just quietly opt out.
Now, I fully understand I’m in the minority on this position — but I’m against both ‘every interpretation is valid’ type thinking (why would every interpretation be valid as it relates to a group activity where your behavior effects the whole group?).
Likewise, pushing back against “shaming people whose main sin is that they aren’t particularly persuaded by this ritual actually being significant” — isn’t that actually both good and necessary if we want to be able to coordinate and actually solve problems?
There’s a dozen or so Yudkowsky citations about this. Here’s another:
Let’s say we have two groups of soldiers. In group 1, the privates are ignorant of tactics and strategy; only the sergeants know anything about tactics and only the officers know anything about strategy. In group 2, everyone at all levels knows all about tactics and strategy.
Should we expect group 1 to defeat group 2, because group 1 will follow orders, while everyone in group 2 comes up with better ideas than whatever orders they were given?
In this case I have to question how much group 2 really understands about military theory, because it is an elementary proposition that an uncoordinated mob gets slaughtered.
And finally,
Now it may be the case—a more agreeable part of me wants to interject—that this ritual actually is important, and that it should be treated as more than just a game.
But.
If so, I have never seen a particularly strong case being made for it.
I even did, like, math and stuff. The “shut up and multiply” thing.
Long story short — I think shared trust and demonstrated cooperation are super valuable, good leadership is incredibly underappreciated, and whimsical defection is really bad.
Again though — all written respectfully, etc etc, and I know I’m in the minority position here in terms of many subjective personal values, especially harm/care and seriousness/fun.
Finally, it’s undoubtedly true my estimate of the potential utility of building out a base of successfully navigated low-stakes cooperative endeavors is undoubtedly multiple orders of magnitude higher than others. I put the dollar-value of that as, actually, pretty high. Reasonable minds can differ on many of these points, but that’s my logic.
Thanks for engaging :) My upset part feels much calmer now that it has been spoken for, so I’m actually pretty chill about this right now. You’ve had a lot of stuff that I’ve gotten value from, too.
But note also that that post contains a lengthy excerpt about how the “Dark Side” descends into cultishness and insanity in situations where the word of leaders is accepted without question. That was clearly also depicted as the opposite failure mode.
I agree that rationalists don’t cooperate enough, and that often just offer criticism when it’s not warranted. But… it feels like a Fully General Counterargument if you take to that the point of “no coordination may be criticized, ever, including situation where people are arguably being shamed for having good epistemics”. That sounds like this bit from the post:
How do things work on the Dark Side?
The respected leader speaks, and there comes a chorus of pure agreement: if there are any who harbor inward doubts, they keep them to themselves. So all the individual members of the audience see this atmosphere of pure agreement, and they feel more confident in the ideas presented—even if they, personally, harbored inward doubts, why, everyone else seems to agree with it.
If anyone is still unpersuaded after that, they leave the group (or in some places, are executed)—and the remainder are more in agreement, and reinforce each other with less interference.
Re: Well-Kept Gardens—again, that feels like a Fully General Counterargument. Yes, certainly there should be moderator action when necessary… like, I am on the mod team, I have seen discussions about banning users etc. and all of that’s been fine to me.
But we’re not even talking about banning users here. This isn’t about keeping the garden, it’s about one particular ritual being picked to be important. Does Well-Kept Gardens imply that everything the admins do should be treated as correct?
Likewise, pushing back against “shaming people whose main sin is that they aren’t particularly persuaded by this ritual actually being significant” — isn’t that actually both good and necessary if we want to be able to coordinate and actually solve problems?
Back when I banned Eugine Nier, there were people who disagreed with that decision… but if they didn’t find my argument for the ban to be particularly persuasive, it never even crossed my mind that I should shame them because I had made an argument that didn’t compel them. They disagreed, the coordination necessary for saving the site still happened, no reason not to let them disagree.
So I read this to be a central premise that much of the rest of your comment builds on:
For Ben at least, the button thing was a symbolic exercise analogous to not nuking another country and he specifically asked you not to and said he’s trusting you.
Which to me feels like it’s assuming thing that I was asking someone to prove. Yes, Ben feels that the button was symbolic and analogous to not nuking another country. But it does not feel at all analogous to me; I feel like the ritual is picking a surface-level aspect of what happened to Petrov (something like the general idea of “you should not push the red button, doing so would have bad consequences”), and that’s about it when it comes to having a resemblance to it. The outcome matrix and psychological situation for the users is something completely different than what it was for Petrov. If the ritual actually did have a clear structural resemblance to Petrov’s dilemma, then I would have much less of a problem with it. But as it is, it does not feel like it’s training the site’s users in the thing that it claims to be training them in.
I’ll review and think more carefully later — out at dinner with a friend now — but my quick thought is that the proper venue, time, and place for expressing discontent with a cooperative community project is probably afterwards, possibly beforehand, and certainly not during… I don’t believe in immunity from criticism, obviously, but I am against defection when one doesn’t agree with a choice of norms.
That’s the quick take, will review more closely later.
Hey—to preface—obviously I’m a great admirer of yours Kaj and I’ve been grateful to learn a lot from you, particularly in some of the exceptional research papers you’ve shared with me.
With that said, of course your emotions are your own but in terms of group ethics and standards, I’m very much in disagreement.
On the one hand, you’re totally right. On the other hand, basically the entire world is made up of abstractions along these lines. What can the Supreme Court opinion in Marbury vs Madison be recognized as other than a purely symbolic gesture? Madison wasn’t going to deliver the commissions, Justice Marshall (no relation) knew that for sure, and he made a largely symbolic gesture in how he navigated the thing. It had no practical importance for a long time but now forms one of the foundations of American jurisprudence effecting, indirectly, billions of lives. But at the time, if you dig into the history, it really was largely symbolic at the time.
The world is built out of all sorts of abstract symbolism and intersubjective convention.
Canonical reply is this one:
https://www.lesswrong.com/s/pvim9PZJ6qHRTMqD3/p/7FzD7pNm9X68Gp5ZC
(“Canonical” was intentionally chosen, incidentally.)
https://www.lesswrong.com/posts/tscc3e5eujrsEeFN4/well-kept-gardens-die-by-pacifism
From Well-Kept Gardens:
Honestly, for anything that wasn’t clearly egregiously wrong, I’d support the leadership team on here even if my feelings ran in a different direction. Like, leadership is hard. Really really really hard. If there was something I didn’t believe in, I’d just quietly opt out.
Now, I fully understand I’m in the minority on this position — but I’m against both ‘every interpretation is valid’ type thinking (why would every interpretation be valid as it relates to a group activity where your behavior effects the whole group?).
Likewise, pushing back against “shaming people whose main sin is that they aren’t particularly persuaded by this ritual actually being significant” — isn’t that actually both good and necessary if we want to be able to coordinate and actually solve problems?
There’s a dozen or so Yudkowsky citations about this. Here’s another:
https://www.lesswrong.com/posts/KsHmn6iJAEr9bACQW/bayesians-vs-barbarians
And finally,
I made that case last year extensively:
https://www.lesswrong.com/posts/vvzfFcbmKgEsDBRHh/honoring-petrov-day-on-lesswrong-in-2019?commentId=ZZ87dbYiGDu6uMtF8
I even did, like, math and stuff. The “shut up and multiply” thing.
Long story short — I think shared trust and demonstrated cooperation are super valuable, good leadership is incredibly underappreciated, and whimsical defection is really bad.
Again though — all written respectfully, etc etc, and I know I’m in the minority position here in terms of many subjective personal values, especially harm/care and seriousness/fun.
Finally, it’s undoubtedly true my estimate of the potential utility of building out a base of successfully navigated low-stakes cooperative endeavors is undoubtedly multiple orders of magnitude higher than others. I put the dollar-value of that as, actually, pretty high. Reasonable minds can differ on many of these points, but that’s my logic.
Thanks for engaging :) My upset part feels much calmer now that it has been spoken for, so I’m actually pretty chill about this right now. You’ve had a lot of stuff that I’ve gotten value from, too.
But note also that that post contains a lengthy excerpt about how the “Dark Side” descends into cultishness and insanity in situations where the word of leaders is accepted without question. That was clearly also depicted as the opposite failure mode.
I agree that rationalists don’t cooperate enough, and that often just offer criticism when it’s not warranted. But… it feels like a Fully General Counterargument if you take to that the point of “no coordination may be criticized, ever, including situation where people are arguably being shamed for having good epistemics”. That sounds like this bit from the post:
Re: Well-Kept Gardens—again, that feels like a Fully General Counterargument. Yes, certainly there should be moderator action when necessary… like, I am on the mod team, I have seen discussions about banning users etc. and all of that’s been fine to me.
But we’re not even talking about banning users here. This isn’t about keeping the garden, it’s about one particular ritual being picked to be important. Does Well-Kept Gardens imply that everything the admins do should be treated as correct?
Back when I banned Eugine Nier, there were people who disagreed with that decision… but if they didn’t find my argument for the ban to be particularly persuasive, it never even crossed my mind that I should shame them because I had made an argument that didn’t compel them. They disagreed, the coordination necessary for saving the site still happened, no reason not to let them disagree.
So I read this to be a central premise that much of the rest of your comment builds on:
Which to me feels like it’s assuming thing that I was asking someone to prove. Yes, Ben feels that the button was symbolic and analogous to not nuking another country. But it does not feel at all analogous to me; I feel like the ritual is picking a surface-level aspect of what happened to Petrov (something like the general idea of “you should not push the red button, doing so would have bad consequences”), and that’s about it when it comes to having a resemblance to it. The outcome matrix and psychological situation for the users is something completely different than what it was for Petrov. If the ritual actually did have a clear structural resemblance to Petrov’s dilemma, then I would have much less of a problem with it. But as it is, it does not feel like it’s training the site’s users in the thing that it claims to be training them in.
Good points.
I’ll review and think more carefully later — out at dinner with a friend now — but my quick thought is that the proper venue, time, and place for expressing discontent with a cooperative community project is probably afterwards, possibly beforehand, and certainly not during… I don’t believe in immunity from criticism, obviously, but I am against defection when one doesn’t agree with a choice of norms.
That’s the quick take, will review more closely later.