This was the first major, somewhat adversarial doublecrux that I’ve participated in.
(Perhaps this is a wrong framing. I participated in many other significant, somewhat adversarial doublecruxes before. But, I dunno, this felt significantly harder than all the previous ones, the point where it feels like a difference in kind)
It was a valuable learning experience for me. My two key questions for “Does this actually make sense as part of the 2019 Review Book” are:
Is this useful to others for learning how to doublecrux, pass ITTs, etc in a lowish-trust-setting?
Is this useful to others takeaways for the object level disagreement?
On the object level, my tl;dr takes the form of “which blogposts should someone write as a followup?”, which I think are:
Criticism and Accusations are importantly different, and should be distinguished. (I think some miscommunication came people implicitly lumping these together)
Harmony matters for group truthseeking, but, Alice telling Bob ‘you should be nicer. Here’s how to say the same thing more nicely’ is really scary if Alice in fact didn’t understand exactly what they were trying to say.
(I realize Benquo/Jessica probably still disagreement with my beliefs/emphasis on the first part. But this was a concrete update I made while reviewing the post, and a mistake I think I was making a lot. Even if I later change my mind about how much harmony matters for group truthseeking, it’d still be necessary for the post to directly address the benefits in order to be understood by past-me)
I think there are more points worth lifting out of here, but I’m not sure how oddly specific they were to the particular people in this conversation, rather than generally useful.
On “how did this go as a doublecrux”, I notice:
Well, I learned a lot, at least on the object level.
We didn’t reach total or even significant agreement. Benquo/Jessica I think began commenting less sometime after this.
That might be fine. I don’t think this is was Benquo/Jessica’s goal (I think their goal was more like ‘figure out if the LessWrong Team is aligned with them enough to be worth investing in LessWrong’, and I think they succeeded at that)
I’m a bit sad about the outcome, but not sure whether I should be. I do think if this was my second rodeo I could have done better. (I’m guessing in large part everyone involved burned through their budget for having intense disagreements with people who didn’t seem aligned, and that most of that budget was burned through prior to the final conversations on mistakes that could have been avoided, and if we had had another month of energy we could have actually figured out enough common ground to be more closely allied.)
On “can other people learn from this as a doublecrux?”
I… don’t know. I think maybe, but that’s mostly up to other people.
Note on Framing:
I notice is that a large chunk of the text of this post are direct quotes from Benquo and Jessicata, but it’s wrapped in a post where I control the frame. If this were considered for inclusion-in-the-book, I’d be interested in having them write reviews of their year-later-takeaways, written in their own frames.
Some notes regarding object level ideas in this post and the discussion:
(each quoted section is basically a new topic)
Benquo: (emphasis mine)
What I see as under threat is the ability to say in a way that’s actually heard, not only that opinion X is false, but that the process generating opinion X is untrustworthy, and perhaps actively optimizing in an objectionable direction. Frequently, attempts to say this are construed primarily as moves to attack some person or institution, pushing them into the outgroup. Frequently, people suggest to me an “equivalent” wording with a softer tone, which in fact omits important substantive criticisms I mean to make, while claiming to understand what’s at issue.
While I still have many complaints about the overall strategy Benquo was following at the time, I think (hope?) I’m more understanding now about the failure mode pointed at here. I do think I’ve contributed to that failure mode, i.e. “try to be more diplomatic to preserve group harmony, in a way that comes at expense of clarity.”
I still think there are good truthseeking reasons to preserve group harmony. But I think the concrete updates I’ve made are that we need to (at least) be very clear about when we’re doing that, and notice when attempts to smooth things over are destroying information.
In particular, it is pretty orwellian/gaslighty to have someone tell you “You’re being too mean. Here’s a different thing you could have said with the same truth value that wouldn’t have been as mean, see?” and watch in horror as they then describe a sentence that leaves out important information you meant to convey.
In my other review, I mentioned “hmm, I think I still have promises to keep regarding ‘what aesthetic updates should I make?’”. I think one aesthetic update I am happy to make is that I should have some kind of disgust/horror when someone (including me) claims to be preserving local truth value, or implying that truth value is preserved, when in fact it wasn’t.
(This is a specific subset of the overall worldview/aesthetic I think Benquo was trying to convey, and I’m guessing there is still major disagreement in other nearby areas)
Benquo:
It seemed like the hidden second half of the core claim [of the “5 words” post]was “and therefore we should coordinate around simpler slogans,” and not the obvious alternative conclusion “and therefore we should scale up more carefully, with an uncompromising emphasis on some aspects of quality control.” (See On the Construction of Beacons for the relevant argument.)
It seemed to me like there was some motivated ambiguity on this point. The emphasis seemed to consistently recommend public behavior that was about mobilization rather than discourse, and back-channel discussions among well-connected people (including me) that felt like they were more about establishing compatibility than making intellectual progress
I am still mulling this over. I think it might be pointing at something I haven’t yet fully grokked.
I would agree with the phrase “we should scale up more carefully, with an uncompromising emphasis on some aspects of quality control”. (I think I would have agreed with it at the time, which is part of why the doublecrux was tricky. I eventually realized that Benquo meant a stronger version of this sentence than I meant)
My current (revealed) belief is something like “We don’t really have the luxury of stopping all mobilization while we figure out the ideal coordination mechanisms. Meanwhile I think current mobilization efforts are net positive. I also think the process of actually mobilizing is also useful for forcing your ivory tower coordination process to be more connected with the reality of how large scale coordination actually works.”
(My understanding is that Benquo-at-the-time thought the current way large scale coordination works is fundamentally doomed and don’t have much choice but to start over. That does feel pretty cruxy – if I believed that I’d be doing different things.)
Benquo:
This, even though it seems like you explicitly agree with me that our current social coordination mechanisms are massively inadequate, in a way that (to me obviously) implies that they can’t possibly solve FAI.
(By contrast, I do agree with the first half of the sentence, that our current coordination mechanisms are massively inadequate, and am grateful for various gears about what’s going on there that I gained during this conversation)
Jessicata:
That said, there’s still a complicated question of “how do you make criticisms well”. I think advice on this is important. I think the correct advice usually looks more like advice to whistleblowers than advice for diplomacy.
This feels aesthetically cruxy. I think it’s a few steps removed from whatever the real disagreement is about.
I think a key piece here is the distinction between “criticism” and “accusations of norm violation.” I mention this at the bottom of the post, but I think it warrants a separate top level post that delves into more details.
Jessica:
Note, my opinion of your opinions, and my opinions, are expressed in pretty different ontologies.
One thing I noticed at the time and still notice now is that it’s not actually obvious to me (from Jessica’s written words in the preceding section) that our claims are in different ontologies. I derive that they must be in different ontologies (given observations about how challenging this whole conversation was). But, it is worth noting that Jessica’s claims/beliefs seem to make sense in my ontology.
Zack, in the comments:
> politics backpropogates into truthseeking, causes people to view truthseeking norms as a political weapon.
Imagine that this had already happened. How would you go about starting to fix it, other than by trying to describe the problem as clearly as possible (that is, “invent[ing] truthseeking-politics-on-the-fly”)?
I was distracted by another piece of this comment, but I agree that having a good answer for this is pretty important.
“Defining Clarity”
After writing this post, there was significant disagreement in the comments about this line of mine:
I define clarity in terms of what gets understood, rather than what gets said. So, using words with non-standard connotations, without doing a lot of up-front work to redefine your terms, seems to me to be reducing clarity, and/or mixing clarity, rather than improving it.
I’m still not entirely sure what happened here, but the failure mode that Jessica/Zvi/Zack were pointing at was “You auto-lose if you incentive people not to understand.” That seems true to me, but mostly unrelated to what I was trying to say here, and some of my own response was perhaps overly exasperated with them seeming to change the subject on me.
Zvi eventually said:
Imagine three levels of explanation: Straightforward to you, straightforward to those without motivated cognition, straightforward even to those with strong motivated cognition.
It is reasonable to say that getting from level 1 to level 2 is often a hard problem, that it is on you to solve that problem.
It is not reasonable, if you want clarity to win, to say that level 2 is insufficient and you must reach level 3. It certainly isn’t reasonable to notice that level 2 has been reached, but level 3 has not, and thus judge the argument insufficient and a failure. It would be reasonable to say that reaching level 3 would be *better* and suggest ways of doing so.
I think it’s possible that at that point I could have said “Okay. I’m talking about level 2, and the point is you make it much harder to get to level-2 if you’re making up new words or using them with nonstandard connotations.” But by the time we got to that point of the conversation I was pretty exhausted and still confused about how everything fit together. Today, I’m not 100% sure whether my hypothetical reply was straightforwardly true.
...
I feel like I want to tie this all up together somehow, but I think I mostly did that in the tl;dr at the top. Thanks for reading I guess. Still interested in delving into individual threads if people are interested.
This was the first major, somewhat adversarial doublecrux that I’ve participated in.
(Perhaps this is a wrong framing. I participated in many other significant, somewhat adversarial doublecruxes before. But, I dunno, this felt significantly harder than all the previous ones, the point where it feels like a difference in kind)
It was a valuable learning experience for me. My two key questions for “Does this actually make sense as part of the 2019 Review Book” are:
Is this useful to others for learning how to doublecrux, pass ITTs, etc in a lowish-trust-setting?
Is this useful to others takeaways for the object level disagreement?
On the object level, my tl;dr takes the form of “which blogposts should someone write as a followup?”, which I think are:
Criticism and Accusations are importantly different, and should be distinguished. (I think some miscommunication came people implicitly lumping these together)
Harmony matters for group truthseeking, but, Alice telling Bob ‘you should be nicer. Here’s how to say the same thing more nicely’ is really scary if
Alice in fact didn’t understand exactly what they were trying to say.
(I realize Benquo/Jessica probably still disagreement with my beliefs/emphasis on the first part. But this was a concrete update I made while reviewing the post, and a mistake I think I was making a lot. Even if I later change my mind about how much harmony matters for group truthseeking, it’d still be necessary for the post to directly address the benefits in order to be understood by past-me)
I think there are more points worth lifting out of here, but I’m not sure how oddly specific they were to the particular people in this conversation, rather than generally useful.
On “how did this go as a doublecrux”, I notice:
Well, I learned a lot, at least on the object level.
We didn’t reach total or even significant agreement. Benquo/Jessica I think began commenting less sometime after this.
That might be fine. I don’t think this is was Benquo/Jessica’s goal (I think their goal was more like ‘figure out if the LessWrong Team is aligned with them enough to be worth investing in LessWrong’, and I think they succeeded at that)
I’m a bit sad about the outcome, but not sure whether I should be. I do think if this was my second rodeo I could have done better. (I’m guessing in large part everyone involved burned through their budget for having intense disagreements with people who didn’t seem aligned, and that most of that budget was burned through prior to the final conversations on mistakes that could have been avoided, and if we had had another month of energy we could have actually figured out enough common ground to be more closely allied.)
That said, this was the fuel that eventually output Noticing Frames and Propagating Facts into Aesthetics, which I’m pretty happy with.
On “can other people learn from this as a doublecrux?”
I… don’t know. I think maybe, but that’s mostly up to other people.
Note on Framing:
I notice is that a large chunk of the text of this post are direct quotes from Benquo and Jessicata, but it’s wrapped in a post where I control the frame. If this were considered for inclusion-in-the-book, I’d be interested in having them write reviews of their year-later-takeaways, written in their own frames.
Some notes regarding object level ideas in this post and the discussion:
(each quoted section is basically a new topic)
Benquo: (emphasis mine)
While I still have many complaints about the overall strategy Benquo was following at the time, I think (hope?) I’m more understanding now about the failure mode pointed at here. I do think I’ve contributed to that failure mode, i.e. “try to be more diplomatic to preserve group harmony, in a way that comes at expense of clarity.”
I still think there are good truthseeking reasons to preserve group harmony. But I think the concrete updates I’ve made are that we need to (at least) be very clear about when we’re doing that, and notice when attempts to smooth things over are destroying information.
In particular, it is pretty orwellian/gaslighty to have someone tell you “You’re being too mean. Here’s a different thing you could have said with the same truth value that wouldn’t have been as mean, see?” and watch in horror as they then describe a sentence that leaves out important information you meant to convey.
In my other review, I mentioned “hmm, I think I still have promises to keep regarding ‘what aesthetic updates should I make?’”. I think one aesthetic update I am happy to make is that I should have some kind of disgust/horror when someone (including me) claims to be preserving local truth value, or implying that truth value is preserved, when in fact it wasn’t.
(This is a specific subset of the overall worldview/aesthetic I think Benquo was trying to convey, and I’m guessing there is still major disagreement in other nearby areas)
Benquo:
I am still mulling this over. I think it might be pointing at something I haven’t yet fully grokked.
I would agree with the phrase “we should scale up more carefully, with an uncompromising emphasis on some aspects of quality control”. (I think I would have agreed with it at the time, which is part of why the doublecrux was tricky. I eventually realized that Benquo meant a stronger version of this sentence than I meant)
My current (revealed) belief is something like “We don’t really have the luxury of stopping all mobilization while we figure out the ideal coordination mechanisms. Meanwhile I think current mobilization efforts are net positive. I also think the process of actually mobilizing is also useful for forcing your ivory tower coordination process to be more connected with the reality of how large scale coordination actually works.”
(My understanding is that Benquo-at-the-time thought the current way large scale coordination works is fundamentally doomed and don’t have much choice but to start over. That does feel pretty cruxy – if I believed that I’d be doing different things.)
Benquo:
“Can’t possibly solve FAI” still sounds like an obviously false marketing claim to me. I wrote a blogpost arguing you should be suspicious when you find yourself saying this.
(By contrast, I do agree with the first half of the sentence, that our current coordination mechanisms are massively inadequate, and am grateful for various gears about what’s going on there that I gained during this conversation)
Jessicata:
This feels aesthetically cruxy. I think it’s a few steps removed from whatever the real disagreement is about.
I think a key piece here is the distinction between “criticism” and “accusations of norm violation.” I mention this at the bottom of the post, but I think it warrants a separate top level post that delves into more details.
Jessica:
One thing I noticed at the time and still notice now is that it’s not actually obvious to me (from Jessica’s written words in the preceding section) that our claims are in different ontologies. I derive that they must be in different ontologies (given observations about how challenging this whole conversation was). But, it is worth noting that Jessica’s claims/beliefs seem to make sense in my ontology.
Zack, in the comments:
I was distracted by another piece of this comment, but I agree that having a good answer for this is pretty important.
“Defining Clarity”
After writing this post, there was significant disagreement in the comments about this line of mine:
I’m still not entirely sure what happened here, but the failure mode that Jessica/Zvi/Zack were pointing at was “You auto-lose if you incentive people not to understand.” That seems true to me, but mostly unrelated to what I was trying to say here, and some of my own response was perhaps overly exasperated with them seeming to change the subject on me.
Zvi eventually said:
I think it’s possible that at that point I could have said “Okay. I’m talking about level 2, and the point is you make it much harder to get to level-2 if you’re making up new words or using them with nonstandard connotations.” But by the time we got to that point of the conversation I was pretty exhausted and still confused about how everything fit together. Today, I’m not 100% sure whether my hypothetical reply was straightforwardly true.
...
I feel like I want to tie this all up together somehow, but I think I mostly did that in the tl;dr at the top. Thanks for reading I guess. Still interested in delving into individual threads if people are interested.