Thank you for adding your detailed take/observations.
My own take on some of the details of CFAR that’re discussed in your comment:
Debugging sessions with Anna and with other members of the leadership was nigh unavoidable and asymmetric, meaning that while the leadership could avoid getting debugged it was almost impossible to do so as a rank-and-file member. Sometimes Anna described her process as “implanting an engine of desperation” within the people she was debugging deeply. This obviously had lots of ill psychological effects on the people involved, but some of them did seem to find a deeper kind of motivation.
I think there were serious problems here, though our estimates of the frequencies might differ.
To describe the overall situation in detail:
I often got debugging help from other members of CFAR, but, as noted in the quote, it was voluntary. I picked when and about what and did not feel pressure to do so.
I can think of at least three people at CFAR who had a lot of debugging sort of forced on them (visibly expected as part of their job set-up or of check-in meetings or similar; they didn’t make clear complaints but that is still “sort of forced”), in ways that were large and that seem to me clearly not okay in hindsight. I think lots of other people mostly did not experience this. There are a fair number of people about whom I am not sure or would make an in-between guess. To be clear, I think this was bad (predictably harmful, in ways I didn’t quite get at the time but that e.g. standard ethical guidelines in therapy have long known about), and I regret it and intend to avoid “people doing extensive debugging of those they have direct power over” contexts going forward.
I believe this sort of problem was more present in the early years, and less true as CFAR became older, better structured, somewhat “more professional”, and less centered around me. In particular, I think Pete’s becoming ED helped quite a bit. I also think the current regime (“holocracy”) has basically none of this, and is structured so as to predictably have basically none of this—predictably, since there’s not much in the way of power imbalances now.
It’s plausible I’m wrong about how much of this happened, and how bad it was, in different eras. In particular, it is easy for those in power (e.g., me) to underestimate aspects of how bad it is not to have power; and I did not do much to try to work around the natural blindspot. If anyone wants to undertake a survey of CFAR’s past and present staff on this point (ideally someone folks know and can accurately trust to maintain their anonymity while aggregating their data, say, and then posting the results to LW), I’d be glad to get email addresses for CFAR’s past and present staff for the purpose.
I’m sure I did not describe my process as “implanting an engine of desperation”; I don’t remember that and it doesn’t seem like a way I would choose to describe what I was doing. “Implanting” especially doesn’t. As Eli notes (this hadn’t occurred to me, but might be what you’re thinking of?), I did talk some about trying to get in touch with one’s “quiet desperation”, and referenced Pink Floyd’s song “Time” and “the mass of men lead lives of quiet desperation” and developed concepts around that; but this was about accessing a thing that was already there, not “implanting” a thing. I also led many people in “internal double cruxes around existential risk”, which often caused fairly big reactions as people viscerally noticed “we might all die.”
Relatedly, the organization uses a technique called goal factoring during debugging which was in large part inspired by Geoff Anders’ Connection Theory and was actually taught by Geoff at CFAR workshops at some point. This means that CFAR debugging in many ways resembles Leverage’s debugging and the similarity in naming isn’t just a coincidence of terms.
I disagree with this point overall.
Goal-Factoring was first called “use fungibility”, a technique I taught within a class called “microeconomics 1” at the CFAR 2012 minicamps prior to Geoff doing any teaching. It was also discussed at times in some form at the old SingInst visiting fellows program, IIRC.
Geoff developed it, and taught it at many CFAR workshops in early years (2013-2014, I think). The choice that it was Goal-Factoring that Geoff (was asked to teach? wanted to teach? I don’t actually remember; probably both?) was I think partly to do with its resemblance to the beginning/repeated basic move in Connection Theory.
No one at CFAR was required to use the double-crux conversational technique for reaching agreement, but if a rank-and-file member refused to they were treated as if they were being intellectually dishonest, while if a leader refused to they were just exercising their right to avoid double-cruxing. While I believe the technique is epistemically beneficial, the uneven demands on when it is used biases outcomes of conversations.
My guess is that there were asymmetries like this, and that they were important, and that they were not worse than most organizations (though that’s really not the right benchmark). Insofar as you have experience at other organizations (e.g. mainstream tech companies or whatnot), or have friends with such experience who you can ask questions of, I am curious how you think they compare.
On my own list of “things I would do really differently if I was back in 2012 starting CFAR again”, the top-ranked item is probably:
Share information widely among staff, rather than (mostly unconsciously/not-that-endorsedly) using lack-of-information-sharing to try to control people and outcomes.
Do consider myself to have some duty to explain decisions and reply to questions. Not “before acting”, because the show must go on and attempts to reach consensus would be endless. And not “with others as an authority that can prevent me from acting if they don’t agree.” But yes with a sincere attempt to communicate my actual beliefs and causes of actions, and to hear others’ replies, insofar as time permits.
I don’t think I did worse than typical organizations in the wider world, on the above points.
I’m honestly uncertain how much this is/isn’t related to the quoted complaint.
There were required sessions of a social/relational practice called circling (which kind of has a cult of its own). It should be noted that circling as a practice is meant to be egalitarian and symmetric, but circling within the context of CFAR had a weird power dynamic because subordinates would circle with the organizational leaders. The whole point of circling is to create a state of emotional vulnerability and openness in the person who is being circled. This often required rank-and-file members to be emotionally vulnerable to the leadership who perhaps didn’t actually have their best interests at heart.
Duncan’s reply here is probably more accurate to the actual situation at CFAR than mine would be. (I wrote much of the previous paragraphs before seeing his, but endorsing Duncan’s on this here seems best.) If Pete wants to weigh in I would also take his perspective quite seriously here. I don’t quite remember some of the details.
As Duncan noted, “creating a state of emotional vulnerability and openness” is really not supposed to be the point of circling, but it is a thing that happens pretty often and that a person might not know how to avoid.
The point of circling IMO is to break all the fourth walls that conversations often skirt around, let the subtext or manner in which the conversation is being done be made explicit text, and let it all thereby be looked at together.
A different thing that I in hindsight think was an error (that I already had on my explicit list of “things to do differently going forward”, and had mentioned in this light to a few people) was using circling in the way we did at AIRCS workshops, where some folks were there to try to get jobs. My current view, as mentioned a bit above, is that something pretty powerfully bad sometimes happens when a person accesses bits of their insides (in the way that e.g. therapy or some self-help techniques lead people to) while also believing they need to please an external party who is looking at them and has power over them.
(My guess is that well-facilitated circling is fine at AIRCS-like programs that are less directly recruiting-oriented. Also that circling at AIRCS had huge upsides. This is a can of worms I don’t plan to go into right now, in the middle of this comment reply, but flagging it to make my above paragraph not overgenralized-from.)
The overall effect of all this debugging and circling was that it was hard to maintain the privacy and integrity of your mind if you were a rank-and-file employee at CFAR.
I believe this was your experience, and am sorry. My non-confident guess is that some others experienced this and most didn’t, and that the impact on folks’ mental privacy was considerably more invasive than a standard workplace would’ve been, and that the impact on folks’ integrity was probably less bad than my guess at many mainstream workplace’s impact but still a lot worse than the CFAR we ought to aim for.
Personally I am not much trying to maintain the privacy of my own mind at this point, but I am certainly trying to maintain its integrity, and I think being debugged by people with power over me would not be good for that.
The longer you stayed with the organization, the more it felt like your family and friends on the outside could not understand the problems facing the world, because they lacked access to the reasoning tools and intellectual leaders you had access to. This led to a deep sense of alienation from the rest of society. Team members ended up spending most of their time around other members and looking down on outsiders as “normies”.
This wasn’t my experience at all, personally. I did have some feeling of distance when I first started caring about AI risk in ~2008, but it didn’t get worse across CFAR. I also stayed in a lot of contact with folks outside the CFAR / EA / rationalist / AI risk spheres through almost all of it. I don’t think I looked down on outsiders.
There was a rarity narrative around being part of the only organization trying to “actually figure things out”, ignoring other organizations in the ecosystem working on AI safety and rationality and other communities with epistemic merit. CFAR/MIRI perpetuated the sense that there was nowhere worthwhile to go if you left the organization.
I thought CFAR and MIRI were part of a rare and important thing, but I did not think CFAR (nor CFAR + MIRI) was the only thing to matter. I do think there’s some truth in the “rarity narrative” claim, at CFAR, mostly via me and to a much smaller extent some others at CFAR having some of this view of MIRI.
There was a rarity narrative around the sharpness of Anna’s critical thinking skills, which made it so that if Anna knew everything you knew about a concern and disagreed with you, there was a lot of social pressure to defer to her judgment.
I agree that this happened and that it was a problem. I didn’t consciously intend to set this up, but my guess is that I did a bunch of things to cause it anyhow. In particular, there’s a certain way I used to sort of take the ground out from under people when we talked, that I think contributed to this. (I used to often do something like: stay cagey about my own opinions; listen carefully to how my interlocutor was modeling the world; show bits of evidence that refuted some of their assumptions; listen to their new model; repeat; … without showing my work. And then they would defer to me, instead of having stubborn opinions I didn’t know how to shift, which on some level was what I wanted.)
People at current-CFAR respect my views still, but it actually feels way healthier to me now. Partly because I’m letting my own views and their causes be more visible, which I think makes it easier to respond to. And because I somehow have less of a feeling of needing to control what other people think or do via changing their views.
(I haven’t checked the above much against others’ perceptions, so would be curious for anyone from current or past CFAR with a take.)
There was rampant use of narrative warfare (called “narrativemancy” within the organization) by leadership to cast aspersions and blame on employees and each other. There was frequent non-ironic use of magical and narrative schemas which involved comparing situations to fairy-tales or myths and then drawing conclusions about those situations with high confidence. The narrativemancer would operate by casting various members of the group into roles and then using the narrative arc of the story to make predictions about how the relationship dynamics of the people involved would play out. There were usually obvious controlling motives behind the narrative framings being employed, but the framings were hard to escape for most employees.
I believe this was your experience, mostly because I’m pretty sure I know who you are (sorry; I didn’t mean to know and won’t make it public) and I can think of at least one over-the-top (but sincere) conversation you could reasonably describe at least sort of this way (except for the “with high confidence”, I guess, and the “frequent”; and some other bits), plus some repeated conflicts.
I don’t think this was a common experience, or that it happened much at all (or at all at all?) in contexts not involving you, but it’s possible I’m being an idiot here somehow in which case someone should speak up.
Which I guess is to say that the above bullet point seems to me, from my experiences/observations, to be mostly or almost-entirely false, but that I think you’re describing your experiences and guesses about the place accurately and that I appreciate you speaking up.
[all the other bullet points]
I agree with parts and disagree with parts; but seemed mostly less interesting than the above.
—
Anyhow, thanks for writing, and I’m sorry you had bad experiences at CFAR, especially about the fairly substantial parts of the above bad parts that were my fault.
I expect my reply will accidentally make some true points you’re making harder to see (as well as hopefully adding light to some other parts), and I hope you’ll push back in those places.
Related to my reply to PhoenixFriend (in the parent comment), but hopping meta from it:
I have a question for whoever out there thinks they know how the etiquette of this kind of conversation should go. I had a first draft of my reply to PhoenixFriend, where I … basically tried to err on the side of being welcoming, looking for and affirming the elements of truth I could hear in what PhoenixFriend had written, and sort of emphasizing those elements more than my also-real disagreements. I ran it by a CFAR colleague at my colleague’s request, who said something like “look, I think your reply is pretty misleading; you should be louder and clearer about the ways your best guess about what happened differed from what’s described in PhoenixFriend’s comment. Especially since I and others at CFAR have our names on the organization too, so if you phrase things in ways that’ll cause strangers who’re skim-reading to guess that things at CFAR were worse than they were, you’ll inaccurately and unjustly mess with other peoples’ reputations too.” (Paraphrased.)
So then I went back and made my comments more disagreeable and full of details about where my and PhoenixFriend’s models differ. (Though probably still less than the amount that would’ve fully addressed my colleague’s complaints.)
This… seems better in that it addresses my colleague’s pretty reasonable desire, but worse in that it is not welcoming to someone who is trying to share info and is probably finding that hard. I am curious if anyone has good thoughts on how this sort of etiquette should go, if we want to have an illuminating, get-it-all-out-there, non-misleading conversation.
Part of why I’m worried, is it seems to me pretty easy for people who basically think the existing organizations are good, and also that mainstream workplaces are non-damaging and so on, to upvote/downvote each new datum based on those priors plus a (sane and sensible) desire to avoid hurting others’ feelings and reputations without due cause, etc., in ways that despite their reasonability may make it hard for real and needed conversations that are contrary to our current patterns of seeing to get started.
For example, I think PhoenixFriend indeed saw some real things at CFAR that many of those downvoting their comment did not see and mistakenly wouldn’t expect to see, but that also many of the details of PhoenixFriend’s comment are off, partly maybe because they were mis-generalizing from their experiences and partly because it’s hard to name things exactly (especially to people who have a bit of an incentive to mishear.)
(Also, to try briefly and poorly to spell out why I’m rooting for a “get it all out on the table” conversation, and not just a more limited “hear and acknowledge the mostly blatant/known harms, correct those where possible, and leave the rest of our reputation intact” conversation: basically, I think there’s a bunch of built-up “technical debt”, in the form of confusion and mistrust and trying-not-to-talk-about-particular-things-because-others-will-form-“unreasonable”-conflusions-if-we-do and who-knows-why-we-do-that-but-we-do-so-there’s-probably-a-reason, that I’m hoping gets cleared out by the long and IMO relatively high-quality and contentful conversation that’s been happening so far. I want more of that if we can get it. I want culture and groups to be able to build around here without building on top of technical debt. I also want information about how organizations do/don’t work well, and, in terms of means of acquiring this information, I much prefer bad-looking conversations on LW to wasting another five years doing it wrong.)
Personally I am not much trying to maintain the privacy of my own mind at this point,
This sounds like an extreme and surprising statement. I wrote out some clarifying questions like “what do you mean by privacy here”, but maybe it’d be better to just say:
I think it strikes me funny because it sounds sort of like a PR statement. And it sounds like a statement that could set up a sort of “iterations of the Matrix”-like effect. Where, you say “ok now I want to clear out all the miasma, for real”, and then you and your collaborators do a pretty good job at that; but also, something’s been lost or never gained, namely the logical common knowledge that there’s probably-ongoing, probably difficult to see dynamics that give rise to the miasma of {ungrounded shared narrative, information cascades, collective blindspots, deferrals, circular deferrals, misplaced/miscalibrated trust, etc. ??}. In other words, since these things happened in a context where you and your collaborators were already using reflection, introspection, reasoning, communication, etc., we learn that the ongoing accumulation of miasma is a more permanent state of affairs, and this should be common knowledge. Common knowledge would for example help with people being able to bring up information about these dynamics, and expect their information to be put to good use.
(I notice an analogy between iterations of the Matrix and economic boom-bust cycles.)
“get it all out on the table” conversation
“technical debt” [...] I’m hoping gets cleared out
These statements also seem to imply a framing that potentially has the (presumably unintentional) effect of subtly undermining the common knowledge of ongoing miasma-or-whatever. Like, it sort of directs attention to the content but not the generator, or something; like, one could go through all the “stuff” and then one would be done.
This sounds like an extreme and surprising statement.
Well, maybe I phrased it poorly; I don’t think what I’m doing is extreme; “much” is doing a bunch of work in my “I am not much trying to...” sentence.
I mean, there’s plenty I don’t want to share, like a normal person. I have confidential info of other peoples that I’m committed to not sharing, and plenty of my own stuff that I am private about for whatever reason. But in terms of rough structural properties of my mind, or most of my beliefs, I’m not much trying for privacy. Like when I imagine being in a context where a bunch of circling is happening or something (circling allows silence/ignoring questions/etc..; still, people sometimes complain that facial expressions leak through and they don’t know how to avoid it), I’m not personally like “I need my privacy though.” And I’ve updated some toward sharing more compared to what I used to do.
Ok, thanks for clarifying. (To reiterate my later point, since it sounds like you’re considering the “narrative pyramid schemes” hypothesis: I think there is not common knowledge that narrative pyramid schemes happen, and that common knowledge might help people continuously and across contexts share more information, especially information that is pulling against the pyramid schemes, by giving them more of a true expectation that they’ll be heard by a something-maximizing person rather than a narrative-executer).
I have concrete thoughts about the specific etiquette of such conversations (they’re not off the cuff; I’ve been thinking more-or-less continuously about this sort of thing for about eight years now).
However, I’m going to hold off for a bit because:
a) Like Anna, I was a part of the dynamics surrounding PhoenixFriend’s experience, and so I don’t want to seize the reins
b) I’ve also had a hard time coordinating with Anna on conversational norms and practices, both while at CFAR and recently
… so I sort of want to not-pretend-I-don’t-have-models-and-opinions-here (I do) but also do something like “wait several days and let other people propose things first” or “wait until directly asked, having made it clear that I have thoughts if people want them” or something.
Goal-Factoring was first called “use fungibility”, a technique I taught within a class called “microeconomics 1” at the CFAR 2012 minicamps prior to Geoff doing any teaching.
As a participant of Rationality Minicamp in 2012, I confirm this. Actually, found the old textbook, look here!
Okay, so, that old textbook does not look like a picture of goal-factoring, at least not on that page. But I typed “goal-factoring” into my google drive and got up these old notes that used the word while designing classes for the 2012 minicamps. A rabbithole, but one I enjoyed so maybe others will.
Thank you for adding your detailed take/observations.
My own take on some of the details of CFAR that’re discussed in your comment:
I think there were serious problems here, though our estimates of the frequencies might differ. To describe the overall situation in detail:
I often got debugging help from other members of CFAR, but, as noted in the quote, it was voluntary. I picked when and about what and did not feel pressure to do so.
I can think of at least three people at CFAR who had a lot of debugging sort of forced on them (visibly expected as part of their job set-up or of check-in meetings or similar; they didn’t make clear complaints but that is still “sort of forced”), in ways that were large and that seem to me clearly not okay in hindsight. I think lots of other people mostly did not experience this. There are a fair number of people about whom I am not sure or would make an in-between guess. To be clear, I think this was bad (predictably harmful, in ways I didn’t quite get at the time but that e.g. standard ethical guidelines in therapy have long known about), and I regret it and intend to avoid “people doing extensive debugging of those they have direct power over” contexts going forward.
I believe this sort of problem was more present in the early years, and less true as CFAR became older, better structured, somewhat “more professional”, and less centered around me. In particular, I think Pete’s becoming ED helped quite a bit. I also think the current regime (“holocracy”) has basically none of this, and is structured so as to predictably have basically none of this—predictably, since there’s not much in the way of power imbalances now.
It’s plausible I’m wrong about how much of this happened, and how bad it was, in different eras. In particular, it is easy for those in power (e.g., me) to underestimate aspects of how bad it is not to have power; and I did not do much to try to work around the natural blindspot. If anyone wants to undertake a survey of CFAR’s past and present staff on this point (ideally someone folks know and can accurately trust to maintain their anonymity while aggregating their data, say, and then posting the results to LW), I’d be glad to get email addresses for CFAR’s past and present staff for the purpose.
I’m sure I did not describe my process as “implanting an engine of desperation”; I don’t remember that and it doesn’t seem like a way I would choose to describe what I was doing. “Implanting” especially doesn’t. As Eli notes (this hadn’t occurred to me, but might be what you’re thinking of?), I did talk some about trying to get in touch with one’s “quiet desperation”, and referenced Pink Floyd’s song “Time” and “the mass of men lead lives of quiet desperation” and developed concepts around that; but this was about accessing a thing that was already there, not “implanting” a thing. I also led many people in “internal double cruxes around existential risk”, which often caused fairly big reactions as people viscerally noticed “we might all die.”
I disagree with this point overall. Goal-Factoring was first called “use fungibility”, a technique I taught within a class called “microeconomics 1” at the CFAR 2012 minicamps prior to Geoff doing any teaching. It was also discussed at times in some form at the old SingInst visiting fellows program, IIRC.
Geoff developed it, and taught it at many CFAR workshops in early years (2013-2014, I think). The choice that it was Goal-Factoring that Geoff (was asked to teach? wanted to teach? I don’t actually remember; probably both?) was I think partly to do with its resemblance to the beginning/repeated basic move in Connection Theory.
My guess is that there were asymmetries like this, and that they were important, and that they were not worse than most organizations (though that’s really not the right benchmark). Insofar as you have experience at other organizations (e.g. mainstream tech companies or whatnot), or have friends with such experience who you can ask questions of, I am curious how you think they compare.
On my own list of “things I would do really differently if I was back in 2012 starting CFAR again”, the top-ranked item is probably:
Share information widely among staff, rather than (mostly unconsciously/not-that-endorsedly) using lack-of-information-sharing to try to control people and outcomes.
Do consider myself to have some duty to explain decisions and reply to questions. Not “before acting”, because the show must go on and attempts to reach consensus would be endless. And not “with others as an authority that can prevent me from acting if they don’t agree.” But yes with a sincere attempt to communicate my actual beliefs and causes of actions, and to hear others’ replies, insofar as time permits.
I don’t think I did worse than typical organizations in the wider world, on the above points.
I’m honestly uncertain how much this is/isn’t related to the quoted complaint.
Duncan’s reply here is probably more accurate to the actual situation at CFAR than mine would be. (I wrote much of the previous paragraphs before seeing his, but endorsing Duncan’s on this here seems best.) If Pete wants to weigh in I would also take his perspective quite seriously here. I don’t quite remember some of the details.
As Duncan noted, “creating a state of emotional vulnerability and openness” is really not supposed to be the point of circling, but it is a thing that happens pretty often and that a person might not know how to avoid.
The point of circling IMO is to break all the fourth walls that conversations often skirt around, let the subtext or manner in which the conversation is being done be made explicit text, and let it all thereby be looked at together.
A different thing that I in hindsight think was an error (that I already had on my explicit list of “things to do differently going forward”, and had mentioned in this light to a few people) was using circling in the way we did at AIRCS workshops, where some folks were there to try to get jobs. My current view, as mentioned a bit above, is that something pretty powerfully bad sometimes happens when a person accesses bits of their insides (in the way that e.g. therapy or some self-help techniques lead people to) while also believing they need to please an external party who is looking at them and has power over them.
(My guess is that well-facilitated circling is fine at AIRCS-like programs that are less directly recruiting-oriented. Also that circling at AIRCS had huge upsides. This is a can of worms I don’t plan to go into right now, in the middle of this comment reply, but flagging it to make my above paragraph not overgenralized-from.)
I believe this was your experience, and am sorry. My non-confident guess is that some others experienced this and most didn’t, and that the impact on folks’ mental privacy was considerably more invasive than a standard workplace would’ve been, and that the impact on folks’ integrity was probably less bad than my guess at many mainstream workplace’s impact but still a lot worse than the CFAR we ought to aim for.
Personally I am not much trying to maintain the privacy of my own mind at this point, but I am certainly trying to maintain its integrity, and I think being debugged by people with power over me would not be good for that.
This wasn’t my experience at all, personally. I did have some feeling of distance when I first started caring about AI risk in ~2008, but it didn’t get worse across CFAR. I also stayed in a lot of contact with folks outside the CFAR / EA / rationalist / AI risk spheres through almost all of it. I don’t think I looked down on outsiders.
I thought CFAR and MIRI were part of a rare and important thing, but I did not think CFAR (nor CFAR + MIRI) was the only thing to matter. I do think there’s some truth in the “rarity narrative” claim, at CFAR, mostly via me and to a much smaller extent some others at CFAR having some of this view of MIRI.
I agree that this happened and that it was a problem. I didn’t consciously intend to set this up, but my guess is that I did a bunch of things to cause it anyhow. In particular, there’s a certain way I used to sort of take the ground out from under people when we talked, that I think contributed to this. (I used to often do something like: stay cagey about my own opinions; listen carefully to how my interlocutor was modeling the world; show bits of evidence that refuted some of their assumptions; listen to their new model; repeat; … without showing my work. And then they would defer to me, instead of having stubborn opinions I didn’t know how to shift, which on some level was what I wanted.)
People at current-CFAR respect my views still, but it actually feels way healthier to me now. Partly because I’m letting my own views and their causes be more visible, which I think makes it easier to respond to. And because I somehow have less of a feeling of needing to control what other people think or do via changing their views.
(I haven’t checked the above much against others’ perceptions, so would be curious for anyone from current or past CFAR with a take.)
I believe this was your experience, mostly because I’m pretty sure I know who you are (sorry; I didn’t mean to know and won’t make it public) and I can think of at least one over-the-top (but sincere) conversation you could reasonably describe at least sort of this way (except for the “with high confidence”, I guess, and the “frequent”; and some other bits), plus some repeated conflicts. I don’t think this was a common experience, or that it happened much at all (or at all at all?) in contexts not involving you, but it’s possible I’m being an idiot here somehow in which case someone should speak up. Which I guess is to say that the above bullet point seems to me, from my experiences/observations, to be mostly or almost-entirely false, but that I think you’re describing your experiences and guesses about the place accurately and that I appreciate you speaking up.
Anyhow, thanks for writing, and I’m sorry you had bad experiences at CFAR, especially about the fairly substantial parts of the above bad parts that were my fault.
I expect my reply will accidentally make some true points you’re making harder to see (as well as hopefully adding light to some other parts), and I hope you’ll push back in those places.
Related to my reply to PhoenixFriend (in the parent comment), but hopping meta from it:
I have a question for whoever out there thinks they know how the etiquette of this kind of conversation should go. I had a first draft of my reply to PhoenixFriend, where I … basically tried to err on the side of being welcoming, looking for and affirming the elements of truth I could hear in what PhoenixFriend had written, and sort of emphasizing those elements more than my also-real disagreements. I ran it by a CFAR colleague at my colleague’s request, who said something like “look, I think your reply is pretty misleading; you should be louder and clearer about the ways your best guess about what happened differed from what’s described in PhoenixFriend’s comment. Especially since I and others at CFAR have our names on the organization too, so if you phrase things in ways that’ll cause strangers who’re skim-reading to guess that things at CFAR were worse than they were, you’ll inaccurately and unjustly mess with other peoples’ reputations too.” (Paraphrased.)
So then I went back and made my comments more disagreeable and full of details about where my and PhoenixFriend’s models differ. (Though probably still less than the amount that would’ve fully addressed my colleague’s complaints.)
This… seems better in that it addresses my colleague’s pretty reasonable desire, but worse in that it is not welcoming to someone who is trying to share info and is probably finding that hard. I am curious if anyone has good thoughts on how this sort of etiquette should go, if we want to have an illuminating, get-it-all-out-there, non-misleading conversation.
Part of why I’m worried, is it seems to me pretty easy for people who basically think the existing organizations are good, and also that mainstream workplaces are non-damaging and so on, to upvote/downvote each new datum based on those priors plus a (sane and sensible) desire to avoid hurting others’ feelings and reputations without due cause, etc., in ways that despite their reasonability may make it hard for real and needed conversations that are contrary to our current patterns of seeing to get started.
For example, I think PhoenixFriend indeed saw some real things at CFAR that many of those downvoting their comment did not see and mistakenly wouldn’t expect to see, but that also many of the details of PhoenixFriend’s comment are off, partly maybe because they were mis-generalizing from their experiences and partly because it’s hard to name things exactly (especially to people who have a bit of an incentive to mishear.)
(Also, to try briefly and poorly to spell out why I’m rooting for a “get it all out on the table” conversation, and not just a more limited “hear and acknowledge the mostly blatant/known harms, correct those where possible, and leave the rest of our reputation intact” conversation: basically, I think there’s a bunch of built-up “technical debt”, in the form of confusion and mistrust and trying-not-to-talk-about-particular-things-because-others-will-form-“unreasonable”-conflusions-if-we-do and who-knows-why-we-do-that-but-we-do-so-there’s-probably-a-reason, that I’m hoping gets cleared out by the long and IMO relatively high-quality and contentful conversation that’s been happening so far. I want more of that if we can get it. I want culture and groups to be able to build around here without building on top of technical debt. I also want information about how organizations do/don’t work well, and, in terms of means of acquiring this information, I much prefer bad-looking conversations on LW to wasting another five years doing it wrong.)
This sounds like an extreme and surprising statement. I wrote out some clarifying questions like “what do you mean by privacy here”, but maybe it’d be better to just say:
I think it strikes me funny because it sounds sort of like a PR statement. And it sounds like a statement that could set up a sort of “iterations of the Matrix”-like effect. Where, you say “ok now I want to clear out all the miasma, for real”, and then you and your collaborators do a pretty good job at that; but also, something’s been lost or never gained, namely the logical common knowledge that there’s probably-ongoing, probably difficult to see dynamics that give rise to the miasma of {ungrounded shared narrative, information cascades, collective blindspots, deferrals, circular deferrals, misplaced/miscalibrated trust, etc. ??}. In other words, since these things happened in a context where you and your collaborators were already using reflection, introspection, reasoning, communication, etc., we learn that the ongoing accumulation of miasma is a more permanent state of affairs, and this should be common knowledge. Common knowledge would for example help with people being able to bring up information about these dynamics, and expect their information to be put to good use.
(I notice an analogy between iterations of the Matrix and economic boom-bust cycles.)
These statements also seem to imply a framing that potentially has the (presumably unintentional) effect of subtly undermining the common knowledge of ongoing miasma-or-whatever. Like, it sort of directs attention to the content but not the generator, or something; like, one could go through all the “stuff” and then one would be done.
Well, maybe I phrased it poorly; I don’t think what I’m doing is extreme; “much” is doing a bunch of work in my “I am not much trying to...” sentence.
I mean, there’s plenty I don’t want to share, like a normal person. I have confidential info of other peoples that I’m committed to not sharing, and plenty of my own stuff that I am private about for whatever reason. But in terms of rough structural properties of my mind, or most of my beliefs, I’m not much trying for privacy. Like when I imagine being in a context where a bunch of circling is happening or something (circling allows silence/ignoring questions/etc..; still, people sometimes complain that facial expressions leak through and they don’t know how to avoid it), I’m not personally like “I need my privacy though.” And I’ve updated some toward sharing more compared to what I used to do.
Ok, thanks for clarifying. (To reiterate my later point, since it sounds like you’re considering the “narrative pyramid schemes” hypothesis: I think there is not common knowledge that narrative pyramid schemes happen, and that common knowledge might help people continuously and across contexts share more information, especially information that is pulling against the pyramid schemes, by giving them more of a true expectation that they’ll be heard by a something-maximizing person rather than a narrative-executer).
I have concrete thoughts about the specific etiquette of such conversations (they’re not off the cuff; I’ve been thinking more-or-less continuously about this sort of thing for about eight years now).
However, I’m going to hold off for a bit because:
a) Like Anna, I was a part of the dynamics surrounding PhoenixFriend’s experience, and so I don’t want to seize the reins
b) I’ve also had a hard time coordinating with Anna on conversational norms and practices, both while at CFAR and recently
… so I sort of want to not-pretend-I-don’t-have-models-and-opinions-here (I do) but also do something like “wait several days and let other people propose things first” or “wait until directly asked, having made it clear that I have thoughts if people want them” or something.
link to the essay if/when you write it?
I endorse Anna’s commentary.
As a participant of Rationality Minicamp in 2012, I confirm this. Actually, found the old textbook, look here!
Okay, so, that old textbook does not look like a picture of goal-factoring, at least not on that page. But I typed “goal-factoring” into my google drive and got up these old notes that used the word while designing classes for the 2012 minicamps. A rabbithole, but one I enjoyed so maybe others will.