Crucially, assurances that the power structure is trying optimize for something good, are not rules. I’m sure the judges of the Inquisition or Soviet show trials would have told you that they weren’t exercising power arbitrarily, because they were making judgements about an external standard—the platonic ideal of God’s will, or the common good. I’m sure they were being perfectly sincere about that. The rule of law is about imposing more stringent limits on the use of power than an authority figure’s subjectively sincere allegiance to an abstract ideal that isn’t written down.
FWIW, my own take here is indeed that we should try to get some of the benefits of the rule of law (and indeed that one of the central components here is putting limits on the power of the moderators), but that an online forum should aspire to a much lower standard of justice than a country, and should be closer to the standard that we hold companies to (where at least in the U.S. you have things like at-will employment and a broad understanding that it’s often the right choice to fire someone even if they didn’t do anything legibly bad). I don’t feel super confident on this though.
Company/corporate cultures are hardly a good model to emulate if we want to optimize for truth-seeking, as such cultures famously select for distortions of truth, often lack any incentives for truth-seeking and truth-telling, and generally reward sociopathy (instrumentally) and bullshit (epistemically) to an appalling degree. Is that really the standard to aim for, here?
I mean, really depends on which company. The variance between different companies here is huge. My current model is that both the world’s best teams and cultures are located in for-profit companies, as well as the world’s worst epistemic environments. So I find it very hard to speak in generalities here (and think it’s somewhat obviously wrong to claim that for-profit companies in-general select for distortions of truth).
When you say that “world’s best teams and cultures are located in for-profit companies”, what companies do you have in mind? SpaceX? Google? Jane Street…?
Bell Labs is the classical example here, as clearly one of history’s most intellectually generative places.
It also appears that sadly, at least for the purpose of people really understanding engineering and computer science, Deepmind appears to be a quite good place for thinking, as have some other parts of Google been.
Bridgewater also seems quite good as far as I can tell. At least when I’ve talked to people who worked there, and when I read Ray Dalio’s work, I get a pretty good impression, though it’s of course hard to tell from the outside (I remember you shared an article like 4 years ago with some concerns, though I don’t currently buy the contents of that).
(Note: this comment delayed by rate limit. Next comment on this topic, if any, won’t be for a week, for the same reason.)
Very ironic! I had all three of those in mind as counterexamples to your claim. (Well, not Deepmind specifically, but Google in general; but the other two for sure.)
Bell Labs was indeed “one of history’s most intellectually generative places”. But the striking thing about Bell Labs (and similarly Xerox PARC, and IBM Research) is the extent to which the people working there were isolated from ordinary corporate politics, corporate pressures, and day-to-day business concerns. In other words, these corporate research labs are notable precisely for being enclaves within which corporate/company culture essentially does not operate.
As far as Google and/or Deepmind goes, well… I don’t know enough about Deepmind in particular to comment on it. But Google, in general, is famous for being a place where fixing/improving things is low-prestige, and the way to get ahead is to be seen as developing shiny new features/products/etc. This has predictable consequences for, e.g., usability (Google’s products are infamous for having absolutely horrific interaction and UX design—Google Plus being one egregious example). Everything I’ve heard about Google indicates that the stereotypical “moral maze” dynamics of corporate culture are in full swing there.
Re: Bridgewater, you remember correctly, although “some concerns” is rather an understatement; it’s more like “the place is a real-life Orwellian panopticon, with all the crushing stress and social/psychological dysfunction that implies”. Even more damning is that they never even bother to verify that all of this helps their investing performance in any way. This seems to me to be very obviously the opposite of a healthy epistemic environment—something to avoid as assiduously as we possibly can.
FWIW, my own take here is indeed that we should try to get some of the benefits of the rule of law (and indeed that one of the central components here is putting limits on the power of the moderators), but that an online forum should aspire to a much lower standard of justice than a country, and should be closer to the standard that we hold companies to (where at least in the U.S. you have things like at-will employment and a broad understanding that it’s often the right choice to fire someone even if they didn’t do anything legibly bad). I don’t feel super confident on this though.
Company/corporate cultures are hardly a good model to emulate if we want to optimize for truth-seeking, as such cultures famously select for distortions of truth, often lack any incentives for truth-seeking and truth-telling, and generally reward sociopathy (instrumentally) and bullshit (epistemically) to an appalling degree. Is that really the standard to aim for, here?
I mean, really depends on which company. The variance between different companies here is huge. My current model is that both the world’s best teams and cultures are located in for-profit companies, as well as the world’s worst epistemic environments. So I find it very hard to speak in generalities here (and think it’s somewhat obviously wrong to claim that for-profit companies in-general select for distortions of truth).
When you say that “world’s best teams and cultures are located in for-profit companies”, what companies do you have in mind? SpaceX? Google? Jane Street…?
Bell Labs is the classical example here, as clearly one of history’s most intellectually generative places.
It also appears that sadly, at least for the purpose of people really understanding engineering and computer science, Deepmind appears to be a quite good place for thinking, as have some other parts of Google been.
Bridgewater also seems quite good as far as I can tell. At least when I’ve talked to people who worked there, and when I read Ray Dalio’s work, I get a pretty good impression, though it’s of course hard to tell from the outside (I remember you shared an article like 4 years ago with some concerns, though I don’t currently buy the contents of that).
(Note: this comment delayed by rate limit. Next comment on this topic, if any, won’t be for a week, for the same reason.)
Very ironic! I had all three of those in mind as counterexamples to your claim. (Well, not Deepmind specifically, but Google in general; but the other two for sure.)
Bell Labs was indeed “one of history’s most intellectually generative places”. But the striking thing about Bell Labs (and similarly Xerox PARC, and IBM Research) is the extent to which the people working there were isolated from ordinary corporate politics, corporate pressures, and day-to-day business concerns. In other words, these corporate research labs are notable precisely for being enclaves within which corporate/company culture essentially does not operate.
As far as Google and/or Deepmind goes, well… I don’t know enough about Deepmind in particular to comment on it. But Google, in general, is famous for being a place where fixing/improving things is low-prestige, and the way to get ahead is to be seen as developing shiny new features/products/etc. This has predictable consequences for, e.g., usability (Google’s products are infamous for having absolutely horrific interaction and UX design—Google Plus being one egregious example). Everything I’ve heard about Google indicates that the stereotypical “moral maze” dynamics of corporate culture are in full swing there.
Re: Bridgewater, you remember correctly, although “some concerns” is rather an understatement; it’s more like “the place is a real-life Orwellian panopticon, with all the crushing stress and social/psychological dysfunction that implies”. Even more damning is that they never even bother to verify that all of this helps their investing performance in any way. This seems to me to be very obviously the opposite of a healthy epistemic environment—something to avoid as assiduously as we possibly can.
Xerox PARC also had some impressive achievements.