Looking at a dialogue like this one, I am tempted to try and steelman the mental motion that that wants to defend the status hierarchy from upstarts. Like, when I step into Pat or Maude’s mental shoes, what do I want to defend?How are they right?
(This seems like exactly the sort of situation where they’re action is attempting to avoid a bucket error. Can I disentangle the buckets and get the value of what Eliezer is pushing for while defending the thing that matters to Pat and Maude?)
When I do that I’m reminded of an unusual meetup I attended some years ago, hosted by a rationalist adjacent meditation-expert. Most of the attendees were rationalists, but one person was a newbie, a new-agey type that the host had met on okcupid. I only talked with them a little, but I got the sense that they maybe believed in the literal physical reality of psychic energy and whatnot.
One of the rationalist (a academic himself) was talking about the ways science is broken: p-hacking, and replication crises, and misaligned incentives. I thought all of this was correct, but I was also concerned about the impact it was likely to have on the new-agey person. I was concerned that they would take this as “science is unreliable, and I don’t need to heed it.” So I kept emphasizing that, “yes, science has all of these issues, but also, it is the best we have, and the scientific consensus, while often wrong or flawed, is our collective best guess about what’s true.” [Not a statement I would agree with today, for what it’s worth.]
That context was at least a little bit a theater of a culture war, and arguments are soldiers. It felt important to me that the new-ager did not conclude from our conversation (with an actual scientist!) that “science doesn’t know anything.” I felt like I was straddling both sides of a valley of bad rationality: on the one side “‘materialistic, atheistic science’ is biased, and can’t account for the supernatural”, in the middle, the skeptic movement’s promotion of science, and on the other side “wait, actually science is pretty broken.”
If people go around saying that science as an institution is inadequate, and that amateurs can make progress on important problems, then bystanders are apt to conclude that they don’t need to respect the institution of science...and can therefore believe whatever they want.
Like, most people don’t have the intelligence or rationality to distinguish between the places where science is trustworthy, and you really do just need to study the textbooks, and the places where “science” is unreliable, and one can make progress outside the hallowed halls of academia. So most people have a single switch that can be toggled back and forth: “trust science” vs. “don’t trust science.” If those are my only two options, I really do want most people to be flipped to “trust science.”
This gives me some hint about what’s up for Maud, but it doesn’t really explain Pat.
When I step into Pat, I get something like, “God-damn it, don’t belittle the existing authors, by asserting that you’re better than them!” Like there is an actually important status/credit allocation problem, where we want to allocate social reward to the people that do good work. Claims that you’re going to blow the good existing work out of the water, seem to upset that credit allocation, sort of like you threatened to reduce Nonjon’s salary, or something. Part of me is like “they worked hard for their social standing. You don’t get to steal some of it just by making unsubstantiated claims!”
I’m imagining a bunch of people at a party, and Eliezer getting a lot of attention for his claims about what he’s going to do, while authors who have actually done good work (which of course doesn’t sound as good as Eliezer’s stories about how awesome his working is going to be, because there are no constraints on how awesome something you’re going to write will be). It’s like Eliezer is trying to get social credit with big, cheap, claims, at the cost of someone who actually made something good.
If you expect that there are people around who will be swayed and impressed by those big insubstantial claims, it might annoy you, or piss you off. It makes sense to want to smack them down. (Whereas if you feel no one would pay attention to this crackpot anyway, then it is sufficient to just ignore and, maybe snicker, at them.)
Hmm...
Looking at a dialogue like this one, I am tempted to try and steelman the mental motion that that wants to defend the status hierarchy from upstarts. Like, when I step into Pat or Maude’s mental shoes, what do I want to defend?How are they right?
(This seems like exactly the sort of situation where they’re action is attempting to avoid a bucket error. Can I disentangle the buckets and get the value of what Eliezer is pushing for while defending the thing that matters to Pat and Maude?)
When I do that I’m reminded of an unusual meetup I attended some years ago, hosted by a rationalist adjacent meditation-expert. Most of the attendees were rationalists, but one person was a newbie, a new-agey type that the host had met on okcupid. I only talked with them a little, but I got the sense that they maybe believed in the literal physical reality of psychic energy and whatnot.
One of the rationalist (a academic himself) was talking about the ways science is broken: p-hacking, and replication crises, and misaligned incentives. I thought all of this was correct, but I was also concerned about the impact it was likely to have on the new-agey person. I was concerned that they would take this as “science is unreliable, and I don’t need to heed it.” So I kept emphasizing that, “yes, science has all of these issues, but also, it is the best we have, and the scientific consensus, while often wrong or flawed, is our collective best guess about what’s true.” [Not a statement I would agree with today, for what it’s worth.]
That context was at least a little bit a theater of a culture war, and arguments are soldiers. It felt important to me that the new-ager did not conclude from our conversation (with an actual scientist!) that “science doesn’t know anything.” I felt like I was straddling both sides of a valley of bad rationality: on the one side “‘materialistic, atheistic science’ is biased, and can’t account for the supernatural”, in the middle, the skeptic movement’s promotion of science, and on the other side “wait, actually science is pretty broken.”
If people go around saying that science as an institution is inadequate, and that amateurs can make progress on important problems, then bystanders are apt to conclude that they don’t need to respect the institution of science...and can therefore believe whatever they want.
Like, most people don’t have the intelligence or rationality to distinguish between the places where science is trustworthy, and you really do just need to study the textbooks, and the places where “science” is unreliable, and one can make progress outside the hallowed halls of academia. So most people have a single switch that can be toggled back and forth: “trust science” vs. “don’t trust science.” If those are my only two options, I really do want most people to be flipped to “trust science.”
This gives me some hint about what’s up for Maud, but it doesn’t really explain Pat.
When I step into Pat, I get something like, “God-damn it, don’t belittle the existing authors, by asserting that you’re better than them!” Like there is an actually important status/credit allocation problem, where we want to allocate social reward to the people that do good work. Claims that you’re going to blow the good existing work out of the water, seem to upset that credit allocation, sort of like you threatened to reduce Nonjon’s salary, or something. Part of me is like “they worked hard for their social standing. You don’t get to steal some of it just by making unsubstantiated claims!”
I’m imagining a bunch of people at a party, and Eliezer getting a lot of attention for his claims about what he’s going to do, while authors who have actually done good work (which of course doesn’t sound as good as Eliezer’s stories about how awesome his working is going to be, because there are no constraints on how awesome something you’re going to write will be). It’s like Eliezer is trying to get social credit with big, cheap, claims, at the cost of someone who actually made something good.
If you expect that there are people around who will be swayed and impressed by those big insubstantial claims, it might annoy you, or piss you off. It makes sense to want to smack them down. (Whereas if you feel no one would pay attention to this crackpot anyway, then it is sufficient to just ignore and, maybe snicker, at them.)