I think I agree with the models here, and also want to add a complicating factor that I think impacts the relevance of this.
I think running a site like this in a fully consequentialist way is bad. When you’re public and a seed of something, you want to have an easily-understandable interface with the world; you want it to be the case that other people who reason about you (of which there will be many, and who are crucial to your plan’s success!) can easily reason about you. Something more like deontology or virtue ethics (“these are the rules I will follow” or “these are the virtues we will seek to embody”) makes it much easier for other agents to reason about you.
And so the more that I as a mod (or the mod team in general) rely on our individual prudence or models or so on, the more difficult it becomes for users to predict what will happen, and that has costs. (I still think that it ultimately comes down to our prudence—the virtues that we’re trying to embody do in fact conflict sometimes, and it’s not obvious how to resolve those conflicts—but one of the things my prudence is considering are those legibility costs.)
And when we try to figure out what virtues we should embody on Less Wrong, I feel much better about Rationality: Common Interest of Many Causes than I do about “whatever promotes AI safety”, even tho I think the ‘common interest’ dream didn’t turn out as well as one might have hoped, looking forward from 2009, and I think AI safety is much closer to ‘the only game in town’ than it might seem on first glance. Like, I want us to be able to recover if in fact it turns out AI safety isn’t that big a deal. I also want LessWrong to survive as a concern even if someone figures out AI safety, in a way that I might not for something like the AI Alignment Forum. I would like people who aren’t in tune with x-risk to still be around here (so long as they make the place better).
That said, as pointed out in my other comment, I care more about reaching the heights than I do about raising the sanity waterline or w/e, and I suspect that lines up with “better babble” more than it does “better prune”.
+1 to all this, and in particular I’m very strongly on board with rationality going beyond AI safety. I’m a big fan of LessWrong’s current nominal mission to “accelerate intellectual progress”, and when I’m thinking about making progress in a high-dimensional world, that’s usually the kind of progress I’m thinking about. (… Which, in turn, is largely because intellectual/scientific/engineering progress seem to be the “directions” which matter most for everything else.)
I think I agree with the models here, and also want to add a complicating factor that I think impacts the relevance of this.
I think running a site like this in a fully consequentialist way is bad. When you’re public and a seed of something, you want to have an easily-understandable interface with the world; you want it to be the case that other people who reason about you (of which there will be many, and who are crucial to your plan’s success!) can easily reason about you. Something more like deontology or virtue ethics (“these are the rules I will follow” or “these are the virtues we will seek to embody”) makes it much easier for other agents to reason about you.
And so the more that I as a mod (or the mod team in general) rely on our individual prudence or models or so on, the more difficult it becomes for users to predict what will happen, and that has costs. (I still think that it ultimately comes down to our prudence—the virtues that we’re trying to embody do in fact conflict sometimes, and it’s not obvious how to resolve those conflicts—but one of the things my prudence is considering are those legibility costs.)
And when we try to figure out what virtues we should embody on Less Wrong, I feel much better about Rationality: Common Interest of Many Causes than I do about “whatever promotes AI safety”, even tho I think the ‘common interest’ dream didn’t turn out as well as one might have hoped, looking forward from 2009, and I think AI safety is much closer to ‘the only game in town’ than it might seem on first glance. Like, I want us to be able to recover if in fact it turns out AI safety isn’t that big a deal. I also want LessWrong to survive as a concern even if someone figures out AI safety, in a way that I might not for something like the AI Alignment Forum. I would like people who aren’t in tune with x-risk to still be around here (so long as they make the place better).
That said, as pointed out in my other comment, I care more about reaching the heights than I do about raising the sanity waterline or w/e, and I suspect that lines up with “better babble” more than it does “better prune”.
+1 to all this, and in particular I’m very strongly on board with rationality going beyond AI safety. I’m a big fan of LessWrong’s current nominal mission to “accelerate intellectual progress”, and when I’m thinking about making progress in a high-dimensional world, that’s usually the kind of progress I’m thinking about. (… Which, in turn, is largely because intellectual/scientific/engineering progress seem to be the “directions” which matter most for everything else.)