I think it’s good to sometimes meditate on whether you are making the world worse (and get others’ advice), and I’d more often recommend it for crowds other than EA and certainly wouldn’t discourage people from doing it sometimes.
I’m sympathetic to arguments that you should be super paranoid in domains like biosecurity since it honestly does seem asymmetrically easier to make things worse rather than better. But when people talk about it in the context of e.g. AI or policy interventions or gathering better knowledge about the world that might also have some negative side-effects, I often feel like there’s little chance that predictable negative effects they are imagining loom large in the cost-benefit unless the whole thing is predictably pointless. Which isn’t a reason not to consider those effects, just a push-back against the conclusion (and a heuristic push-back against the state of affairs where people are paralyzed by the possibility of negative consequences based on kind of tentative arguments).
For advancing or deploying AI I generally have an attitude like “Even if actively trying to push the field forward full-time I’d be a small part of that effort, whereas I’m a much larger fraction of the stuff-that-we-would-be-sad-about-not-happening-if-the-field-went-faster, and I’m not trying to push the field forward,” so while I’m on board with being particularly attentive to harms if you’re in a field you think can easily cause massive harms, in this case I feel pretty comfortable about the expected cost-benefit unless alignment work isn’t really helping much (in which case I have more important reasons not to work on it). I would feel differently about this if pushing AI faster was net bad on e.g. some common-sense perspective on which alignment was not very helpful, but I feel like I’ve engaged enough with those perspectives to be mostly not having it.
“Even if actively trying to push the field forward full-time I’d be a small part of that effort”
I think conditioning on something like ‘we’re broadly correct about AI safety’ implies ‘we’re right about some important things about how AI development will go that the rest of the ML community is surprisingly wrong about’. In that world we’re maybe able to contribute as much as a much larger fraction of the field, due to being correct about some things that everyone else is wrong about.
I think your overall point still stands, but it does seem like you sometimes overestimate how obvious things are to the rest of the ML community
I think it’s good to sometimes meditate on whether you are making the world worse (and get others’ advice), and I’d more often recommend it for crowds other than EA and certainly wouldn’t discourage people from doing it sometimes.
I’m sympathetic to arguments that you should be super paranoid in domains like biosecurity since it honestly does seem asymmetrically easier to make things worse rather than better. But when people talk about it in the context of e.g. AI or policy interventions or gathering better knowledge about the world that might also have some negative side-effects, I often feel like there’s little chance that predictable negative effects they are imagining loom large in the cost-benefit unless the whole thing is predictably pointless. Which isn’t a reason not to consider those effects, just a push-back against the conclusion (and a heuristic push-back against the state of affairs where people are paralyzed by the possibility of negative consequences based on kind of tentative arguments).
For advancing or deploying AI I generally have an attitude like “Even if actively trying to push the field forward full-time I’d be a small part of that effort, whereas I’m a much larger fraction of the stuff-that-we-would-be-sad-about-not-happening-if-the-field-went-faster, and I’m not trying to push the field forward,” so while I’m on board with being particularly attentive to harms if you’re in a field you think can easily cause massive harms, in this case I feel pretty comfortable about the expected cost-benefit unless alignment work isn’t really helping much (in which case I have more important reasons not to work on it). I would feel differently about this if pushing AI faster was net bad on e.g. some common-sense perspective on which alignment was not very helpful, but I feel like I’ve engaged enough with those perspectives to be mostly not having it.
“Even if actively trying to push the field forward full-time I’d be a small part of that effort”
I think conditioning on something like ‘we’re broadly correct about AI safety’ implies ‘we’re right about some important things about how AI development will go that the rest of the ML community is surprisingly wrong about’. In that world we’re maybe able to contribute as much as a much larger fraction of the field, due to being correct about some things that everyone else is wrong about.
I think your overall point still stands, but it does seem like you sometimes overestimate how obvious things are to the rest of the ML community