I see this kind of argument a lot, but to my thinking, the next iteration of AI researchers will only have the tools today’s researchers build for them. You’re not trying to compete with them. You’re trying to empower them.
Sure, this sounds like what I was saying. I was trying to say something like “We should mostly focus on ensuring that future AI safety researchers can safely and productively use these early transformative AIs and ensuring these early transformative AIs don’t pose other direct risks and then safety researchers in this period can worry about safety for the next generation of more powerful models.”
Separately, it’s worth noting that many general purpose tools for productively using AIs (for research) will be built with non-safety motivations, so safety researchers don’t necessarily need to invest in building general purpose tools.
are we so resource constrained that putting more effort into whatever alignment research we can try today is actually net-negative
I’m confused about what you’re responding to here.
To the latter: my point is that except to the extent we’re resource constrained, I’m not sure why anyone (and I’m not saying you are necessarily) would argue against any safe line of research even if they thought it was unlikely to work.
To the former: I think one of the things we can usefully bestow on future researchers (in any field) is a pile of lines of inquiry, including ones that failed and ones we realized we couldn’t properly investigate yet, and ones where we made even a tiny bit of headway.
my point is that except to the extent we’re resource constrained, I’m not sure why anyone (and I’m not saying you are necessarily) would argue against any safe line of research even if they thought it was unlikely to work.
I mean, all claims that research X is good are claims that X is relatively good compared to the existing alternatives Y. That doesn’t mean that you should only do X, probably should diversify in many cases.
We absolutely do have resource contraints: many good directions aren’t currently being explored because there are even better directions.
Sure, this sounds like what I was saying. I was trying to say something like “We should mostly focus on ensuring that future AI safety researchers can safely and productively use these early transformative AIs and ensuring these early transformative AIs don’t pose other direct risks and then safety researchers in this period can worry about safety for the next generation of more powerful models.”
Separately, it’s worth noting that many general purpose tools for productively using AIs (for research) will be built with non-safety motivations, so safety researchers don’t necessarily need to invest in building general purpose tools.
I’m confused about what you’re responding to here.
To the latter: my point is that except to the extent we’re resource constrained, I’m not sure why anyone (and I’m not saying you are necessarily) would argue against any safe line of research even if they thought it was unlikely to work.
To the former: I think one of the things we can usefully bestow on future researchers (in any field) is a pile of lines of inquiry, including ones that failed and ones we realized we couldn’t properly investigate yet, and ones where we made even a tiny bit of headway.
I mean, all claims that research X is good are claims that X is relatively good compared to the existing alternatives Y. That doesn’t mean that you should only do X, probably should diversify in many cases.
We absolutely do have resource contraints: many good directions aren’t currently being explored because there are even better directions.