I interpret your comment as assuming that new researchers with good ideas produce more impact on their own than in teams working towards a shared goal
I don’t believe that, although I see how my summary could be interpreted that way. I agree with basically all the reasons in your recent comment and most in the original comment. I could add a few reasons of my own doing independent grant-funded work sucks. But I think it’s really important to track how founding projects tracks to increased potential safety instead of intermediates, and push hard against potential tail wagging the dog scenarios.
I was trying to figure out why this was important to me, given how many of your points I agree with. I think it’s a few things:
Alignment work seems to be prone to wagging the dog, and is harder to correct, due to poor feedback loops.
The consequences of this can be dire
making it harder to identify and support the best projects.
making it harder to identify and stop harmful projects
making it harder to identify when a decent idea isn’t panning out, leading to people and money getting stuck in the mediocre project instead of moving on.
One of the general concerns about MATS is it spins up potential capabilities researchers. If the market can’t absorb the talent, that suggests maybe MATS should shrink.
OTOH if you told me that for every 10 entrants MATS spins up 1 amazing safety researcher and 9 people who need makework to prevent going into capabilities, I’d be open to arguments that that was a good trade.
I don’t believe that, although I see how my summary could be interpreted that way. I agree with basically all the reasons in your recent comment and most in the original comment. I could add a few reasons of my own doing independent grant-funded work sucks. But I think it’s really important to track how founding projects tracks to increased potential safety instead of intermediates, and push hard against potential tail wagging the dog scenarios.
I was trying to figure out why this was important to me, given how many of your points I agree with. I think it’s a few things:
Alignment work seems to be prone to wagging the dog, and is harder to correct, due to poor feedback loops.
The consequences of this can be dire
making it harder to identify and support the best projects.
making it harder to identify and stop harmful projects
making it harder to identify when a decent idea isn’t panning out, leading to people and money getting stuck in the mediocre project instead of moving on.
One of the general concerns about MATS is it spins up potential capabilities researchers. If the market can’t absorb the talent, that suggests maybe MATS should shrink.
OTOH if you told me that for every 10 entrants MATS spins up 1 amazing safety researcher and 9 people who need makework to prevent going into capabilities, I’d be open to arguments that that was a good trade.