If the latter gets too large, then you start getting swarmed with people who want money and prestige but don’t necessarily understand how to contibute, who are incentivized to degrade the signal of what’s actually important.
During this decade the field of AI in general became one of the most prestigious and high-status academic fields to work in. But as far as I can tell, it hasn’t slowed down the rate of progress in advancing AI capability. If anything, it has sped it up—by quite a bit. It’s possible that a lot of newcomers to the field are largely driven by the prospect of status gain and money. And there are quite a few “AI” hype-driven startups that have popped up and seem doomed to fail, but despite this, it doesn’t seem to be slowing the pace of the most productive research groups. Maybe the key here is that if you suddenly increase the prestige of a scientific field by a dramatic amount, you are bound to get a lot of nonsense or fraudulent activity, but this might be constrained to being outside of serious research circles. And the most serious people working in the field are likely to be helped by the rising tide as well, due to increased visibility and funding to their labs and so on.
It’s also my understanding that the last few years (during the current AI boom) have been some of the most successful (financially and productively) for MIRI in their entire history.
During this decade the field of AI in general became one of the most prestigious and high-status academic fields to work in. But as far as I can tell, it hasn’t slowed down the rate of progress in advancing AI capability. If anything, it has sped it up—by quite a bit. It’s possible that a lot of newcomers to the field are largely driven by the prospect of status gain and money. And there are quite a few “AI” hype-driven startups that have popped up and seem doomed to fail, but despite this, it doesn’t seem to be slowing the pace of the most productive research groups. Maybe the key here is that if you suddenly increase the prestige of a scientific field by a dramatic amount, you are bound to get a lot of nonsense or fraudulent activity, but this might be constrained to being outside of serious research circles. And the most serious people working in the field are likely to be helped by the rising tide as well, due to increased visibility and funding to their labs and so on.
It’s also my understanding that the last few years (during the current AI boom) have been some of the most successful (financially and productively) for MIRI in their entire history.
This is an interesting point I hadn’t considered. Still mulling it over a bit.