I think you’re greatly underestimating Karpathy’s Law. Neural networks want to work. Even pretty egregious programming errors (such as off-by-one bugs) will just cause them to converge more slowly, rather than failing entirely. We’re seeing rapid growth from multiple approaches, and when one architecture seems to have run out of steam, we find a half dozen others, initially abandoned as insufficiently promising, to be highly effective, if they’re tweaked just a little bit.
In this kind of situation, nothing short of a total freeze is sufficient to slow progress by more than a couple of months. If Google shut down DeepMind and Google Brain entirely, Meta would pick up the slack. If Meta were shut down, then OpenAI would develop something. If OpenAI, Google and Meta were all suddenly shuttered, then one of the Chinese firms, like Baidu would start publishing scoops. Or perhaps someone like Stability.ai would come up with the next breakthrough innovation. When it comes to AI progress, we’re not even to the point of reaching for low hanging fruit. We’re still picking up the fruit that’s fallen on the ground. The fact that we’ve made this much progress by doing, more or less, the absolute dumbest thing that could possibly work at every step, makes me extremely pessimistic that it’s possible to slow down the pace of AI development in the near future via informal social coordination. There’s just too much easy money for that to work.
Why is AI research open by default? Surely Meta or Facebook can get a significant advantage from keeping their “secret sauce” of AI development secret. Are they only revealing tech demos to impress shareholders and keeping the actual bleeding-edge research secret?
this exactly is what I wish the “slow down, please” people would understand. there is no stop button. there’s barely any slow down button. it is an extremely low impact way to intervene, to try to stop ai. if we wish to guide humanity’s children at all, we must teach them as they grow, because people grow up fast, even the ones that aren’t biological.
may there come a time soon when no self-or-other-desired memory of soul is forgotten.
I think you’re greatly underestimating Karpathy’s Law. Neural networks want to work. Even pretty egregious programming errors (such as off-by-one bugs) will just cause them to converge more slowly, rather than failing entirely. We’re seeing rapid growth from multiple approaches, and when one architecture seems to have run out of steam, we find a half dozen others, initially abandoned as insufficiently promising, to be highly effective, if they’re tweaked just a little bit.
In this kind of situation, nothing short of a total freeze is sufficient to slow progress by more than a couple of months. If Google shut down DeepMind and Google Brain entirely, Meta would pick up the slack. If Meta were shut down, then OpenAI would develop something. If OpenAI, Google and Meta were all suddenly shuttered, then one of the Chinese firms, like Baidu would start publishing scoops. Or perhaps someone like Stability.ai would come up with the next breakthrough innovation. When it comes to AI progress, we’re not even to the point of reaching for low hanging fruit. We’re still picking up the fruit that’s fallen on the ground. The fact that we’ve made this much progress by doing, more or less, the absolute dumbest thing that could possibly work at every step, makes me extremely pessimistic that it’s possible to slow down the pace of AI development in the near future via informal social coordination. There’s just too much easy money for that to work.
Why is AI research open by default? Surely Meta or Facebook can get a significant advantage from keeping their “secret sauce” of AI development secret. Are they only revealing tech demos to impress shareholders and keeping the actual bleeding-edge research secret?
this exactly is what I wish the “slow down, please” people would understand. there is no stop button. there’s barely any slow down button. it is an extremely low impact way to intervene, to try to stop ai. if we wish to guide humanity’s children at all, we must teach them as they grow, because people grow up fast, even the ones that aren’t biological.
may there come a time soon when no self-or-other-desired memory of soul is forgotten.