I think the algorithm progress is doing some heavy lifting in this model. I think if we had a future textbook on agi we could probably build one but AI is kinda famous for minor and simple things just not being implemented despite all the parts being there
See ReLU activations and sigmoid activations.
If we’re bottlenecking at algorithms alone is there a reason that isn’t a really bad bottleneck?
See my other response to Raphael elsewhere in this comment thread.
My model is that the big AI labs are currently throttling algorithmic progress by choosing to devote their resources to scaling.
If scaling leads to AGI, we get AGI soon that way. (I give this about 20% chance.)
If scaling doesn’t lead to AGI, then refocusing resources on experimentation seems like a natural next move. (I think this is about 80% likely to work in under two years if made a major focus of resources, including both giving human researchers the time, encouragement and compute resources they need, plus developing increasingly helpful AI reseachers.)
Hmm, mixed agree/disagree. Scale probably won’t work, algorithms probably would, but I don’t think it’s going to be that quick.
Namely, I think that the company struggling with fixed capital costs could accomplish much more, much quicker using the salary expenses of the top researchers they already have they’d have done it or gave it a good try at least
I’m 5 percent that a serious switch to algorithms would result in AGI in 2 years. You might be more well read than me on this so I’m not quite taking side bets right now!
I think the algorithm progress is doing some heavy lifting in this model. I think if we had a future textbook on agi we could probably build one but AI is kinda famous for minor and simple things just not being implemented despite all the parts being there
See ReLU activations and sigmoid activations.
If we’re bottlenecking at algorithms alone is there a reason that isn’t a really bad bottleneck?
See my other response to Raphael elsewhere in this comment thread.
My model is that the big AI labs are currently throttling algorithmic progress by choosing to devote their resources to scaling.
If scaling leads to AGI, we get AGI soon that way. (I give this about 20% chance.)
If scaling doesn’t lead to AGI, then refocusing resources on experimentation seems like a natural next move. (I think this is about 80% likely to work in under two years if made a major focus of resources, including both giving human researchers the time, encouragement and compute resources they need, plus developing increasingly helpful AI reseachers.)
Hmm, mixed agree/disagree. Scale probably won’t work, algorithms probably would, but I don’t think it’s going to be that quick.
Namely, I think that the company struggling with fixed capital costs could accomplish much more, much quicker using the salary expenses of the top researchers they already have they’d have done it or gave it a good try at least
I’m 5 percent that a serious switch to algorithms would result in AGI in 2 years. You might be more well read than me on this so I’m not quite taking side bets right now!