Karpathy mentions offhand in this video that he thinks he has the correct approach to AGI but doesnt say what it is. Before that he lists a few common approaches, so I assume it’s not one of those. What do you think he suggests?
P.S. If this worries you that AGI is closer than you expected do not watch Jeff dean’s overview lecture of DL research at Google.
The overview lecture doesn’t really get me worried. It basically means that we are at the point where we can use machine learning to solve well-defined problems with plenty of training data. At the moment that seems to require a human machine learning expert and recent Google experiments suggest that they are confident to develop an API that can do this without machine learning experts being involved.
At a recent LW discussion someone told me that this kind of research doesn’t even count as an attempt to develop AGI.
At the moment that seems to require a human machine learning expert and recent Google experiments suggest that they are confident to develop an API that can do this without machine learning experts being involved.
At a recent LW discussion someone told me that this kind of research doesn’t even count as an attempt to develop AGI.
Not in itself, sure, but yeah there was the bit about the progress made so you wont need a ML engineer for developing the right net to solve a problem. However, there was also the bit whee they have nets doing novel research (e.g. new activation functions with better performance than sota, novel architectures etc.). And for going further in that direction, they just want more compute which they’re going to be getting more and more of.
I mean, if we’ve entered the point where we AI research is a problem tackalable by (narrow) AI, which can further benefit from that research and apply it to make further improvements faster/wtih more accuracy.. then maybe there is something to potentially worry about .
Unless of course you think that AGI will be built in such a different way that no/very few DL findings are likely to be applicable. But even then I wouldn’t be convinced that doing this completely separate AGI research wont also be the kind of problem that DL wont be able to handle—as AGI research is in the end a “narrow” problem.
To me the question isn’t whether new DL findings are applicable but whether they are sufficient. I don’t think they are sufficient to be able to solve problems where there isn’t a big dataset available.
I think I don’t know the solution, and if so it’s impossible for me to guess what he thinks if he’s right :)
But maybe he’s thinking of something vague like CIRL, or hierarchical self-supervised learning with generation, etc. But I think he’s thinking of some kind of recurrent network. So maybe he has some clever idea for unsupervised credit assignment?
Karpathy mentions offhand in this video that he thinks he has the correct approach to AGI but doesnt say what it is. Before that he lists a few common approaches, so I assume it’s not one of those. What do you think he suggests?
P.S. If this worries you that AGI is closer than you expected do not watch Jeff dean’s overview lecture of DL research at Google.
The overview lecture doesn’t really get me worried. It basically means that we are at the point where we can use machine learning to solve well-defined problems with plenty of training data. At the moment that seems to require a human machine learning expert and recent Google experiments suggest that they are confident to develop an API that can do this without machine learning experts being involved.
At a recent LW discussion someone told me that this kind of research doesn’t even count as an attempt to develop AGI.
Not in itself, sure, but yeah there was the bit about the progress made so you wont need a ML engineer for developing the right net to solve a problem. However, there was also the bit whee they have nets doing novel research (e.g. new activation functions with better performance than sota, novel architectures etc.). And for going further in that direction, they just want more compute which they’re going to be getting more and more of.
I mean, if we’ve entered the point where we AI research is a problem tackalable by (narrow) AI, which can further benefit from that research and apply it to make further improvements faster/wtih more accuracy.. then maybe there is something to potentially worry about .
Unless of course you think that AGI will be built in such a different way that no/very few DL findings are likely to be applicable. But even then I wouldn’t be convinced that doing this completely separate AGI research wont also be the kind of problem that DL wont be able to handle—as AGI research is in the end a “narrow” problem.
To me the question isn’t whether new DL findings are applicable but whether they are sufficient. I don’t think they are sufficient to be able to solve problems where there isn’t a big dataset available.
I think I don’t know the solution, and if so it’s impossible for me to guess what he thinks if he’s right :)
But maybe he’s thinking of something vague like CIRL, or hierarchical self-supervised learning with generation, etc. But I think he’s thinking of some kind of recurrent network. So maybe he has some clever idea for unsupervised credit assignment?