thanks for this post! I think it is always great when people share their opinions about the timelines and more people(even the ones not directly involved in ML) should be encouraged to freely express their view without the fear that they will be held accountable in the case they are wrong. In my opinion, even the people directly involved in ML research seem to be too reluctant to share their timelines and how they impact their work which might be useful for others. Essentially, I think that people should share their view when it is something that is going to somehow influence their decision making, rather than when they feel it crosses some level of rigour/certainty, therefore posts like this one should receive a bit more praise (and LW should have two types of voting also for posts not just comments).
While I disagree with the overall point of the post, I agree that there is probably a lot of wishful thinking/curiosity driving this forum and impacting some predictions. However, even despite this, I still think that AGI is very close. My prediction is that TAI will happen in the next 2-5 years(70%) and AGI in the next 8 (75%). I guess it will be based on something like scaled-up GATO pre-trained on youtube videos with RL and some memory. The main reason for this is that deep-learning was operating on a very small scale just two years ago(less than a billion parameters) which made it very difficult to test some ideas. The algorithmic improvements to me seem just too easy to come up with. For example, almost all important problems e.g. language,vision, audio, RL were solved/almost solved in a very short time and the the ideas there didn’t require much ingenuity.
Just a slight exaggeration—if you take a five year old and ask him to draw a random diagram, chances are quite high that if scaled up, this is a SOTA architecture for something. It is just hard to test the ideas, because of the engineering difficulty and the lack of compute. However, this is likely to be overcomed soon with either more money being thrown on the problem or architecture improvements—e.g. Cerebras and Graphcore seem to be doing some promising work here.
thanks for this post! I think it is always great when people share their opinions about the timelines and more people(even the ones not directly involved in ML) should be encouraged to freely express their view without the fear that they will be held accountable in the case they are wrong. In my opinion, even the people directly involved in ML research seem to be too reluctant to share their timelines and how they impact their work which might be useful for others. Essentially, I think that people should share their view when it is something that is going to somehow influence their decision making, rather than when they feel it crosses some level of rigour/certainty, therefore posts like this one should receive a bit more praise (and LW should have two types of voting also for posts not just comments).
While I disagree with the overall point of the post, I agree that there is probably a lot of wishful thinking/curiosity driving this forum and impacting some predictions. However, even despite this, I still think that AGI is very close. My prediction is that TAI will happen in the next 2-5 years(70%) and AGI in the next 8 (75%). I guess it will be based on something like scaled-up GATO pre-trained on youtube videos with RL and some memory. The main reason for this is that deep-learning was operating on a very small scale just two years ago(less than a billion parameters) which made it very difficult to test some ideas. The algorithmic improvements to me seem just too easy to come up with. For example, almost all important problems e.g. language,vision, audio, RL were solved/almost solved in a very short time and the the ideas there didn’t require much ingenuity.
Just a slight exaggeration—if you take a five year old and ask him to draw a random diagram, chances are quite high that if scaled up, this is a SOTA architecture for something. It is just hard to test the ideas, because of the engineering difficulty and the lack of compute. However, this is likely to be overcomed soon with either more money being thrown on the problem or architecture improvements—e.g. Cerebras and Graphcore seem to be doing some promising work here.