As someone who is just starting on machine learning as an autodidact, here are my thoughts on what’s happening (probably an incoherent ramble).
I wouldn’t have expected gradient descent to find particularly great solutions for Dota. When OpenAI released the 1v1 bot with all the restrictions of the case, I figured that was about the maximum achievable through gradient descent and that if they managed to do more in the following years (like they just did), and if someone managed to make a similar architecture work for even a slightly broader range of games, e.g. first person shooters (similarly to what happened from AlphaGo to AlphaZero), let’s call this architecture GamesZero, then basically GamesZero had to be equivalent to a full human level AGI. (but even in that case, I expected the 5v5 bot to come out after 3-5 more years if at all possible, not one year)
There was also some confusion that I had at the time, which I guess I still have but it’s gradually dissolving, about the fact that if GamesZero could come out of gradient descent then gradient descent also had to be an AGI, in the sense that e.g. our brains do something very similar to gradient descent, or that any AGI must implement some sort of gradient descent. It just wasn’t behaving like one because we needed faster algorithms or something.
My current opinion is a bit different. Gradient descent is an optimization process more similar to natural selection than to what our brains do. GamesZero can come out of gradient descent in the same way that human brains came out of natural selection, and while it can be said that all four of them are optimization processes, there is a sense in which I would put human brains and GamesZero in the “AGI category” (whatever that means), while I would put natural selection and gradient descent in the “blind idiot god category”. Which is to say, I don’t expect neither human brains or GamesZero to make use of gradient descent, anymore than I expect human brains to make use of simulated natural selections. But of course I could be horribly wrong.
Now that gradient descent accomplished 30% of what I thought was sufficient to make an AGI and that I thought it could never do, I wouldn’t be surprised if GamesZero comes out in two years, although I would be panicked (I will be surprised in addition to being panicked if it comes out next year).
The question then is: is GamesZero an AGI?
What makes me suspicious that the answer is yes is the categorization that I mentioned earlier. If GamesZero can work in a variety of games ranging from MOBAs to FPSs then I also expect it to work in the outside world, similarly to how human brains could only work in certain environments and only behave in certain ways before discovering fire (or whatever was the critical historical point) and then natural selection made something happen that made them able in principle to go to the moon.
If GamesZero can work in a variety of games ranging from MOBAs to FPSs then I also expect it to work in the outside world
The thing you are describing as GamesZero would almost certainly require quite a lot of game-specific training data (OpenAI Five required hundreds of years of training data). Getting this much data is completely impractical in the real world; this is the main reason why deep learning is not very successful in robotics.
As someone who is just starting on machine learning as an autodidact, here are my thoughts on what’s happening (probably an incoherent ramble).
I wouldn’t have expected gradient descent to find particularly great solutions for Dota. When OpenAI released the 1v1 bot with all the restrictions of the case, I figured that was about the maximum achievable through gradient descent and that if they managed to do more in the following years (like they just did), and if someone managed to make a similar architecture work for even a slightly broader range of games, e.g. first person shooters (similarly to what happened from AlphaGo to AlphaZero), let’s call this architecture GamesZero, then basically GamesZero had to be equivalent to a full human level AGI. (but even in that case, I expected the 5v5 bot to come out after 3-5 more years if at all possible, not one year)
There was also some confusion that I had at the time, which I guess I still have but it’s gradually dissolving, about the fact that if GamesZero could come out of gradient descent then gradient descent also had to be an AGI, in the sense that e.g. our brains do something very similar to gradient descent, or that any AGI must implement some sort of gradient descent. It just wasn’t behaving like one because we needed faster algorithms or something.
My current opinion is a bit different. Gradient descent is an optimization process more similar to natural selection than to what our brains do. GamesZero can come out of gradient descent in the same way that human brains came out of natural selection, and while it can be said that all four of them are optimization processes, there is a sense in which I would put human brains and GamesZero in the “AGI category” (whatever that means), while I would put natural selection and gradient descent in the “blind idiot god category”. Which is to say, I don’t expect neither human brains or GamesZero to make use of gradient descent, anymore than I expect human brains to make use of simulated natural selections. But of course I could be horribly wrong.
Now that gradient descent accomplished 30% of what I thought was sufficient to make an AGI and that I thought it could never do, I wouldn’t be surprised if GamesZero comes out in two years, although I would be panicked (I will be surprised in addition to being panicked if it comes out next year).
The question then is: is GamesZero an AGI?
What makes me suspicious that the answer is yes is the categorization that I mentioned earlier. If GamesZero can work in a variety of games ranging from MOBAs to FPSs then I also expect it to work in the outside world, similarly to how human brains could only work in certain environments and only behave in certain ways before discovering fire (or whatever was the critical historical point) and then natural selection made something happen that made them able in principle to go to the moon.
The thing you are describing as GamesZero would almost certainly require quite a lot of game-specific training data (OpenAI Five required hundreds of years of training data). Getting this much data is completely impractical in the real world; this is the main reason why deep learning is not very successful in robotics.