AlphaZero did some (pruned) game tree exploration in a hardcoded way that allowed the NN to focus on the evaluation of how good a given position was; this allowed it to kind of be a “best of both worlds” between previous algorithms like Stockfish and a pure deep reinforcement learner.
This is impossible for a game with an action space as large as StarCraft II, though; but in order to modify a game like Go, it would have to become completely different.
I’m not 100% sure about the example you raise, but it seems to me it’s either going to have a decently prune-able game tree, or that humans won’t be capable of playing the game at a very sophisticated level, so I’d expect AlphaZero-esque things to get superhuman at it. StarCraft is easier for humans relative to AIs because we naturally chunk concepts together (visually and strategically) that are tricky for the AI to learn.
Pruning the game tree, or doing MC tree search, is impossible in StarCraft, not because of the size of the action space but because the game has incomplete information. At least in the standard form of those algorithms.
AlphaZero did some (pruned) game tree exploration in a hardcoded way that allowed the NN to focus on the evaluation of how good a given position was; this allowed it to kind of be a “best of both worlds” between previous algorithms like Stockfish and a pure deep reinforcement learner.
This is impossible for a game with an action space as large as StarCraft II, though; but in order to modify a game like Go, it would have to become completely different.
I’m not 100% sure about the example you raise, but it seems to me it’s either going to have a decently prune-able game tree, or that humans won’t be capable of playing the game at a very sophisticated level, so I’d expect AlphaZero-esque things to get superhuman at it. StarCraft is easier for humans relative to AIs because we naturally chunk concepts together (visually and strategically) that are tricky for the AI to learn.
So basically game tree search was the “reasoning” part of AlphaZero?
Pruning the game tree, or doing MC tree search, is impossible in StarCraft, not because of the size of the action space but because the game has incomplete information. At least in the standard form of those algorithms.
Well, it’s overdetermined. Action space, tree depth, incomplete information; any one of these is enough to make MC tree search impossible.