I think it’s a mistake to think of current chess or go engines as being at maximum capability. If we would throw a few billion dollar worth of compute at them they would likely get significantly better.
Narrow Optimisers Outperform General Optimisers on Narrow Domains
That’s true sometimes but not always. Notably, GATO is better at controlling a Sawyer arm than more specialized optimizers. Given that the company that sells the Sawyer arm spent a lot of time developing software to control it, that’s impressive.
These takes aren’t totally opposite. Elo is capped due to the way it treats draws, but there’s other metrics that can be devised, where “significantly better” is still viable. For example, how close to a perfect game (with no tied positions becoming game-theoretically lost, or winning positions becoming game-theoretically tied) does the AI play? And ignoring matches where there are ties, only paying attention to games where either player wins, you remove the ceiling.
I think it’s a mistake to think of current chess or go engines as being at maximum capability. If we would throw a few billion dollar worth of compute at them they would likely get significantly better.
That’s true sometimes but not always. Notably, GATO is better at controlling a Sawyer arm than more specialized optimizers. Given that the company that sells the Sawyer arm spent a lot of time developing software to control it, that’s impressive.
I have the totally opposite take on chess engines (see my comment).
These takes aren’t totally opposite. Elo is capped due to the way it treats draws, but there’s other metrics that can be devised, where “significantly better” is still viable. For example, how close to a perfect game (with no tied positions becoming game-theoretically lost, or winning positions becoming game-theoretically tied) does the AI play? And ignoring matches where there are ties, only paying attention to games where either player wins, you remove the ceiling.
I did say given similar levels of cognitive investment.
My guess is that the cognitive work put in GATO’s architectures/algorithms was much better than the specialised arms it dominates.
That or GATO was running on a much larger compute budget.