@gwern and @lc are right. Stockfish is terrible at odds and this post could really use some follow-up.
As @simplegeometry points out in the comments, we now have much stronger odds-playing engines that regularly win against much stronger players than OP.
As the name suggests, Leela Queen Odds is trained specifically to play without a queen, which is of course an absolutely bonkers disadvantage against 2k+ elo players. One interesting wrinkle is the time constraint. AIs are better at fast chess (obviously), and apparently no one who’s tried is yet able to beat it consistently at 3+0 (3 minutes with no timing increment)
At rapid time controls, it seems like we could maybe go even against Magnus with knight odds? If not Magnus, perhaps other high-rated GMs.
There was a match with the most recently updated LeelaKnightOdds and GM Alex Lenderman but I don’t recall the score exactly. EDIT: which was 19-3-2 win draw loss.
I am very skeptical of this on priors, for the record. I think this statement could be true for superblitz time controls and whatnot, but I would be shocked if knight odds would be enough to beat Magnus in a 10+0 or 15+0 game. That being said, I have no inside knowledge, and I would update a lot of my beliefs significantly if your statement as currently written actually ends up being true.
LeelaKnightOdds has convincing beaten both Awonder Liang and Anish Giri at 3+2 by large margins, and has an extremely strong record at 5+3 against people who have challenged it.
I think 15+0 and probably also 10+0 would be a relatively easy win for Magnus based on Awonder, a ~150 elo weaker player, taking two draws at 8+3 and a win and a draw at 10+5. At 5+3 I’m not sure because we have so little data at winnable time controls, but wouldn’t expect an easy win for either player.
It’s also certainly not the case that these few-months-old networks running a somewhat improper algorithm are the best we could build—it’s known at minimum that this Leela is tactically weaker than normal and can drop endgame wins, even if humans rarely capitalize on that.
Hissha from the Lc0 server reports 19 wins, 3 draws, and 2 losses against Lenderman (currently ~2500 FIDE) at 15+10 from a knight odds match 2 months ago—with the caveat that Lenderman started playing too fast after 10 games. I haven’t run the numbers but suspect this would be enough to go even against a 2750, if not Magnus?
I was surprised too. I think it’s an exciting development :)
Hmm, that sounds about right based on the usual human-vs-human transfer from Elo difference to performance… but I am still not sure if that holds up when you have odds games, which feel qualitatively different to me than regular games. Based on my current chess intuition, I would expect the ability to win odds games to scale better than ELO near the top level, but I could be wrong about this.
@gwern and @lc are right. Stockfish is terrible at odds and this post could really use some follow-up.
As @simplegeometry points out in the comments, we now have much stronger odds-playing engines that regularly win against much stronger players than OP.
https://lichess.org/@/LeelaQueenOdds
https://marcogio9.github.io/LeelaQueenOdds-Leaderboard/
That’s really cool! Do you have any sense of what kind of material advantage these odd-playing engines could use against the best humans?
As the name suggests, Leela Queen Odds is trained specifically to play without a queen, which is of course an absolutely bonkers disadvantage against 2k+ elo players. One interesting wrinkle is the time constraint. AIs are better at fast chess (obviously), and apparently no one who’s tried is yet able to beat it consistently at 3+0 (3 minutes with no timing increment)
At rapid time controls, it seems like we could maybe go even against Magnus with knight odds? If not Magnus, perhaps other high-rated GMs.
There was a match with the most recently updated LeelaKnightOdds and GM Alex Lenderman
but I don’t recall the score exactly.EDIT: which was 19-3-2 win draw loss.I am very skeptical of this on priors, for the record. I think this statement could be true for superblitz time controls and whatnot, but I would be shocked if knight odds would be enough to beat Magnus in a 10+0 or 15+0 game. That being said, I have no inside knowledge, and I would update a lot of my beliefs significantly if your statement as currently written actually ends up being true.
LeelaKnightOdds has convincing beaten both Awonder Liang and Anish Giri at 3+2 by large margins, and has an extremely strong record at 5+3 against people who have challenged it.
I think 15+0 and probably also 10+0 would be a relatively easy win for Magnus based on Awonder, a ~150 elo weaker player, taking two draws at 8+3 and a win and a draw at 10+5. At 5+3 I’m not sure because we have so little data at winnable time controls, but wouldn’t expect an easy win for either player.
It’s also certainly not the case that these few-months-old networks running a somewhat improper algorithm are the best we could build—it’s known at minimum that this Leela is tactically weaker than normal and can drop endgame wins, even if humans rarely capitalize on that.
Hissha from the Lc0 server reports 19 wins, 3 draws, and 2 losses against Lenderman (currently ~2500 FIDE) at 15+10 from a knight odds match 2 months ago—with the caveat that Lenderman started playing too fast after 10 games. I haven’t run the numbers but suspect this would be enough to go even against a 2750, if not Magnus?
I was surprised too. I think it’s an exciting development :)
Hmm, that sounds about right based on the usual human-vs-human transfer from Elo difference to performance… but I am still not sure if that holds up when you have odds games, which feel qualitatively different to me than regular games. Based on my current chess intuition, I would expect the ability to win odds games to scale better than ELO near the top level, but I could be wrong about this.
Knight odds is pretty challenging even for grandmasters.