I’m very confident that Magnus absolutely crushes a beginner who has been given a personal chess book, of normal book length, written by God. Magnus still has all the advantages.
Magnus can evaluate moves faster and has a deeper search tree.
The book of chess can provide optimal opening lines, but the beginner needs to memorize them, and Magnus has a greater capacity for memorizing openings.
The book of chess can provide optimal principles for evaluating moves, but the beginner has to apply them, and decide what to do when they point in different directions. This comes from practice. A book of normal size can only provide limited practice examples.
The beginner will have a higher rate of blunders. It is hard to “capture toward the center” when you don’t even see the capture.
Some intuitions from chess books: the book God would give to a beginner is different to the book God would give a 1200 player. After reading a chess book, it is normal for ability to initially go down, until the advice has been distilled and integrated with practice. Reading a chess book helps
improve faster, not to be instantly better.
Some intuitions from chess programs: they lose a lot of power if you cut down their search time to simulate the ability of a beginner to calculate variations, and also cut down their opening database to simulate the ability of a beginner to memorize openings, and also give them a random error chance to simulate a beginner’s rate of blunders.
Sorry for the double response, but a separate point here is that your method of estimating the effectiveness of the best possible book seems dubious to me. It seems to be “let’s take the best book we have; the perfect book won’t be that much better”. But why would this be true, at all? We have applied tons of optimization pressure to chess and probably know that the ceiling isn’t that far above Stockfish, but we haven’t applied tons of optimization pressure to distilling chess. How do you know that the best possible book won’t be superior by some large factor? Why can’t the principles be so simple that applying them is easy? (This is a more general question; how can you e.g. estimate the effectiveness of the best possible text book for some subfield of math?)
I’m a bit more sympathetic to this if we play Blitz, but for the most interesting argument, I think we should assume classical time format, where any beginner can see all possible captures.
Thanks for the double response. This line seems potentially important. If we could safely create an Oracle that can create a book of chess that massively boosts chess ability, then we could maybe possibly miraculously do the same thing to create a book that massively boosts AI safety research ability.
I agree that my argument above was pretty sketchy, just “intuitions” really. Here’s something a bit more solid, after further reflection.
I’m aware of adversarial examples and security vulnerabilities, so I’m not surprised if a superintelligence is able to severely degrade human performance via carefully selected input. A chess book that can make Magnus lose to a beginner wouldn’t surprise me. Neither would a chess book that degraded a beginner’s priorities such that they obsessed about chess, for however many Elo points that would be worth.
But mostly this problem is in the opposite direction: can we provide carefully curated input that allows an intelligence to learn much faster? In this direction the results seem much less dramatic. My impression is that the speed of learning is limited by both the inputs and the learner. If the book of chess is a perfect input, then the limiting factor is the reader, and an average reader won’t get outsized benefits from perfect inputs.
Possible counter-argument: supervised learning can outperform unsupervised learning by some large factor, data quality can likewise have a big impact. That’s fine, but every chess book I’ve read has been supervised learning, and chess books are already higher data quality than scraping r/chess. So those optimizations have already been made.
Possible counter-argument: few-shot learning in GPT-3? This seems more like surface knowledge that is already in the language model. So maybe a chess beginner already has the perfect chess algorithm somewhere in their brain, and the chess book just needs to surface that model and suppress all the flawed models that are competing with it? I don’t buy it, that’s not what it feels like learning chess from the inside, but maybe I need to give the idea some weight.
Possible counter-argument: maybe humans are actually really intelligent and really good learners and the reason we’re so flawed is that we have bad inputs? Eg from other flawed humans, random chance hiding things, biases in what we pay attention to, etc. I don’t buy this, but I don’t actually have a clear reason why.
But mostly this problem is in the opposite direction: can we provide carefully curated input that allows an intelligence to learn much faster? In this direction the results seem much less dramatic. My impression is that the speed of learning is limited by both the inputs and the learner. If the book of chess is a perfect input, then the limiting factor is the reader, and an average reader won’t get outsized benefits from perfect inputs.
Which results did you have in mind? The ‘machine teaching’ results are pretty dramatic and surprising, although one could question whether they have any practical implications.
I wasn’t aware of them. Thanks. Yes, that’s exactly the sort of thing I’d expect to see if there was a large possible upside in better teaching materials that an Oracle could produce. So I no longer disagree with Rafael & Richard on this.
But mostly this problem is in the opposite direction: can we provide carefully curated input that allows an intelligence to learn much faster? In this direction the results seem much less dramatic. My impression is that the speed of learning is limited by both the inputs and the learner. If the book of chess is a perfect input, then the limiting factor is the reader, and an average reader won’t get outsized benefits from perfect inputs.
My problem with this is that you’re treating the amount of material as fixed and abstracting it as “speed”; however, what makes me unsure about the power of the best possible book is that it may choose a completely different approach.
E.g., consider the “ontology” of high-level chess principles. We think in terms of “development” and “centralization [of pieces]” and “activity” and “pressure” and “attacking” and “discoveries” and so forth. Presumably, most of these are quite helpful; if you have no concept of discoveries, you will routinely place your queen or king on inconvenient squares and get punished. If you have no concept of pressure, you have no elegant way of pre-emptive reaction if your opponent starts aligning a lot of pieces toward your king, et cetera.
So, at the upper end of my probability distribution for how good a book would be, it may introduce a hundred more such concepts, each one highly useful to elegantly compress various states. It will explain them all in the maximally intuitive and illustrative way, such that they all effortlessly stick, in the same way that sometimes things you hear just make sense and fit your aesthetic, and you recall them effortlessly. After reading this book, a beginner will look at a bunch of moves of a 2000 elo player, and go “ah, these two moves clearly violate principle Y”. Even though this player has far less ability to calculate lines, they know so many elegant compressions that they may compensate in a direct match. Much in the same way that you may beat someone who has practiced twice as long as you but has no concept of pressure; they just can’t figure out how to spot situations from afar where their king is suddenly in trouble.
Isn’t it trivial for the beginner to beat Magnus using this book? God just needs to predict Magnus perfectly, and write down a single list of moves that the beginner needs to follow to beat him. Half a page is enough.
In general, you ignored this approach, which is the main reason why I’m unsure whether a book from a superintelligence could beat Magnus.
I read your idea of “a line that seems forcing for black”, and I interpreted it as being forcing for black in general, and responded in terms of memorizing optimal opening lines. It sounds like you meant a line that would cause Magnus in particular to respond in predictable ways? Sorry for missing that.
I can imagine a scenario with an uploaded beginner and an uploaded Magnus in a sealed virtual environment running on error-correcting hardware with a known initial state and a deterministic algorithm, and your argument goes through there, and in sufficiently similar scenarios.
Whereas I had in mind a much more chaotic scenario. For example, I expect Magnus’s moves to depend in part on the previous games he played, so predicting Magnus requires predicting all of those games, and thus the exponential tree of previous games. And I expect his moves to depend in part on his mood, eg how happy he’d be with a draw. So our disagreement could be mostly about the details of the hypothetical, such as how much time passes between creating the book and playing the game?
I read your idea of “a line that seems forcing for black”, and I interpreted it as being forcing for black in general
So to clarify: this interpretation was correct. I was assuming that a superintelligence cannot perfectly predict Magnus, pretty much for the reasons you mention (dependency on previous games, mood, etc.) But I then changed that standard when you said
I’m very confident that Magnus absolutely crushes a beginner who has been given a personal chess book, of normal book length, written by God.
Unlike a superintelligence, surely god could simulate Magnus perfectly no matter what; this is why I called the problem trivial—if you invoke god.
If you don’t invoke god (and thus can’t simulate magnus), I remain unsure. There are already games where world champions play the top move recommended by the engine 10 times in a row, and those have not been optimized for forcing lines. You may overestimate how much uncertainty or variance there really is. (Though again, if Magnus knows what you’re doing, it gets much harder since then he could just play a few deliberately bad moves and get you out of preparation.)
Yes, I used “God” to try to avoid ambiguity about (eg) how smart the superintelligence is, and ended up just introducing ambiguity about (eg) whether God plays dice. Oops. I think the God hypothetical ends up showing the usual thing: Oracles fail[1] at large/chaotic tasks, and succeed at small/narrow tasks. Sure, more things are small and narrow if you are God, but that’s not very illuminating.
So, back to an Oracle, not invoking God, writing a book of chess for a beginner, filling it with lines that are forcing for black, trying to get >50% of the tree. Why do we care, why are we discussing this? I think because chess is so much smaller and less chaotic than most domains we care about, so if an Oracle fails at chess, it’s probably going to also fail at AI alignment, theorem proving, pivotal acts, etc.
There’s some simple failure cases we should get out of the way:
As you said, if Magnus knows or suspects what he’s playing against, he plays a few lower probability moves and gets out of the predicted tree. Eg, 1. e4 d6 is a 1% response from Magnus. Or, if Magnus thinks he’s playing a beginner, then he uses the opportunity to experiment, and becomes less predictable. So assume that he plays normally, predictably.
If Magnus keeps playing when he’s in a lost position, it’s really hard for a move to be “forced” if all moves lead to a loss with correct play. One chess principle I got from a book: don’t resign before the end game if you don’t know that your opponent can play the end game well. Well, assume that Magnus resigns a lost position.
What if the beginner misremembers something, and plays the wrong move? How many moves can a beginner remember, working from an Oracle-created book that has everything pre-staged with optimized mnemonics? I assume 1,000 moves, perfect recall. 10 moves per page for a 100 page book.
So we need to optimize for lines that are forcing, short, and winning[2]. Shortness is important because a 40 move line where each move is 98% forced is overall ~45% forcing, and because we can fit more short lines into our beginner’s memory. If you search through all top-level chess games and find ones where the players play the engine-recommended move ten times in a row, that is optimizing for winning (from the players) and forcing (from the search). Ten moves isn’t long enough, we need ~30 moves for a typical game.
Terrible estimate: with 500,000 games in chessgames.com, say there are 50 games with forcing lines of ten moves, a 10,000x reduction. An Oracle can search better, for games that haven’t been played yet. So maybe if Oracle searched through 5 trillion games it would find a game with a forcing line of 20 moves? At some point I question whether chess can be both low variance enough to have these long forcing lines, and also high variance enough to have so many potential games to search through. Of course chess has ample variance if you allow white to play bad moves, but then you’re not winning.
Another approach, trying to find a forcing opening, running through the stats on chessgames.com in a greedy way, I get this “Nimzo-Indian, Samisch” variation, which seems to be playable for both sides, but perhaps slightly favors black:
d4 Nf6 (73% forced—Magnus games)
c4 e6 (72% forced—Magnus games)
Nc3 Bb4 (83% forced—all games)
a3 Bxc3+ (100% forced—all games)
bxc3 c5 (55% forced—all games)
f3 d5 (85% forced—all games)
Multiplying that through gets 20% forcing over six moves. So maybe Oracle is amazingly lucky and there are hitherto undiscovered forcing lines directly from this well-known position to lost positions for black, missed by Stockfish, AlphaZero, and all humans. Well, then Oracle still needs to cover another 30% of the tree and get just as lucky a few more times. If that happens, I think I’m in crisis of faith mode where I have to reevaluate whether grandmaster chess was an elaborate hoax. So many positions we thought were even turn out to be winning for white, everyone missed it, what happened?
Where “fail” means “no plan found”, “memory and time exhausted”, “here’s a plan that involves running a reasoner in real-time” or “feed me observations in real-time and ask me only to generate a local and by-default-inscrutable action”, as listed by so8res above.
Yeah, this is part of what I was getting at. The narrowness of the task “write a set of instructions for a one-off victory against a particular player” is a crucial part of what makes it seem not-obviously-impossible to me. Fully simulating Magnus should be adequate, but then obviously you’re invoking a reasoner. What I’m uncertain about is if you can write such instructions without invoking a reasoner.
I’m very confident that Magnus absolutely crushes a beginner who has been given a personal chess book, of normal book length, written by God. Magnus still has all the advantages.
Magnus can evaluate moves faster and has a deeper search tree.
The book of chess can provide optimal opening lines, but the beginner needs to memorize them, and Magnus has a greater capacity for memorizing openings.
The book of chess can provide optimal principles for evaluating moves, but the beginner has to apply them, and decide what to do when they point in different directions. This comes from practice. A book of normal size can only provide limited practice examples.
The beginner will have a higher rate of blunders. It is hard to “capture toward the center” when you don’t even see the capture.
Some intuitions from chess books: the book God would give to a beginner is different to the book God would give a 1200 player. After reading a chess book, it is normal for ability to initially go down, until the advice has been distilled and integrated with practice. Reading a chess book helps improve faster, not to be instantly better.
Some intuitions from chess programs: they lose a lot of power if you cut down their search time to simulate the ability of a beginner to calculate variations, and also cut down their opening database to simulate the ability of a beginner to memorize openings, and also give them a random error chance to simulate a beginner’s rate of blunders.
Sorry for the double response, but a separate point here is that your method of estimating the effectiveness of the best possible book seems dubious to me. It seems to be “let’s take the best book we have; the perfect book won’t be that much better”. But why would this be true, at all? We have applied tons of optimization pressure to chess and probably know that the ceiling isn’t that far above Stockfish, but we haven’t applied tons of optimization pressure to distilling chess. How do you know that the best possible book won’t be superior by some large factor? Why can’t the principles be so simple that applying them is easy? (This is a more general question; how can you e.g. estimate the effectiveness of the best possible text book for some subfield of math?)
I’m a bit more sympathetic to this if we play Blitz, but for the most interesting argument, I think we should assume classical time format, where any beginner can see all possible captures.
Thanks for the double response. This line seems potentially important. If we could safely create an Oracle that can create a book of chess that massively boosts chess ability, then we could maybe possibly miraculously do the same thing to create a book that massively boosts AI safety research ability.
I agree that my argument above was pretty sketchy, just “intuitions” really. Here’s something a bit more solid, after further reflection.
I’m aware of adversarial examples and security vulnerabilities, so I’m not surprised if a superintelligence is able to severely degrade human performance via carefully selected input. A chess book that can make Magnus lose to a beginner wouldn’t surprise me. Neither would a chess book that degraded a beginner’s priorities such that they obsessed about chess, for however many Elo points that would be worth.
But mostly this problem is in the opposite direction: can we provide carefully curated input that allows an intelligence to learn much faster? In this direction the results seem much less dramatic. My impression is that the speed of learning is limited by both the inputs and the learner. If the book of chess is a perfect input, then the limiting factor is the reader, and an average reader won’t get outsized benefits from perfect inputs.
Possible counter-argument: supervised learning can outperform unsupervised learning by some large factor, data quality can likewise have a big impact. That’s fine, but every chess book I’ve read has been supervised learning, and chess books are already higher data quality than scraping r/chess. So those optimizations have already been made.
Possible counter-argument: few-shot learning in GPT-3? This seems more like surface knowledge that is already in the language model. So maybe a chess beginner already has the perfect chess algorithm somewhere in their brain, and the chess book just needs to surface that model and suppress all the flawed models that are competing with it? I don’t buy it, that’s not what it feels like learning chess from the inside, but maybe I need to give the idea some weight.
Possible counter-argument: maybe humans are actually really intelligent and really good learners and the reason we’re so flawed is that we have bad inputs? Eg from other flawed humans, random chance hiding things, biases in what we pay attention to, etc. I don’t buy this, but I don’t actually have a clear reason why.
Which results did you have in mind? The ‘machine teaching’ results are pretty dramatic and surprising, although one could question whether they have any practical implications.
I wasn’t aware of them. Thanks. Yes, that’s exactly the sort of thing I’d expect to see if there was a large possible upside in better teaching materials that an Oracle could produce. So I no longer disagree with Rafael & Richard on this.
My problem with this is that you’re treating the amount of material as fixed and abstracting it as “speed”; however, what makes me unsure about the power of the best possible book is that it may choose a completely different approach.
E.g., consider the “ontology” of high-level chess principles. We think in terms of “development” and “centralization [of pieces]” and “activity” and “pressure” and “attacking” and “discoveries” and so forth. Presumably, most of these are quite helpful; if you have no concept of discoveries, you will routinely place your queen or king on inconvenient squares and get punished. If you have no concept of pressure, you have no elegant way of pre-emptive reaction if your opponent starts aligning a lot of pieces toward your king, et cetera.
So, at the upper end of my probability distribution for how good a book would be, it may introduce a hundred more such concepts, each one highly useful to elegantly compress various states. It will explain them all in the maximally intuitive and illustrative way, such that they all effortlessly stick, in the same way that sometimes things you hear just make sense and fit your aesthetic, and you recall them effortlessly. After reading this book, a beginner will look at a bunch of moves of a 2000 elo player, and go “ah, these two moves clearly violate principle Y”. Even though this player has far less ability to calculate lines, they know so many elegant compressions that they may compensate in a direct match. Much in the same way that you may beat someone who has practiced twice as long as you but has no concept of pressure; they just can’t figure out how to spot situations from afar where their king is suddenly in trouble.
Isn’t it trivial for the beginner to beat Magnus using this book? God just needs to predict Magnus perfectly, and write down a single list of moves that the beginner needs to follow to beat him. Half a page is enough.
In general, you ignored this approach, which is the main reason why I’m unsure whether a book from a superintelligence could beat Magnus.
I read your idea of “a line that seems forcing for black”, and I interpreted it as being forcing for black in general, and responded in terms of memorizing optimal opening lines. It sounds like you meant a line that would cause Magnus in particular to respond in predictable ways? Sorry for missing that.
I can imagine a scenario with an uploaded beginner and an uploaded Magnus in a sealed virtual environment running on error-correcting hardware with a known initial state and a deterministic algorithm, and your argument goes through there, and in sufficiently similar scenarios.
Whereas I had in mind a much more chaotic scenario. For example, I expect Magnus’s moves to depend in part on the previous games he played, so predicting Magnus requires predicting all of those games, and thus the exponential tree of previous games. And I expect his moves to depend in part on his mood, eg how happy he’d be with a draw. So our disagreement could be mostly about the details of the hypothetical, such as how much time passes between creating the book and playing the game?
So to clarify: this interpretation was correct. I was assuming that a superintelligence cannot perfectly predict Magnus, pretty much for the reasons you mention (dependency on previous games, mood, etc.) But I then changed that standard when you said
Unlike a superintelligence, surely god could simulate Magnus perfectly no matter what; this is why I called the problem trivial—if you invoke god.
If you don’t invoke god (and thus can’t simulate magnus), I remain unsure. There are already games where world champions play the top move recommended by the engine 10 times in a row, and those have not been optimized for forcing lines. You may overestimate how much uncertainty or variance there really is. (Though again, if Magnus knows what you’re doing, it gets much harder since then he could just play a few deliberately bad moves and get you out of preparation.)
Yes, I used “God” to try to avoid ambiguity about (eg) how smart the superintelligence is, and ended up just introducing ambiguity about (eg) whether God plays dice. Oops. I think the God hypothetical ends up showing the usual thing: Oracles fail[1] at large/chaotic tasks, and succeed at small/narrow tasks. Sure, more things are small and narrow if you are God, but that’s not very illuminating.
So, back to an Oracle, not invoking God, writing a book of chess for a beginner, filling it with lines that are forcing for black, trying to get >50% of the tree. Why do we care, why are we discussing this? I think because chess is so much smaller and less chaotic than most domains we care about, so if an Oracle fails at chess, it’s probably going to also fail at AI alignment, theorem proving, pivotal acts, etc.
There’s some simple failure cases we should get out of the way:
As you said, if Magnus knows or suspects what he’s playing against, he plays a few lower probability moves and gets out of the predicted tree. Eg,
1. e4 d6is a 1% response from Magnus. Or, if Magnus thinks he’s playing a beginner, then he uses the opportunity to experiment, and becomes less predictable. So assume that he plays normally, predictably.If Magnus keeps playing when he’s in a lost position, it’s really hard for a move to be “forced” if all moves lead to a loss with correct play. One chess principle I got from a book: don’t resign before the end game if you don’t know that your opponent can play the end game well. Well, assume that Magnus resigns a lost position.
What if the beginner misremembers something, and plays the wrong move? How many moves can a beginner remember, working from an Oracle-created book that has everything pre-staged with optimized mnemonics? I assume 1,000 moves, perfect recall. 10 moves per page for a 100 page book.
So we need to optimize for lines that are forcing, short, and winning[2]. Shortness is important because a 40 move line where each move is 98% forced is overall ~45% forcing, and because we can fit more short lines into our beginner’s memory. If you search through all top-level chess games and find ones where the players play the engine-recommended move ten times in a row, that is optimizing for winning (from the players) and forcing (from the search). Ten moves isn’t long enough, we need ~30 moves for a typical game.
Terrible estimate: with 500,000 games in chessgames.com, say there are 50 games with forcing lines of ten moves, a 10,000x reduction. An Oracle can search better, for games that haven’t been played yet. So maybe if Oracle searched through 5 trillion games it would find a game with a forcing line of 20 moves? At some point I question whether chess can be both low variance enough to have these long forcing lines, and also high variance enough to have so many potential games to search through. Of course chess has ample variance if you allow white to play bad moves, but then you’re not winning.
Another approach, trying to find a forcing opening, running through the stats on chessgames.com in a greedy way, I get this “Nimzo-Indian, Samisch” variation, which seems to be playable for both sides, but perhaps slightly favors black:
d4 Nf6 (73% forced—Magnus games)
c4 e6 (72% forced—Magnus games)
Nc3 Bb4 (83% forced—all games)
a3 Bxc3+ (100% forced—all games)
bxc3 c5 (55% forced—all games)
f3 d5 (85% forced—all games)
Multiplying that through gets 20% forcing over six moves. So maybe Oracle is amazingly lucky and there are hitherto undiscovered forcing lines directly from this well-known position to lost positions for black, missed by Stockfish, AlphaZero, and all humans. Well, then Oracle still needs to cover another 30% of the tree and get just as lucky a few more times. If that happens, I think I’m in crisis of faith mode where I have to reevaluate whether grandmaster chess was an elaborate hoax. So many positions we thought were even turn out to be winning for white, everyone missed it, what happened?
Where “fail” means “no plan found”, “memory and time exhausted”, “here’s a plan that involves running a reasoner in real-time” or “feed me observations in real-time and ask me only to generate a local and by-default-inscrutable action”, as listed by so8res above.
It doesn’t help that chess players also search for lines that are forcing, short, and winning, at least some of the time.
You can consider me convinced that the “find forcing lines” approach isn’t going to work.
(How well the perfect book could “genuinely” teach someone is a different question, but that’s definitely not enough to beat Magnus.)
Yeah, this is part of what I was getting at. The narrowness of the task “write a set of instructions for a one-off victory against a particular player” is a crucial part of what makes it seem not-obviously-impossible to me. Fully simulating Magnus should be adequate, but then obviously you’re invoking a reasoner. What I’m uncertain about is if you can write such instructions without invoking a reasoner.