Okay, I downloaded KataGo to see how it plays and read its rules description. It seems actually been trained so that under area rules it doesn’t maximize its points.
This is surprising to me because one of the aspects of AlphaGo that was annoying was that it didn’t maximize the number of points with which it wins the game but only cared about winning. KataGo seems to play under territory rules to maximize points and not do those negative point moves that AlphaGo makes at the end of the game if it’s ahead by a lot of points.
Humans generally do care about the score at the end of the game so that behavior, under rules that care about area, is surprising to me.
Official Chinese rules do have a concept of removing dead stones. All the KGS rulesets also have an option for handling dead stone removal.
A fix that would let KataGo beat the adversarial policy would be to implement rules for Chinese go that are more like the actual KGS rules (likely by just letting it have the cleanup phase with Chinese rules as well) and generally tell KataGo to optimize winning with the highest importance and then optimize the score and lastly optimize for a minimum amount of moves played before passing.
If you do that you could train it on the different rule sets and it would produce this problem. The fact that you need to do that to prevent the adversarial policy is indeed interesting.
That suggests if you have one metric, adding a second metric that’s a proxy for the first metric as a secondary optimization goal can be helpful to get around some adversarial attacks. Especially, if the first metric is binary and the second one has a lot more possible values.
It’s interesting here that humans, do naturally care about scores when you let them play Go which is what gets them to avoid this kind of adversarial attack.
What KataGo tries to maximize is basically winning probability plus epsilon times score difference. (It’s not exactly that; I don’t remember exactly what it is; but that’s the right kind of idea.) So it mostly wants to win rather than lose, but prefers to win by more if the cost in winning probability is small, which as you say helps to avoid the sort of “slack” moves that AlphaGo and Leela Zero tend to make once the winner is more or less decided.
The problem here seems to be that it’s not preferring to win by more under area rules. If it would prefer by more points under area rules, it would capture all the stones before passing. It doesn’t do that, once it thinks that it has enough points to win anyway under area rules.
This attack is basically about giving KataGo the impression that it has enough points anyway and doesn’t need to capture stones to win.
Likely the heuristic of time score difference does not reward getting more points over passing but it does reward playing a move that’s worth more points over a move that’s worth less.
I’m not sure I understand. With any rules that allow the removal of dead stones, there is no advantage to capturing them. (With territory-scoring rules, capturing them makes you worse off. With area-scoring rules, capturing them makes no difference to the score.) And with rules that don’t allow the removal of dead stones, white is losing outright (and therefore needs to capture those stones even if it’s only winning versus losing that matters). How would caring more about score make KG more inclined to bother capturing the stones?
With area-scoring rules that don’t allow the removal of dead stones in normal training games, KataGo has to decide whether it can already pass or whether it should go through the work of capturing any remaining stones. I was letting KataGo play one training game and it looked to me like its default strategy in games is not to capture all the stones but only enough to win by a sufficient margin.
It doesn’t have a habit of “always capturing all the stones to get maximum score under area rules”. If it would have that habit I don’t think it would show this failure case.
In training games I think the rules it’s using do allow the removal of dead stones. If it chooses not to remove them it isn’t because it’s not caring about points it would have gained by removing them, it’s because it doesn’t think it would gain any points by removing them.
There is no possible habit of “always capturing all the stones to get maximum score under area rules”. Even under area rules you don’t get more points for capturing the stones (unless the stones are not actually dead according to the rules you’re using, or in human games according to negotiation with the opponent).
I think that currently under area scoring rules KataGo behaves in a way that it doesn’t capture all stones that would be dead by human convention but that are not dead by KataGo’s rules provided capturing them isn’t necessary to win the game.
That’s correct, at least roughly—the important difference is that it’s not “isn’t necessary to win the game” but “doesn’t make any difference to the outcome, including score difference”—but I don’t see what it has to do with the more specific thing you said above:
The problem seems to be that it’s not preferring to win by more under area rules.
KataGo does prefer to win by more, whatever rules it’s playing under; a stronger preference for winning by more would not (so far as I can see) make any difference to its play in positions like the ones reached by the adversarial agent; KataGo does not generally think “that it has enough points anyway and doesn’t need to capture stones to win” and even if it did that wouldn’t make the difference between playing on and passing in this situation.
Unless, again, I’m missing something, but we seem to be having some sort of communication difficulty because nothing you write seems to me responsive to what I’m saying (and quite possibly it feels the same way to you, with roles reversed).
What makes you believe that KataGo is “not preferring to win by more under area rules”?
Okay, I downloaded KataGo to see how it plays and read its rules description. It seems actually been trained so that under area rules it doesn’t maximize its points.
This is surprising to me because one of the aspects of AlphaGo that was annoying was that it didn’t maximize the number of points with which it wins the game but only cared about winning. KataGo seems to play under territory rules to maximize points and not do those negative point moves that AlphaGo makes at the end of the game if it’s ahead by a lot of points.
Humans generally do care about the score at the end of the game so that behavior, under rules that care about area, is surprising to me.
Official Chinese rules do have a concept of removing dead stones. All the KGS rulesets also have an option for handling dead stone removal.
A fix that would let KataGo beat the adversarial policy would be to implement rules for Chinese go that are more like the actual KGS rules (likely by just letting it have the cleanup phase with Chinese rules as well) and generally tell KataGo to optimize winning with the highest importance and then optimize the score and lastly optimize for a minimum amount of moves played before passing.
If you do that you could train it on the different rule sets and it would produce this problem. The fact that you need to do that to prevent the adversarial policy is indeed interesting.
That suggests if you have one metric, adding a second metric that’s a proxy for the first metric as a secondary optimization goal can be helpful to get around some adversarial attacks. Especially, if the first metric is binary and the second one has a lot more possible values.
It’s interesting here that humans, do naturally care about scores when you let them play Go which is what gets them to avoid this kind of adversarial attack.
What KataGo tries to maximize is basically winning probability plus epsilon times score difference. (It’s not exactly that; I don’t remember exactly what it is; but that’s the right kind of idea.) So it mostly wants to win rather than lose, but prefers to win by more if the cost in winning probability is small, which as you say helps to avoid the sort of “slack” moves that AlphaGo and Leela Zero tend to make once the winner is more or less decided.
The problem here seems to be that it’s not preferring to win by more under area rules. If it would prefer by more points under area rules, it would capture all the stones before passing. It doesn’t do that, once it thinks that it has enough points to win anyway under area rules.
This attack is basically about giving KataGo the impression that it has enough points anyway and doesn’t need to capture stones to win.
Likely the heuristic of time score difference does not reward getting more points over passing but it does reward playing a move that’s worth more points over a move that’s worth less.
I’m not sure I understand. With any rules that allow the removal of dead stones, there is no advantage to capturing them. (With territory-scoring rules, capturing them makes you worse off. With area-scoring rules, capturing them makes no difference to the score.) And with rules that don’t allow the removal of dead stones, white is losing outright (and therefore needs to capture those stones even if it’s only winning versus losing that matters). How would caring more about score make KG more inclined to bother capturing the stones?
With area-scoring rules that don’t allow the removal of dead stones in normal training games, KataGo has to decide whether it can already pass or whether it should go through the work of capturing any remaining stones. I was letting KataGo play one training game and it looked to me like its default strategy in games is not to capture all the stones but only enough to win by a sufficient margin.
It doesn’t have a habit of “always capturing all the stones to get maximum score under area rules”. If it would have that habit I don’t think it would show this failure case.
In training games I think the rules it’s using do allow the removal of dead stones. If it chooses not to remove them it isn’t because it’s not caring about points it would have gained by removing them, it’s because it doesn’t think it would gain any points by removing them.
There is no possible habit of “always capturing all the stones to get maximum score under area rules”. Even under area rules you don’t get more points for capturing the stones (unless the stones are not actually dead according to the rules you’re using, or in human games according to negotiation with the opponent).
What am I missing?
I think that currently under area scoring rules KataGo behaves in a way that it doesn’t capture all stones that would be dead by human convention but that are not dead by KataGo’s rules provided capturing them isn’t necessary to win the game.
That’s correct, at least roughly—the important difference is that it’s not “isn’t necessary to win the game” but “doesn’t make any difference to the outcome, including score difference”—but I don’t see what it has to do with the more specific thing you said above:
KataGo does prefer to win by more, whatever rules it’s playing under; a stronger preference for winning by more would not (so far as I can see) make any difference to its play in positions like the ones reached by the adversarial agent; KataGo does not generally think “that it has enough points anyway and doesn’t need to capture stones to win” and even if it did that wouldn’t make the difference between playing on and passing in this situation.
Unless, again, I’m missing something, but we seem to be having some sort of communication difficulty because nothing you write seems to me responsive to what I’m saying (and quite possibly it feels the same way to you, with roles reversed).
What makes you believe that KataGo is “not preferring to win by more under area rules”?