There are two agents that play a simplified soccer game. One being the goalkeeper and one trying to score a goal. After an initial training phase, the goalkeeper starts acting strange, apparently random. The movements are adversarial noise, optimized to “confuse” the other player as much as possible to stop them from scoring a goal.
Also an example of weird things that can happen in high-dimensional spaces.
oh yeah, I’ve seen that one before—really awesome stuff! I guess you could say the goalkeeper discovers a “mental” dimension whereby it can beat the attacker easier than if it uses the “physical” dimensions of directly blocking.
This all also feels related to Goodhart’s law—though subtly different...
I’m reminded of an ICLR 2020 paper describing a case of “black magic” in a reinforcement learning setting, if you like:
https://bair.berkeley.edu/blog/2020/03/27/attacks/
There are two agents that play a simplified soccer game. One being the goalkeeper and one trying to score a goal. After an initial training phase, the goalkeeper starts acting strange, apparently random. The movements are adversarial noise, optimized to “confuse” the other player as much as possible to stop them from scoring a goal.
Also an example of weird things that can happen in high-dimensional spaces.
oh yeah, I’ve seen that one before—really awesome stuff! I guess you could say the goalkeeper discovers a “mental” dimension whereby it can beat the attacker easier than if it uses the “physical” dimensions of directly blocking.
This all also feels related to Goodhart’s law—though subtly different...