Sorry if this is a stupid question, but this tournament looked to me like a thinly disguised version of:
“Construct a robot that can read code and interpret what it means.”
which is a Really Hard Problem.
Is that not a fair description? Was there some other way to approach the problem?
The only way I can see to go about constructing a GOOD entrant to this is to write something that can take as its input the code of the opponent and interpret what it will actually DO, that can recognize the equivalence between (say):
return DEFECT
and
if 1:
return DEFECT
return COOPERATE
and can interpret things like:
if opp_code == my_code
return COOPERATE
return DEFECT
And I have no idea how to go about doing that. From the fact that the winning entrants were all random, it seems safe to say that no entrants had any idea how to go about doing that either.
One good way to interpret code is to run the code with “eval”, which many submitted bots did. This method has no problems with the examples you gave. One important place it breaks down is with bots that behave randomly. In that case a robot may, by chance, be simulated to cooperate and defect in whatever sequence would make it seem worth cooperating with even if it actually ends up defecting. This, combined with a little luck, made the random bots come out ahead. There are ways to get around this problem, and a few bots did so, but they still didn’t do better random bots because they had less potential for exploitation in this particular pool of entrants.
I agree, my fellow top-ranking-non-source-ignoring player. Saying “nobody could do any better than randomness in this tournament” is strictly true but a bit misleading; the tiny, defect-happy pool with almost 20% random players (the top 3 and also G; he just obfuscated his somewhat) didn’t provide a very favorable structure for more intelligent bots to intelligently navigate, but there was still certainly some navigation.
I’m pretty pleased with how my bot performed; it never got deterministically CD’d and most of its nonrandom mutual defections were against bots who had some unusual trigger condition for defecting based on source composition, not performance, or had very confused performance triggers (e.g. O—why would you want to play your opponent’s anti-defectbot move when you determine they cooperate with cooperatebot?). Some of its mutual defections were certainly due to my detect-size-changes exploit, but so were its many DCs.
Sorry if this is a stupid question, but this tournament looked to me like a thinly disguised version of:
“Construct a robot that can read code and interpret what it means.”
which is a Really Hard Problem.
Is that not a fair description? Was there some other way to approach the problem?
The only way I can see to go about constructing a GOOD entrant to this is to write something that can take as its input the code of the opponent and interpret what it will actually DO, that can recognize the equivalence between (say):
return DEFECT
and
if 1: return DEFECT return COOPERATE
and can interpret things like:
if opp_code == my_code return COOPERATE return DEFECT
And I have no idea how to go about doing that. From the fact that the winning entrants were all random, it seems safe to say that no entrants had any idea how to go about doing that either.
Am I missing something here?
One good way to interpret code is to run the code with “eval”, which many submitted bots did. This method has no problems with the examples you gave. One important place it breaks down is with bots that behave randomly. In that case a robot may, by chance, be simulated to cooperate and defect in whatever sequence would make it seem worth cooperating with even if it actually ends up defecting. This, combined with a little luck, made the random bots come out ahead. There are ways to get around this problem, and a few bots did so, but they still didn’t do better random bots because they had less potential for exploitation in this particular pool of entrants.
I agree, my fellow top-ranking-non-source-ignoring player. Saying “nobody could do any better than randomness in this tournament” is strictly true but a bit misleading; the tiny, defect-happy pool with almost 20% random players (the top 3 and also G; he just obfuscated his somewhat) didn’t provide a very favorable structure for more intelligent bots to intelligently navigate, but there was still certainly some navigation.
I’m pretty pleased with how my bot performed; it never got deterministically CD’d and most of its nonrandom mutual defections were against bots who had some unusual trigger condition for defecting based on source composition, not performance, or had very confused performance triggers (e.g. O—why would you want to play your opponent’s anti-defectbot move when you determine they cooperate with cooperatebot?). Some of its mutual defections were certainly due to my detect-size-changes exploit, but so were its many DCs.
Perfect simulation is not only really hard, it has been proven to be impossible. See http://en.wikipedia.org/wiki/Halting_problem