Note that according to Bucky’s comment, we still get good play up to move 39 (with ChatGPT instead of Sydney), where “good” doesn’t just mean legal but actual strong moves. So I wouldn’t be at all surprised if it’s still mostly reasonable at move 50 (though I do generally expect it to get somewhat worse the longer the game is). Might get around to testing at least this one point later if no one else does.
I asked Sydney to reconstruct the board position on the 50th move of two different games, and saw what Simon predicted—a significant drop in performance. Here’s a link of two games I tried using your prompt: https://imgur.com/a/ch9U6oZ
While there is some overlap, what Sydney thinks the games look like doesn’t have much resemblance to the actual games.
I also repeatedly asked Sydney to continue the games using Stockfish (with a few slightly different prompts), but for some reason once the game description is long enough, Sydney refuses to do anything. It either says it can’t access Stockfish, or that using Stockfish would be cheating.
Note that according to Bucky’s comment, we still get good play up to move 39 (with ChatGPT instead of Sydney), where “good” doesn’t just mean legal but actual strong moves. So I wouldn’t be at all surprised if it’s still mostly reasonable at move 50 (though I do generally expect it to get somewhat worse the longer the game is). Might get around to testing at least this one point later if no one else does.
I asked Sydney to reconstruct the board position on the 50th move of two different games, and saw what Simon predicted—a significant drop in performance. Here’s a link of two games I tried using your prompt: https://imgur.com/a/ch9U6oZ
While there is some overlap, what Sydney thinks the games look like doesn’t have much resemblance to the actual games.
I also repeatedly asked Sydney to continue the games using Stockfish (with a few slightly different prompts), but for some reason once the game description is long enough, Sydney refuses to do anything. It either says it can’t access Stockfish, or that using Stockfish would be cheating.