It’s definitely possible to get confused playing Pokémon Red, but as a human, you’re much better at getting unstuck. You try new things, have more consistent strategies, and learn better from mistakes. If you tried as long and as consistently as long as Claude is, even as a 6-year-old, you’d do much better.
I played Pokémon Red as a kid too (still have the cartridge!), it wasn’t easy, but I beat it in something like that 26 hour number IIRC. You have a point that howlongtobeat is biased towards gamers, but it’s the most objective number I can find, and it feels reasonable to me.
I’m not sure! Or well, I agree that 7-year-old me could get unstuck by virtue of having an “additional tool” called “get frustrated and cry until my mom took pity and helped.”[1] But we specifically prevent Claude from doing stuff like that!
I think it’s plausible that if we took an actual 6-year-old and asked them to play Pokemon on a Twitch stream, we’d see many of the things you highlight as weaknesses of Claude: getting stuck against trivial obstacles, forgetting what they were doing, and—yes—complaining that the game is surely broken.
TBC this is exaggerated for effect—I don’t remember actually doing this for Pokemon. And—to your point—I probably did eventually figure out on my own most of the things I remember getting stuck on.
Pokemon is a game literally made to be played and beaten by children. Six years old might be pushing the lower bound, but it didn’t become one of the largest gaming and entertainment franchises in the world by being too difficult to play for children, whom the game is designed for.
Yes, kids get stuck and they do use extra resources like searching up info on game guides (old man moment, before the internet you had to find a friend who had the physical version and would let you borrow or look at it). But is the ability to search the internet the bottleneck that prevents Claude from getting past Mt. Moon in under 50 hours? That does not seem likely. In fact giving it access to the internet where it can get even more lost with potentially additional useless or irrelevant information could make the problem worse.
Yeah, I think that probably if the claim had been “worse than a 9 year old” then I wouldn’t have had much to complain about. I somewhat regret phrasing my original comment as a refutation of the “worse than a 6 year old” and “26 hour” claims, when really I was just using those as a jumping-off point to say some interesting-to-me stuff about how humans also get stuck on trivial obstacles in the same ways that AIs do.
I do feel like it’s a bit cleaner to factor apart Claude’s weaknesses into “memory,” “vision,” and “executive function” rather than bundling those issues together in the way the OP does at times. (Though obviously these are related, especially memory and executive function.) Then I would guess that Claude’s executive function actually isn’t that bad and might even be ≥human level. But it’s hard to say because the memory—especially visual memory—really does seem worse than a 6 year old’s.
I think that probably internet access would help substantially.
It would be so awesome to have such a stream as additional reference point—just one six year old without internet and external help doing a Pokemon run
It’s definitely possible to get confused playing Pokémon Red, but as a human, you’re much better at getting unstuck. You try new things, have more consistent strategies, and learn better from mistakes. If you tried as long and as consistently as long as Claude is, even as a 6-year-old, you’d do much better.
I played Pokémon Red as a kid too (still have the cartridge!), it wasn’t easy, but I beat it in something like that 26 hour number IIRC. You have a point that howlongtobeat is biased towards gamers, but it’s the most objective number I can find, and it feels reasonable to me.
I’m not sure! Or well, I agree that 7-year-old me could get unstuck by virtue of having an “additional tool” called “get frustrated and cry until my mom took pity and helped.”[1] But we specifically prevent Claude from doing stuff like that!
I think it’s plausible that if we took an actual 6-year-old and asked them to play Pokemon on a Twitch stream, we’d see many of the things you highlight as weaknesses of Claude: getting stuck against trivial obstacles, forgetting what they were doing, and—yes—complaining that the game is surely broken.
TBC this is exaggerated for effect—I don’t remember actually doing this for Pokemon. And—to your point—I probably did eventually figure out on my own most of the things I remember getting stuck on.
Pokemon is a game literally made to be played and beaten by children. Six years old might be pushing the lower bound, but it didn’t become one of the largest gaming and entertainment franchises in the world by being too difficult to play for children, whom the game is designed for.
Yes, kids get stuck and they do use extra resources like searching up info on game guides (old man moment, before the internet you had to find a friend who had the physical version and would let you borrow or look at it). But is the ability to search the internet the bottleneck that prevents Claude from getting past Mt. Moon in under 50 hours? That does not seem likely. In fact giving it access to the internet where it can get even more lost with potentially additional useless or irrelevant information could make the problem worse.
Yeah, I think that probably if the claim had been “worse than a 9 year old” then I wouldn’t have had much to complain about. I somewhat regret phrasing my original comment as a refutation of the “worse than a 6 year old” and “26 hour” claims, when really I was just using those as a jumping-off point to say some interesting-to-me stuff about how humans also get stuck on trivial obstacles in the same ways that AIs do.
I do feel like it’s a bit cleaner to factor apart Claude’s weaknesses into “memory,” “vision,” and “executive function” rather than bundling those issues together in the way the OP does at times. (Though obviously these are related, especially memory and executive function.) Then I would guess that Claude’s executive function actually isn’t that bad and might even be ≥human level. But it’s hard to say because the memory—especially visual memory—really does seem worse than a 6 year old’s.
I think that probably internet access would help substantially.
It would be so awesome to have such a stream as additional reference point—just one six year old without internet and external help doing a Pokemon run