I’m extremely curious about the design process of the knowledge base. Just learning about ClaudePlaysPokemon today and I’m a bit surprised at how naive the store is. There’s a reasonably large amount of research into artificial neural network memory and I’ve suspected for a few years that improvements in knowledge scaffolding is promising for really overcoming hallucinations and now reasoning deficiencies. It’s to the extent that I’ve supported projects and experiments at work to mature knowledge bases and knowledge graphs in anticipation of marrying them to an LLM (non-agentic but potentially agentic in the future).
I wonder if any of the literature from sites like this could help in designing better reasoning scaffolding, one would think rationalism is precisely about systematizing how and why we retain awareness of the world.
Note that the creator stated that the setup is intentionally somewhat underengineered:
I do not claim this is the world’s most incredible agent harness; in fact, I explicitly have tried not to “hyper engineer” this to be like the best chance that exists to beat Pokemon. I think it’d be trivial to build a better computer program to beat Pokemon with Claude in the loop.
This is like meant to be some combination of like “understand what Claude’s good at and Benchmark and understand Claude-alongside-a-simple-agent-harness”, so what that boils down to is this is like a pretty straightforward tool-using agent.
I’m extremely curious about the design process of the knowledge base. Just learning about ClaudePlaysPokemon today and I’m a bit surprised at how naive the store is. There’s a reasonably large amount of research into artificial neural network memory and I’ve suspected for a few years that improvements in knowledge scaffolding is promising for really overcoming hallucinations and now reasoning deficiencies. It’s to the extent that I’ve supported projects and experiments at work to mature knowledge bases and knowledge graphs in anticipation of marrying them to an LLM (non-agentic but potentially agentic in the future).
I wonder if any of the literature from sites like this could help in designing better reasoning scaffolding, one would think rationalism is precisely about systematizing how and why we retain awareness of the world.
Note that the creator stated that the setup is intentionally somewhat underengineered:
I did begin and then abandon a sequence about this, cognitive algorithms as scaffolding. I’m like halfway to disendorsing it though.