I tried something like this much earlier with a single question, “Can you explain why it’d be hard to make an AGI that believed 222 + 222 = 555”, and got enough pushback from people who didn’t like the framing that I shelved the effort.
Interesting. I kind of like the framing here, but I have written a paper and sequence on the exact opposite question, on why it
would be easy to make an AGI that believes 222+222=555, if you ever had AGI technology, and what you can do with that in terms of safety.
I can honestly say however that the project of writing that thing, in a way that makes the math somewhat accessible, was not easy.
Interesting. I kind of like the framing here, but I have written a paper and sequence on the exact opposite question, on why it would be easy to make an AGI that believes 222+222=555, if you ever had AGI technology, and what you can do with that in terms of safety.
I can honestly say however that the project of writing that thing, in a way that makes the math somewhat accessible, was not easy.