Less prosaically, it’s not impossible that a stronger or more solidly grounded theory of semantics or of interoperable world-models might prove to be the “last missing piece” between us and AGI; that said, given that my research path primarily involves things like finding and constructing conceptual tools, writing mathematical proofs, and reasoning about bounds on accumulating errors—and not things like training new frontier models—I think the risk/dual-use-hazard of my proposed work is minimal.
I don’t really understand this argument. Why wouldn’t having a better theory of semantics and concepts help people build better AIs, but still do a good job of describing what’s going on in smart AIs? Like, you might think the more things you know about smart AIs, the easier it would be to build them—where does this argument break?
The thing you imply here is that it’s pretty different from stuff people currently do to train frontier models, but you already told me that scaling frontier models was really unlikely to lead to AGI, so why should that give me any comfort?
Like, you might think the more things you know about smart AIs, the easier it would be to build them—where does this argument break?
I mean… it doesn’t? I guess I mostly think that either what I’m working on is totally off the capabilities pathway, or if it’s somehow on one, then I don’t think whatever minor framework improvement or suggestion for a mental frame that I come up with is going to push things all that far? Which I agree is kind of a depressing thing to expect of your work, but I argue that that’s the most likely two outcomes here. Does that address that?
I don’t really understand this argument. Why wouldn’t having a better theory of semantics and concepts help people build better AIs, but still do a good job of describing what’s going on in smart AIs? Like, you might think the more things you know about smart AIs, the easier it would be to build them—where does this argument break?
The thing you imply here is that it’s pretty different from stuff people currently do to train frontier models, but you already told me that scaling frontier models was really unlikely to lead to AGI, so why should that give me any comfort?
I mean… it doesn’t? I guess I mostly think that either what I’m working on is totally off the capabilities pathway, or if it’s somehow on one, then I don’t think whatever minor framework improvement or suggestion for a mental frame that I come up with is going to push things all that far? Which I agree is kind of a depressing thing to expect of your work, but I argue that that’s the most likely two outcomes here. Does that address that?