I started a dialogue with @Alex_Altair a few months ago about the tractability of certain agent foundations problems, especially the agent-like structure problem. I saw it as insufficiently well-defined to make progress on anytime soon. I thought the lack of similar results in easy settings, the fuzziness of the “agent”/”robustly optimizes” concept, and the difficulty of proving things about a program’s internals given its behavior all pointed against working on this. But it turned out that we maybe didn’t disagree on tractability much, it’s just that Alex had somewhat different research taste, plus thought fundamental problems in agent foundations must be figured out to make it to a good future, and therefore working on fairly intractable problems can still be necessary. This seemed pretty out of scope and so I likely won’t publish.
Now that this post is out, I feel like I should at least make this known. I don’t regret attempting the dialogue, I just wish we had something more interesting to disagree about.
I started a dialogue with @Alex_Altair a few months ago about the tractability of certain agent foundations problems, especially the agent-like structure problem. I saw it as insufficiently well-defined to make progress on anytime soon. I thought the lack of similar results in easy settings, the fuzziness of the “agent”/”robustly optimizes” concept, and the difficulty of proving things about a program’s internals given its behavior all pointed against working on this. But it turned out that we maybe didn’t disagree on tractability much, it’s just that Alex had somewhat different research taste, plus thought fundamental problems in agent foundations must be figured out to make it to a good future, and therefore working on fairly intractable problems can still be necessary. This seemed pretty out of scope and so I likely won’t publish.
Now that this post is out, I feel like I should at least make this known. I don’t regret attempting the dialogue, I just wish we had something more interesting to disagree about.
This seems valuable! I’d be curious to hear more !!