In particular, for folks reading, I symmetrically agree with this part:
In case it is not obvious to people reading, I endorse John’s research program. (Which can maybe be inferred by the fact that I am arguing that it is similar to my own). I think we disagree about what is the most likely path after becoming less confused about agency, but that part of both our plans is yet to be written, and I think the subgoal is enough of a simple concept that I don’t think disagreements about what to do next to have a strong impact on how to do the first step.
… i.e. I endorse Scott’s research program, mine is indeed similar, I wouldn’t be the least bit surprised if we disagree about what comes next but we’re pretty aligned on what to do now.
Also, I realize now that I didn’t emphasize it in the OP, but a large chunk of my “50/50 chance of success” comes from other peoples’ work playing a central role, and the agent foundations team at MIRI is obviously at the top of the list of people whose work is likely to fit that bill. (There’s also the whole topic of producing more such people, which I didn’t talk about in the OP at all, but I’m tentatively optimistic on that front too.)
This all sounds right.
In particular, for folks reading, I symmetrically agree with this part:
… i.e. I endorse Scott’s research program, mine is indeed similar, I wouldn’t be the least bit surprised if we disagree about what comes next but we’re pretty aligned on what to do now.
Also, I realize now that I didn’t emphasize it in the OP, but a large chunk of my “50/50 chance of success” comes from other peoples’ work playing a central role, and the agent foundations team at MIRI is obviously at the top of the list of people whose work is likely to fit that bill. (There’s also the whole topic of producing more such people, which I didn’t talk about in the OP at all, but I’m tentatively optimistic on that front too.)