Far and away the most common failure mode among self-identifying alignment researchers is to look for Clever Ways To Avoid Doing Hard Things [...]
seems true as well. However, there was something in this section that didn’t seem quite right to me.
Say that you have identified the Hamming Problem at lowest resolution be getting the outcome “AI doesn’t cause extinction or worse”. However, if you zoom in a little bit you might find that there are different narratives that lead to the same goal. For example:
AGI isn’t developed due to event(s)
AGI is developed safely due to event(s)
At this level I would say that it is correct to go for the easier narrative. Going for harder problems seem to be when you zoom into these narratives.
For each path you can imagine a set of events (e.g. research break-throughs) that are necessary and sufficient to solve the end-goal. Here I’m unsure but my intuition tells me that the marginal impact would often be greater working on the necessary parts that are the hardest as these are the ones that are least likely to be solved without intervention.
Of course working on something that isn’t necessary in any narrative would probably be easier in most cases but would never be a Hamming Problem.
For each path you can imagine a set of events (e.g. research break-throughs) that are necessary and sufficient to solve the end-goal. Here I’m unsure but my intuition tells me that the marginal impact would often be greater working on the necessary parts that are the hardest as these are the ones that are least likely to be solved without intervention.
This is exactly right, and those are the things which I would call Hamming Problems or the Hard Parts.
I suppose I would just like to see more people start at an earlier level and from that vantage point you might actually want to switch to a path with easier parts.
I agree with that statement and this statement
seems true as well. However, there was something in this section that didn’t seem quite right to me.
Say that you have identified the Hamming Problem at lowest resolution be getting the outcome “AI doesn’t cause extinction or worse”. However, if you zoom in a little bit you might find that there are different narratives that lead to the same goal. For example:
AGI isn’t developed due to event(s)
AGI is developed safely due to event(s)
At this level I would say that it is correct to go for the easier narrative. Going for harder problems seem to be when you zoom into these narratives.
For each path you can imagine a set of events (e.g. research break-throughs) that are necessary and sufficient to solve the end-goal. Here I’m unsure but my intuition tells me that the marginal impact would often be greater working on the necessary parts that are the hardest as these are the ones that are least likely to be solved without intervention.
Of course working on something that isn’t necessary in any narrative would probably be easier in most cases but would never be a Hamming Problem.
This is exactly right, and those are the things which I would call Hamming Problems or the Hard Parts.
I suppose I would just like to see more people start at an earlier level and from that vantage point you might actually want to switch to a path with easier parts.