Yeah, CEV itself seemed like a long shot—but my thought process was that maintaining human control wouldn’t be enough for step one, both because I think it’s not enough at the limit, but also because the human component might inherently be a limiting factor that makes it not very competitive. But the more I thought about it, the weaker that assumption of inherent-ness seemed, so I agree in that the most this post could be saying is that the timeline gap between something like Task AGI and figuring out step 2 is short—but which I expect isn’t very groundbreaking.
Yeah, CEV itself seemed like a long shot—but my thought process was that maintaining human control wouldn’t be enough for step one, both because I think it’s not enough at the limit, but also because the human component might inherently be a limiting factor that makes it not very competitive. But the more I thought about it, the weaker that assumption of inherent-ness seemed, so I agree in that the most this post could be saying is that the timeline gap between something like Task AGI and figuring out step 2 is short—but which I expect isn’t very groundbreaking.