Oh, I was imagining something like “well, our current metals aren’t strong enough, what if we developed stronger ones?”, and then focusing on metallurgy. And this is making forward progress—you can build a taller tower out of steel than out of iron—but it’s missing more fundamental issues like “you’re not going to be able to drive on a bridge that’s perpendicular to gravity, and the direction of gravity will change over the course of the trip” or “the moon moves relative to the earth, such that your bridge won’t be able to be one object”, which will sink the project even if you can find a supremely strong metal.
For example, let’s consider Anca Dragan’s research direction that I’m going to summarize as “getting present-day robots to understand what humans around them want and are doing so that they can achieve their goals / cooperate more effectively.” (In mildly adversarial situations like driving, you don’t want to make a cooperatebot, but rather something that follows established norms / prevents ‘cutting’ and so on, but when you have a human-robot team you do care mostly about effective cooperation.)
My guess is this 1) will make the world a better place in the short run under ‘normal’ conditions (most obviously through speeding up adoption of autonomous vehicles and making them more effective) and 2) does not represent meaningful progress towards aligning transformative AI systems. [My model of Eliezer notes that actually he’s making a weaker claim, which is something more like “he’s not surprised by the results of her papers”, which still allows for them to be “progress in the published literature”.]
When I imagine “how do I move the ball forward now?” I find myself drawn towards projects like those, and less to projects like “stare at the nature of cognition until I see a way through the constraints”, which feels like the sort of thing that I would need to do to actually shift my sense of doom.
Oh, I was imagining something like “well, our current metals aren’t strong enough, what if we developed stronger ones?”, and then focusing on metallurgy. And this is making forward progress—you can build a taller tower out of steel than out of iron—but it’s missing more fundamental issues like “you’re not going to be able to drive on a bridge that’s perpendicular to gravity, and the direction of gravity will change over the course of the trip” or “the moon moves relative to the earth, such that your bridge won’t be able to be one object”, which will sink the project even if you can find a supremely strong metal.
For example, let’s consider Anca Dragan’s research direction that I’m going to summarize as “getting present-day robots to understand what humans around them want and are doing so that they can achieve their goals / cooperate more effectively.” (In mildly adversarial situations like driving, you don’t want to make a cooperatebot, but rather something that follows established norms / prevents ‘cutting’ and so on, but when you have a human-robot team you do care mostly about effective cooperation.)
My guess is this 1) will make the world a better place in the short run under ‘normal’ conditions (most obviously through speeding up adoption of autonomous vehicles and making them more effective) and 2) does not represent meaningful progress towards aligning transformative AI systems. [My model of Eliezer notes that actually he’s making a weaker claim, which is something more like “he’s not surprised by the results of her papers”, which still allows for them to be “progress in the published literature”.]
When I imagine “how do I move the ball forward now?” I find myself drawn towards projects like those, and less to projects like “stare at the nature of cognition until I see a way through the constraints”, which feels like the sort of thing that I would need to do to actually shift my sense of doom.