I agree that when you know about a critical threshold, as with nukes or orbits, you can and should predict a discontinuity there. (Sufficient specific knowledge is always going to allow you to outperform a general heuristic.) I think that (a) such thresholds are rare in general and (b) in AI in particular there is no such threshold. (According to me (b) seems like the biggest difference between Eliezer and Paul.)
Some thoughts on aging:
It does in fact seem surprising, given the complexity of biology relative to physics, if there is a single core cause and core solution that leads to a discontinuity.
I would a priori guess that there won’t be a core solution. (A core cause seems more plausible, and I’ll roll with it for now.) Instead, we see a sequence of solutions that intervene on the core problem in different ways, each of which leads to some improvement on lifespan, and discovering these at different times leads to a smoother graph.
That being said, are people putting in a lot of effort into solving aging in mice? Everyone seems to constantly be saying that we’re putting in almost no effort whatsoever. If that’s true then a jumpy graph would be much less surprising.
As a more specific scenario, it seems possible that the graph of mouse lifespan over time looks basically flat, because we were making no progress due to putting in ~no effort. I could totally believe in this world that someone puts in some effort and we get a discontinuity, or even that the near-zero effort we’re putting in finds some intervention this year (but not in previous years) which then looks like a discontinuity.
If we had a good operationalization, and people are in fact putting in a lot of effort now, I could imagine putting my $100 to your $300 on this (not going beyond 1:3 odds simply because you know way more about aging than I do).
I’m not particularly enthusiastic about betting at 75%, that seems like it’s already in the right ballpark for where the probability should be. So I guess we’ve successfully Aumann agreed on that particular prediction.
I agree that when you know about a critical threshold, as with nukes or orbits, you can and should predict a discontinuity there. (Sufficient specific knowledge is always going to allow you to outperform a general heuristic.) I think that (a) such thresholds are rare in general and (b) in AI in particular there is no such threshold. (According to me (b) seems like the biggest difference between Eliezer and Paul.)
Some thoughts on aging:
It does in fact seem surprising, given the complexity of biology relative to physics, if there is a single core cause and core solution that leads to a discontinuity.
I would a priori guess that there won’t be a core solution. (A core cause seems more plausible, and I’ll roll with it for now.) Instead, we see a sequence of solutions that intervene on the core problem in different ways, each of which leads to some improvement on lifespan, and discovering these at different times leads to a smoother graph.
That being said, are people putting in a lot of effort into solving aging in mice? Everyone seems to constantly be saying that we’re putting in almost no effort whatsoever. If that’s true then a jumpy graph would be much less surprising.
As a more specific scenario, it seems possible that the graph of mouse lifespan over time looks basically flat, because we were making no progress due to putting in ~no effort. I could totally believe in this world that someone puts in some effort and we get a discontinuity, or even that the near-zero effort we’re putting in finds some intervention this year (but not in previous years) which then looks like a discontinuity.
If we had a good operationalization, and people are in fact putting in a lot of effort now, I could imagine putting my $100 to your $300 on this (not going beyond 1:3 odds simply because you know way more about aging than I do).
I’m not particularly enthusiastic about betting at 75%, that seems like it’s already in the right ballpark for where the probability should be. So I guess we’ve successfully Aumann agreed on that particular prediction.