The relevance should be clear: in the limit of capabilities, such systems could be dangerous.
What I’m saying is that reaching that limit, or reaching any level qualitatively similar to that limit, via that path, is so implausible, at least to me, that I can’t see a lot of point in even devoting more than half a sentence to the possibility, let alone using it as a central hypothesis in your planning. Thus “irrelevant”.
It’s at least somewhat plausible that you could reach a level that was dangerous, but that’s very different from getting anywhere near that limit. For that matter, it’s at least plausible that you could get dangerous just by “imitation” rather than by “prediction”. So, again, why put so much attention into it?
Except for the steadily-increasing capabilities they continue to display as they scale? Also my general objection to the phrase “no reason”/”no evidence”; there obviously is evidence, if you think that evidence should be screened off please argue that explicitly.
OK, there’s not no evidence. There’s just evidence weak enough that I don’t think it’s worth remarking on.
I accept that they’ve scaled a lot better than anybody would have expected even 5 years ago. And I expect them to keep improving for a while.
But...
They’re not so opaque as all that, and they’re still just using basically pure statistics to do their prediction, and they’re still basically doing just prediction, and they’re still operating with finite resources.
When you observe something that looks like an exponential in real life, the right way to bet it is almost always that it’s really a sigmoid.
Whenever you get a significant innovation, you would expect to see a sudden ramp-up in capability, so actually seeing such a ramp-up, even if it’s bigger than you would have expected, shouldn’t cause you to update that much about the final outcome.
If I wanted to find the thing that worries me most, it’d probably be that there’s no rule that somebody building a real system has to keep the architecture pure. Even if you do start to get diminishing returns from “GPTs” and prediction, you don’t have to stop there. If you keep adding more obvious-to-only-somewhat-unintuitive elements to the architecture, you can get in at the bottoms of more sigmoids. And the effects can easily be synergistic. And what we definitely have is a lot of momentum: many smart people’s attention and a lot of money [1] at stake, plus whatever power you get from the tools already built. That kind of thing is how you get those innovations.
What I’m saying is that reaching that limit, or reaching any level qualitatively similar to that limit, via that path, is so implausible, at least to me, that I can’t see a lot of point in even devoting more than half a sentence to the possibility, let alone using it as a central hypothesis in your planning. Thus “irrelevant”.
It’s at least somewhat plausible that you could reach a level that was dangerous, but that’s very different from getting anywhere near that limit. For that matter, it’s at least plausible that you could get dangerous just by “imitation” rather than by “prediction”. So, again, why put so much attention into it?
OK, there’s not no evidence. There’s just evidence weak enough that I don’t think it’s worth remarking on.
I accept that they’ve scaled a lot better than anybody would have expected even 5 years ago. And I expect them to keep improving for a while.
But...
They’re not so opaque as all that, and they’re still just using basically pure statistics to do their prediction, and they’re still basically doing just prediction, and they’re still operating with finite resources.
When you observe something that looks like an exponential in real life, the right way to bet it is almost always that it’s really a sigmoid.
Whenever you get a significant innovation, you would expect to see a sudden ramp-up in capability, so actually seeing such a ramp-up, even if it’s bigger than you would have expected, shouldn’t cause you to update that much about the final outcome.
If I wanted to find the thing that worries me most, it’d probably be that there’s no rule that somebody building a real system has to keep the architecture pure. Even if you do start to get diminishing returns from “GPTs” and prediction, you don’t have to stop there. If you keep adding more obvious-to-only-somewhat-unintuitive elements to the architecture, you can get in at the bottoms of more sigmoids. And the effects can easily be synergistic. And what we definitely have is a lot of momentum: many smart people’s attention and a lot of money [1] at stake, plus whatever power you get from the tools already built. That kind of thing is how you get those innovations.
Added on edit: and, maybe worse, prestige…