Someone pointed out to me that probably we should calling superintelligence a possible “arms race”. In an “arms race”, you’re competing to have a stronger force than the other person. You want to keep your nose in-front in case of a fight.
Developing superintelligence, on the other hand, is just a plain old race. A technology race. You simply want to get to the destination first.
(Likewise with developing the first nuke, which also involved arms but was not an arms race.)
Developing an AGI (and then ASI) will likely involve a serious of steps involving lower intelligences. There’s already an AI arms race between several large technology companies and keeping your nose in front is already practiced because there’s a lot of utility in having the best AI so far.
So it isn’t true to say that it’s simply a race without important intermediate steps. You don’t just want to get to the destination first, you want to make sure your AI is the best for most of the race for a whole heap of reasons.
If your path to superintelligence spends lots of time in regions with intermediate-grade AIs that can generate power or intelligence, then that is true, so of course the phrase “arms race” aptly describes such situations.
It’s the case where people are designing a superintelligence “from scratch” that the term “arms race” seems inappropriate.
Someone pointed out to me that probably we should calling superintelligence a possible “arms race”. In an “arms race”, you’re competing to have a stronger force than the other person. You want to keep your nose in-front in case of a fight.
Developing superintelligence, on the other hand, is just a plain old race. A technology race. You simply want to get to the destination first.
(Likewise with developing the first nuke, which also involved arms but was not an arms race.)
Developing an AGI (and then ASI) will likely involve a serious of steps involving lower intelligences. There’s already an AI arms race between several large technology companies and keeping your nose in front is already practiced because there’s a lot of utility in having the best AI so far.
So it isn’t true to say that it’s simply a race without important intermediate steps. You don’t just want to get to the destination first, you want to make sure your AI is the best for most of the race for a whole heap of reasons.
If your path to superintelligence spends lots of time in regions with intermediate-grade AIs that can generate power or intelligence, then that is true, so of course the phrase “arms race” aptly describes such situations.
It’s the case where people are designing a superintelligence “from scratch” that the term “arms race” seems inappropriate.