Sizes of superintelligence: hidden assumption in AI safety
“Superintelligence” could mean different things, and to deconfuse this I created a short classification:
Levels of superintelligence:
1. Above human
2. Google size
3. Humanity 100 years performance in 1 year.
4. Whole biological evolution equivalent in 1 year.
5. Jupiter brain with billion past simulations
6. Galactic brain.
7. 3^3^3 IQ superintelligence
X-risks appear between 2nd and 3rd levels.
Nanobot is above 3.
Each level also requires a minimum size of code, memory and energy consumption.
An AI which run out of the box into the internet is between levels 1 and 2 and will be limited in computational resources, so modeling of nanobot from first principle will be difficult for it.
I’m not sure what “Whole biological evolution equivalent” means. Clearly, you do not mean the nominal compute of evolution—which is probably close to Jupiter brain. I think you are appealing to something that would be able to simulate evolution with high fidelity?
Actually I meant something like this, but could downsize the claim to ‘create something as complex as human body’. Simulation of billions of other species will be redundant.
Sizes of superintelligence: hidden assumption in AI safety
“Superintelligence” could mean different things, and to deconfuse this I created a short classification:
Levels of superintelligence:
1. Above human
2. Google size
3. Humanity 100 years performance in 1 year.
4. Whole biological evolution equivalent in 1 year.
5. Jupiter brain with billion past simulations
6. Galactic brain.
7. 3^3^3 IQ superintelligence
X-risks appear between 2nd and 3rd levels.
Nanobot is above 3.
Each level also requires a minimum size of code, memory and energy consumption.
An AI which run out of the box into the internet is between levels 1 and 2 and will be limited in computational resources, so modeling of nanobot from first principle will be difficult for it.
I’m not sure what “Whole biological evolution equivalent” means. Clearly, you do not mean the nominal compute of evolution—which is probably close to Jupiter brain. I think you are appealing to something that would be able to simulate evolution with high fidelity?
Actually I meant something like this, but could downsize the claim to ‘create something as complex as human body’. Simulation of billions of other species will be redundant.