A different measure than IQ might be useful at some point. An IQ of X effectively means you would need a population of Y humans or more to expect to find at least one human with an IQ of X. As IQs get larger, say over 300, the number of humans you would need in a population to expect to find at least one human with such an IQ becomes ridiculous. Since there are intelligence levels that will not be found in human populations of any size, the minimum population size needed to expect to find someone with IQ X tends to infinity as IQ approaches some fixed value (say, 1000). IQ above that point is undefined.
It would be nice to find a new measure of intelligence that could be used to measure differences between humans and other humans, and also differences between humans and AI. But can we design such a measure? I think raw computing power doesn’t work (how do you compare humans to other humans? Humans to an AI with great hardware but terrible software?)
Could you design a questionnaire that you know the correct answers to, that a very intelligent AI (500 IQ?) could not score perfectly on, but an extremely intelligent AI (1000+ IQ) could score perfectly on? If not, how could we design a measure of intelligence that goes beyond our own intelligence?
Maybe we could define an intelligence factor x to be something like: The average x value for humans is zero. If your x value is 1 greater than mine, then you will outwit me and get what you want 90% of the time, if our utility functions are in direct conflict, such that only one of us can get what we want, assuming we have equal capabilities, and the environment is sufficiently complex. With this scale, I suspect humans probably range in x-factors from −2 to 2, or −3 to 3 if we’re being generous. This scale could let us talk about superintelligences as having an x-factor of 5, or an x-factor of 10, or so on. For example, a superintelligence with an x-factor of 5 has some chance of winning against a superintelligence with an x-factor of 6, but is basically outmatched by a superintelligence with an x-factor of 8.
The reason the “sufficiently complex environment” clause exists, is that superintelligences with x-factors of 10 and 20 may both find the physically optimal strategy for success in the real world, and so who wins may simply be down to chance. We can say an environment where there ceases to be a difference in the strategies between intelligences with an x-factor of 5 and and x-factor of 6 has a complexity factor of 5. I would guess the real world has a complexity factor of around 8, but I have no idea.
I would be terrified of any AI with an x-factor of 4-ish, and Yudkowsky seems to be describing an AI with an x-factor of 5 or 6.
X-factor does seem better than IQ, of course with the proviso that anybody who starts trying to do actual math with either one, or indeed to use it for anything other than this kind of basically qualitative talk, is in serious epistemic trouble.
I would suggest that humans run more like −2 to 1 than like −3 to 3. I guess there could be a very, very few 2s.
I get the impression that, except when he’s being especially careful for some specific reason, EY tends to speak as though the X-factor of an AI could and would quickly run up high enough that you couldn’t measure it. More like 20 or 30 than 5 or 6; basically deity-level. Maybe it’s a habit from the 1995 era, or maybe he has some reason to believe that that I don’t understand.
Personally, I have the general impression that you’d be hard pressed to get to 3 with an early ML-based AI, and I think that the “equal capabilities” handicap could realistically be made significant. Maybe 3?
A different measure than IQ might be useful at some point. An IQ of X effectively means you would need a population of Y humans or more to expect to find at least one human with an IQ of X. As IQs get larger, say over 300, the number of humans you would need in a population to expect to find at least one human with such an IQ becomes ridiculous. Since there are intelligence levels that will not be found in human populations of any size, the minimum population size needed to expect to find someone with IQ X tends to infinity as IQ approaches some fixed value (say, 1000). IQ above that point is undefined.
It would be nice to find a new measure of intelligence that could be used to measure differences between humans and other humans, and also differences between humans and AI. But can we design such a measure? I think raw computing power doesn’t work (how do you compare humans to other humans? Humans to an AI with great hardware but terrible software?)
Could you design a questionnaire that you know the correct answers to, that a very intelligent AI (500 IQ?) could not score perfectly on, but an extremely intelligent AI (1000+ IQ) could score perfectly on? If not, how could we design a measure of intelligence that goes beyond our own intelligence?
Maybe we could define an intelligence factor x to be something like: The average x value for humans is zero. If your x value is 1 greater than mine, then you will outwit me and get what you want 90% of the time, if our utility functions are in direct conflict, such that only one of us can get what we want, assuming we have equal capabilities, and the environment is sufficiently complex. With this scale, I suspect humans probably range in x-factors from −2 to 2, or −3 to 3 if we’re being generous. This scale could let us talk about superintelligences as having an x-factor of 5, or an x-factor of 10, or so on. For example, a superintelligence with an x-factor of 5 has some chance of winning against a superintelligence with an x-factor of 6, but is basically outmatched by a superintelligence with an x-factor of 8.
The reason the “sufficiently complex environment” clause exists, is that superintelligences with x-factors of 10 and 20 may both find the physically optimal strategy for success in the real world, and so who wins may simply be down to chance. We can say an environment where there ceases to be a difference in the strategies between intelligences with an x-factor of 5 and and x-factor of 6 has a complexity factor of 5. I would guess the real world has a complexity factor of around 8, but I have no idea.
I would be terrified of any AI with an x-factor of 4-ish, and Yudkowsky seems to be describing an AI with an x-factor of 5 or 6.
X-factor does seem better than IQ, of course with the proviso that anybody who starts trying to do actual math with either one, or indeed to use it for anything other than this kind of basically qualitative talk, is in serious epistemic trouble.
I would suggest that humans run more like −2 to 1 than like −3 to 3. I guess there could be a very, very few 2s.
I get the impression that, except when he’s being especially careful for some specific reason, EY tends to speak as though the X-factor of an AI could and would quickly run up high enough that you couldn’t measure it. More like 20 or 30 than 5 or 6; basically deity-level. Maybe it’s a habit from the 1995 era, or maybe he has some reason to believe that that I don’t understand.
Personally, I have the general impression that you’d be hard pressed to get to 3 with an early ML-based AI, and I think that the “equal capabilities” handicap could realistically be made significant. Maybe 3?