If a person wanted to make their prediction of human-level AI entirely based on what was best for them, without regard to truth, when would be the best time? Is twenty years really the sweetest spot?
I think this kind of exercise is helpful for judging the extent to which people’s predictions really are influenced by other motives—I fear it’s tempting to look at whatever people predict and see a story about the incentives that would drive them there, and take their predictions as evidence that they are driven by ulterior motives.
Stock market analysts are in a somewhat similar boat. Strategists and Macro economists make long-term predictions, though their performance is rarely tracked. Short-term forecasts (one month, or one quarter) are centrally compiled, and statistics are assembled on which economists are good forecasters, but this is not the case for long-term predictions. As such, long-term predictions seem to fall into much the same camp as AI predictions here. And many bank analysts explicitly think about the non-epistemic incentives they personally face, in terms of wanting to make an impact, wanting a defensible position, and so on.
However, with economists we see much shorter forecast time horizon. A long-term forecast would be 5 years; many give less. I have never seen an explicit forecast out longer than 10 years. Perhaps this is because they don’t think people would assign any credibility to such forecasts; perhaps the remaining career duration of a macro economist is shorter than those of AI researchers. However, making several incorrect predictions is rarely very damaging to their career. Indeed, because they can selectively emphasis their ex post correct predictions, they’re incentivized to make many short-term predictions.
Prima facie much of this applies to AI commentators as well.
I don’t understand this question. The best time for the emergence of a great optimizer would be shortly after you were born (earlier if your existence were assured somehow).
If an AI is a friendly optimizer, then you want it as soon as possible. If it is randomly friendly or unfriendly, then you don’t want it at all (the quandary we all face). Seems like asking “when” is a lot less relevant than asking “what”. “What” I want is a friendly AI. “When” I get it is of little relevance, so long as it is long enough before my death to grant me “immortality” while maximally (or sufficiently) fulfilling my values.
The question isn’t asking when the best time for the AI to be created is. It’s asking what the best time to predict the AI will be created is. E.g. What prediction sounds close enough to be exciting and to get me that book deal, but far enough away as to be not obviously wrong and so that people will have forgotten about my prediction by the time it hasn’t actually come true. This is an attempt to determine how much the predictions may be influenced by self-interest bias, etc.
The answer to this question depends really heavily on my estimation of MIRI’s capability as an organization and on how hard the control problem turns out to be. My current answer is “the moment the control problem is solved and not a moment sooner”, but I don’t have enough of a grip on the other difficulties involved to say when that would be more concrete.
If a person wanted to make their prediction of human-level AI entirely based on what was best for them, without regard to truth, when would be the best time? Is twenty years really the sweetest spot?
I think this kind of exercise is helpful for judging the extent to which people’s predictions really are influenced by other motives—I fear it’s tempting to look at whatever people predict and see a story about the incentives that would drive them there, and take their predictions as evidence that they are driven by ulterior motives.
Stock market analysts are in a somewhat similar boat. Strategists and Macro economists make long-term predictions, though their performance is rarely tracked. Short-term forecasts (one month, or one quarter) are centrally compiled, and statistics are assembled on which economists are good forecasters, but this is not the case for long-term predictions. As such, long-term predictions seem to fall into much the same camp as AI predictions here. And many bank analysts explicitly think about the non-epistemic incentives they personally face, in terms of wanting to make an impact, wanting a defensible position, and so on.
However, with economists we see much shorter forecast time horizon. A long-term forecast would be 5 years; many give less. I have never seen an explicit forecast out longer than 10 years. Perhaps this is because they don’t think people would assign any credibility to such forecasts; perhaps the remaining career duration of a macro economist is shorter than those of AI researchers. However, making several incorrect predictions is rarely very damaging to their career. Indeed, because they can selectively emphasis their ex post correct predictions, they’re incentivized to make many short-term predictions.
Prima facie much of this applies to AI commentators as well.
I don’t understand this question. The best time for the emergence of a great optimizer would be shortly after you were born (earlier if your existence were assured somehow).
If an AI is a friendly optimizer, then you want it as soon as possible. If it is randomly friendly or unfriendly, then you don’t want it at all (the quandary we all face). Seems like asking “when” is a lot less relevant than asking “what”. “What” I want is a friendly AI. “When” I get it is of little relevance, so long as it is long enough before my death to grant me “immortality” while maximally (or sufficiently) fulfilling my values.
The question isn’t asking when the best time for the AI to be created is. It’s asking what the best time to predict the AI will be created is. E.g. What prediction sounds close enough to be exciting and to get me that book deal, but far enough away as to be not obviously wrong and so that people will have forgotten about my prediction by the time it hasn’t actually come true. This is an attempt to determine how much the predictions may be influenced by self-interest bias, etc.
The answer to this question depends really heavily on my estimation of MIRI’s capability as an organization and on how hard the control problem turns out to be. My current answer is “the moment the control problem is solved and not a moment sooner”, but I don’t have enough of a grip on the other difficulties involved to say when that would be more concrete.