During a panel discussion at the most recent Singularity Summit, Eliezer speculated that he might have ended up as a science fiction author, but then quickly added:
I have to remind myself that it’s not what’s the most fun to do, it’s not even what you have talent to do, it’s what you need to do that you ought to be doing.
Shortly thereafter, Peter Thiel expressed a wish that all the people currently working on string theory would shift their attention to AI or aging; no disagreement was heard from anyone present.
I would therefore like to ask Eliezer whether he in fact believes that the only two legitimate occupations for an intelligent person in our current world are (1) working directly on Singularity-related issues, and (2) making as much money as possible on Wall Street in order to donate all but minimal living expenses to SIAI/Methuselah/whatever.
How much of existing art and science would he have been willing to sacrifice so that those who created it could instead have been working on Friendly AI? If it be replied that the work of, say, Newton or Darwin was essential in getting us to our current perspective wherein we have a hope of intelligently tackling this problem, might the same not hold true in yet unknown ways for string theorists? And what of Michelangelo, Beethoven, and indeed science fiction? Aren’t we allowed to have similar fun today? For a living, even?
Somewhat related, AGI is such an enormously difficult topic, requiring intimate familiarity with so many different fields, that the vast majority of people (and I count myself among them) simply aren’t able to contribute effectively to it.
I’d be interested to know if he thinks there are any singularity-related issues that are important to be worked on, but somewhat more accessible, that are more in need of contributions of man-hours rather than genius-level intellect. Is the only way a person of more modest talents can contribute through donations?
Depends on what you mean by ‘modest’. Probably 60% of Americans could contribute donations without serious lifestyle consequences and 20% of Americans could contribute over a quarter of their incomes without serious consequences. By contrast, only 10% have the reasoning abilities to identify the best large category of causes and only 1% have the reasoning abilities to identify the very best cause without a large luck element being involved. By working hard, most of that 1% could also become part of the affluent 20% of Americans who could make large donations. A similar fraction might be able to help via fund-raising efforts and by aggressively networking and sharing the contacts that they are able to build with us. A smaller but only modestly smaller fraction might be able to meaningfully contribute to SIAI’s effort via seriously committed offers of volunteer effort, but definitely not via volunteer efforts uncoupled to serious commitment. Almost no-one can do FAI, or even recognize talent at a level capable of doing FAI, but if more people were doing the easier things it wouldn’t be nearly so hard to find people who could do the core work.
Almost no-one can do FAI, or even recognize talent at a level capable of doing FAI, but if more people were doing the easier things it wouldn’t be nearly so hard to find people who could do the core work.
SIAI keeps supporting this attitude, yet I don’t believe it, at least in the way it’s presented. A good mathematician who gets to understand the problem statement and succeeds in weeding out the standard misunderstanding can contribute as well as anyone, at this stage where we have no field. Creating a programme that would allow people to reliably get to work on the problem requires material to build upon, and there is still nothing, no matter of what quality. Systematizing the connections with existing science, trying to locate the place of FAI project in it, is something that only requires expertise in that science and understanding of FAI problem statement. At the very least, a dozen steps in, we’ll have a useful curriculum, to get folks up to speed in the right direction.
We have some experience with this, but if you want to discuss the details more with myself or some other SIAI people we will be happy to do so, and probably to have you come visit some time and get some experience with what we do. You may have ways of contributing substantially, theoretically or managerially. We’ll have to see.
Perhaps what we should do is take all our funds and create a school for AI researchers. This would be an investment for the long hall. You and I may not be super geniuses, but I sure think I could raise some super geniuses.
Also, I feel like this topic deserves more than one comment thread.
How much of existing art and science would he have been willing to sacrifice so that those who created it could instead have been working on Friendly AI?
I don’t know about Eliezer, but I would be able to sacrifice quite a lot; perhaps all of art. If humanity spreads through the galaxy there will be way more than enough time for all that.
If it be replied that the work of, say, Newton or Darwin was essential in getting us to our current perspective wherein we have a hope of intelligently tackling this problem, might the same not hold true in yet unknown ways for string theorists?
It might. But their expected contribution would be much greater if they looked at the problem to see how they could contribute most effectively.
And what of Michelangelo, Beethoven, and indeed science fiction? Aren’t we allowed to have similar fun today? For a living, even?
No one’s saying that you’re not allowed to do something. Just that it’s suboptimal under their utility function, and perhaps yours.
My guess is that you overestimate how much of an altruist you are. Consider that lives can be saved using traditional methods for well under $1000. That means every time you spend $1000 on other things, your revealed preference is that having that stuff is more important to you than saving the life of another human being. If you’re upset upon hearing this fact, then you’re suffering from cognitive dissonance. If you’re a true altruist, you’ll be happy after hearing this fact, because you’ll realize that you can be scoring much better on your utility function than you are currently. (Assuming for the moment that happiness corresponds with opportunities to better satisfy your utility function, which seems to be fairy common in humans.)
Regardless of whether you’re a true altruist, it makes sense to spend a chunk of your time on entertainment and relaxation to spend the rest more effectively.
By the way, I would be interested to hear Eliezer address this topic in his video.
Not necessarily; the maximum value of a function may be attained at more than one point of its domain.
(Also, my use of the word “allowed” is clearly rhetorical/figurative. Obviously it’s not illegal to work on things other than AI, and I don’t interpret you folks as saying it should be.)
Point taken. Also, of course, given a variety of human personalities and situations, the optimal activity for a given person can vary quite a bit. I never advocate asceticism.
If it be replied that the work of, say, Newton or Darwin was essential in getting us to our current perspective wherein we have a hope of intelligently tackling this problem, might the same not hold true in yet unknown ways for string theorists? And what of Michelangelo, Beethoven, and indeed science fiction?
String theorists are at least somewhat plausible, but Michelangelo and Beethoven? Do you have any evidence that they actually helped the sciences progress? I’ve asked the same question in the past, and have not been able to adduce any evidence worth a damn. (Science fiction, at least, can try to justify itself as good propaganda.)
String theorists are at least somewhat plausible, but Michelangelo and Beethoven? Do you have any evidence that they actually helped the sciences progress?
No, and I didn’t claim they did. It was intended to be a separate question (”...string theorists? And [then, on another note], what of Michelangelo, Beethoven,....?”).
I have to remind myself that it’s not what’s the most fun to do, it’s not even what you have talent to do, it’s what you need to do that you ought to be doing.
During a panel discussion at the most recent Singularity Summit, Eliezer speculated that he might have ended up as a science fiction author, but then quickly added:
Shortly thereafter, Peter Thiel expressed a wish that all the people currently working on string theory would shift their attention to AI or aging; no disagreement was heard from anyone present.
I would therefore like to ask Eliezer whether he in fact believes that the only two legitimate occupations for an intelligent person in our current world are (1) working directly on Singularity-related issues, and (2) making as much money as possible on Wall Street in order to donate all but minimal living expenses to SIAI/Methuselah/whatever.
How much of existing art and science would he have been willing to sacrifice so that those who created it could instead have been working on Friendly AI? If it be replied that the work of, say, Newton or Darwin was essential in getting us to our current perspective wherein we have a hope of intelligently tackling this problem, might the same not hold true in yet unknown ways for string theorists? And what of Michelangelo, Beethoven, and indeed science fiction? Aren’t we allowed to have similar fun today? For a living, even?
Somewhat related, AGI is such an enormously difficult topic, requiring intimate familiarity with so many different fields, that the vast majority of people (and I count myself among them) simply aren’t able to contribute effectively to it.
I’d be interested to know if he thinks there are any singularity-related issues that are important to be worked on, but somewhat more accessible, that are more in need of contributions of man-hours rather than genius-level intellect. Is the only way a person of more modest talents can contribute through donations?
Depends on what you mean by ‘modest’. Probably 60% of Americans could contribute donations without serious lifestyle consequences and 20% of Americans could contribute over a quarter of their incomes without serious consequences. By contrast, only 10% have the reasoning abilities to identify the best large category of causes and only 1% have the reasoning abilities to identify the very best cause without a large luck element being involved. By working hard, most of that 1% could also become part of the affluent 20% of Americans who could make large donations. A similar fraction might be able to help via fund-raising efforts and by aggressively networking and sharing the contacts that they are able to build with us. A smaller but only modestly smaller fraction might be able to meaningfully contribute to SIAI’s effort via seriously committed offers of volunteer effort, but definitely not via volunteer efforts uncoupled to serious commitment. Almost no-one can do FAI, or even recognize talent at a level capable of doing FAI, but if more people were doing the easier things it wouldn’t be nearly so hard to find people who could do the core work.
SIAI keeps supporting this attitude, yet I don’t believe it, at least in the way it’s presented. A good mathematician who gets to understand the problem statement and succeeds in weeding out the standard misunderstanding can contribute as well as anyone, at this stage where we have no field. Creating a programme that would allow people to reliably get to work on the problem requires material to build upon, and there is still nothing, no matter of what quality. Systematizing the connections with existing science, trying to locate the place of FAI project in it, is something that only requires expertise in that science and understanding of FAI problem statement. At the very least, a dozen steps in, we’ll have a useful curriculum, to get folks up to speed in the right direction.
We have some experience with this, but if you want to discuss the details more with myself or some other SIAI people we will be happy to do so, and probably to have you come visit some time and get some experience with what we do. You may have ways of contributing substantially, theoretically or managerially. We’ll have to see.
Sorry for the bump, but...
Perhaps what we should do is take all our funds and create a school for AI researchers. This would be an investment for the long hall. You and I may not be super geniuses, but I sure think I could raise some super geniuses.
Also, I feel like this topic deserves more than one comment thread.
I don’t know about Eliezer, but I would be able to sacrifice quite a lot; perhaps all of art. If humanity spreads through the galaxy there will be way more than enough time for all that.
It might. But their expected contribution would be much greater if they looked at the problem to see how they could contribute most effectively.
No one’s saying that you’re not allowed to do something. Just that it’s suboptimal under their utility function, and perhaps yours.
My guess is that you overestimate how much of an altruist you are. Consider that lives can be saved using traditional methods for well under $1000. That means every time you spend $1000 on other things, your revealed preference is that having that stuff is more important to you than saving the life of another human being. If you’re upset upon hearing this fact, then you’re suffering from cognitive dissonance. If you’re a true altruist, you’ll be happy after hearing this fact, because you’ll realize that you can be scoring much better on your utility function than you are currently. (Assuming for the moment that happiness corresponds with opportunities to better satisfy your utility function, which seems to be fairy common in humans.)
Regardless of whether you’re a true altruist, it makes sense to spend a chunk of your time on entertainment and relaxation to spend the rest more effectively.
By the way, I would be interested to hear Eliezer address this topic in his video.
It’s a free country. You are allowed to do a lot, but it can only be optimal to do one thing.
Not necessarily; the maximum value of a function may be attained at more than one point of its domain.
(Also, my use of the word “allowed” is clearly rhetorical/figurative. Obviously it’s not illegal to work on things other than AI, and I don’t interpret you folks as saying it should be.)
Point taken. Also, of course, given a variety of human personalities and situations, the optimal activity for a given person can vary quite a bit. I never advocate asceticism.
String theorists are at least somewhat plausible, but Michelangelo and Beethoven? Do you have any evidence that they actually helped the sciences progress? I’ve asked the same question in the past, and have not been able to adduce any evidence worth a damn. (Science fiction, at least, can try to justify itself as good propaganda.)
No, and I didn’t claim they did. It was intended to be a separate question (”...string theorists? And [then, on another note], what of Michelangelo, Beethoven,....?”).
Aren’t these the same thing?