I’m not convinced that slowing economic growth would result in FAI developing faster than UFAI and I think your main point of leverage for getting an advantage lies elsewhere (explained). The key is obviously the proportion between the two, not just slowing down the one or speeding up the other, so I suggest a brainstorm to consider all of the possible ways that slow economic growth could also slow FAI. For one thought: do non-profit organizations do disproportionately poorly during recessions?
The major point of leverage, I think, is people, not the economy. How that would work:
Selfish people have a goal that divides them. If your goal is to be an AI trillionaire, you have to start your own AI company. If your goal is to save the world from AI, your goal benefits if you co-operate with like-minded people as much as possible. The aspiring trillionaire’s goals will not be furthered by co-operating with competitors, and the world-saver’s goals will not be furthered by competing with potential allies. This means that even if UFAI workers outnumber FAI workers 10 to 1, it’s possible for the FAI effort to unite that 10% and conquer the 90% with sheer numbers, assuming that the 90% is divided into fragments smaller than 10% each.
An organization made up of altruistic people might be stronger and more efficient than an organization made up of selfish people. If FAI workers are more honest or less greedy, this could yield practical benefits like a reduction in efficiency-draining office politics games, a lower risk of being stolen from or betrayed by those within the organization, and being able to put more money toward research because paychecks may not need to be as large. Also, people who are passionate about their goal derive meaning from their work. They might work harder or have more moments of creative inspiration than people who are working simply to get more money.
The FAI movement is likely to attract people who are forward-thinking and/or more sensible, realistic or rational. These people may be more likely to succeed at what they do than those attracted to UFAI projects.
People look down on those who hurt others for profit. Right now, the average person probably does not know about the UFAI risk and how important it is that people should not work on such projects. UFAI is a problem that would affect everyone, so it’s likely that the average person will eventually take interest in it and shun those who work on UFAI. If they’re accurately informed about which projects should be considered UFAI, I think it would seriously deter AI workers from choosing UFAI jobs.
It has been said by people knowledgeable about geniuses that they typically don’t prioritize money as highly as others. Unfortunately, there isn’t enough solid research on the gifted population (let alone geniuses) but if it’s true that money is not the most important thing to a genius, you may find that a disproportionate number of geniuses would prioritize working on FAI over taking the biggest possible paycheck.
Unlike the economy, all of these are things that MIRI can take action on. If the FAI movement can take advantage of any of these to get more workers, more good minds, or more good people than the UFAI projects, I want to see it happen.
Uhm, it is not that simple. Perhaps selfish people cooperate less, but among altruistic people often the price for cooperation is worshiping the same applause lights. Selfish people optimize for money or power, altruistic people often optimize for status in altruistic community. Selfish people may be more agenty, simply because they know that if they don’t work for their selfish benefits, no one else will. Altruistic people often talk about what others should do, what the government should do, etc. Altruistic people collect around different causes, they compete for donor money and public attention, even their goals may sometimes be opposed; e.g. “protecting nature” vs “removing the suffering inherent in nature”; “spreading rationality” vs “spreading religious tolerance”; “making people equal and happy” vs “protecting the cultural heritage”. People don’t like those who hurt others, but they also admire high-status people and despise low-status people. Geniuses are often crazy.
I’m not saying it is exactly the other way round as you said. Just: it’s complicated. I scanned through your comment and listed all the counterarguments that immediately came to my mind. If good intentions and intelligence translated to success so directly, then communists wouldn’t have killed millions of people, Mensa would rule the world now, and we all would be living in the post-singularity paradise already.
I’m not convinced that slowing economic growth would result in FAI developing faster than UFAI and I think your main point of leverage for getting an advantage lies elsewhere (explained). The key is obviously the proportion between the two, not just slowing down the one or speeding up the other, so I suggest a brainstorm to consider all of the possible ways that slow economic growth could also slow FAI. For one thought: do non-profit organizations do disproportionately poorly during recessions?
The major point of leverage, I think, is people, not the economy. How that would work:
Selfish people have a goal that divides them. If your goal is to be an AI trillionaire, you have to start your own AI company. If your goal is to save the world from AI, your goal benefits if you co-operate with like-minded people as much as possible. The aspiring trillionaire’s goals will not be furthered by co-operating with competitors, and the world-saver’s goals will not be furthered by competing with potential allies. This means that even if UFAI workers outnumber FAI workers 10 to 1, it’s possible for the FAI effort to unite that 10% and conquer the 90% with sheer numbers, assuming that the 90% is divided into fragments smaller than 10% each.
An organization made up of altruistic people might be stronger and more efficient than an organization made up of selfish people. If FAI workers are more honest or less greedy, this could yield practical benefits like a reduction in efficiency-draining office politics games, a lower risk of being stolen from or betrayed by those within the organization, and being able to put more money toward research because paychecks may not need to be as large. Also, people who are passionate about their goal derive meaning from their work. They might work harder or have more moments of creative inspiration than people who are working simply to get more money.
The FAI movement is likely to attract people who are forward-thinking and/or more sensible, realistic or rational. These people may be more likely to succeed at what they do than those attracted to UFAI projects.
People look down on those who hurt others for profit. Right now, the average person probably does not know about the UFAI risk and how important it is that people should not work on such projects. UFAI is a problem that would affect everyone, so it’s likely that the average person will eventually take interest in it and shun those who work on UFAI. If they’re accurately informed about which projects should be considered UFAI, I think it would seriously deter AI workers from choosing UFAI jobs.
It has been said by people knowledgeable about geniuses that they typically don’t prioritize money as highly as others. Unfortunately, there isn’t enough solid research on the gifted population (let alone geniuses) but if it’s true that money is not the most important thing to a genius, you may find that a disproportionate number of geniuses would prioritize working on FAI over taking the biggest possible paycheck.
Unlike the economy, all of these are things that MIRI can take action on. If the FAI movement can take advantage of any of these to get more workers, more good minds, or more good people than the UFAI projects, I want to see it happen.
Uhm, it is not that simple. Perhaps selfish people cooperate less, but among altruistic people often the price for cooperation is worshiping the same applause lights. Selfish people optimize for money or power, altruistic people often optimize for status in altruistic community. Selfish people may be more agenty, simply because they know that if they don’t work for their selfish benefits, no one else will. Altruistic people often talk about what others should do, what the government should do, etc. Altruistic people collect around different causes, they compete for donor money and public attention, even their goals may sometimes be opposed; e.g. “protecting nature” vs “removing the suffering inherent in nature”; “spreading rationality” vs “spreading religious tolerance”; “making people equal and happy” vs “protecting the cultural heritage”. People don’t like those who hurt others, but they also admire high-status people and despise low-status people. Geniuses are often crazy.
I’m not saying it is exactly the other way round as you said. Just: it’s complicated. I scanned through your comment and listed all the counterarguments that immediately came to my mind. If good intentions and intelligence translated to success so directly, then communists wouldn’t have killed millions of people, Mensa would rule the world now, and we all would be living in the post-singularity paradise already.