Personally, I’m a lot more worried about nasty humans taking early-stage AGIs and using them for massive destruction, than about speculative risks associated with little-understood events like hard takeoffs.
A psychotic egoist like Stalin or an non-humanist like Hitler is indeed terrifying but I’m not convinced that giving a great increase in power and intelligence to someone like a Mao or a Lord Lytton, who caused millions of deaths by doing something they thought would improve people’s lives, would lead to a worse outcome than we got in reality. Granted, for something like the cultural revolution these mistakes might be subtle enough to get into an AI, but it’s hard to imagine them getting a computer to say “yes, the peasants can live on 500 calories a day, increase the tariff” unless they were deliberately trying to be wrong, which they weren’t.
Moral considerations aside, the real causes of the mass famines under Mao and Stalin can be understood from a perspective of pure power and political strategy. From the point of view of a strong centralizing regime trying to solidify its power, the peasants are always the biggest problem.
Urban populations are easy to control for any regime that firmly holds the reins of the internal security forces: just take over the channels of food distribution, ration the food, and make obedience a precondition for eating. Along with a credible threat to meet any attempts at rioting with bayonets and live bullets, this is enough to ensure obedience of the urban dwellers. In contrast, peasants always have the option of withdrawing into an autarkic self-sufficient lifestyle, and they will do it if pressed hard by taxation and requisitioning. In addition, they are widely dispersed, making it hard for the security forces to coerce them effectively. And in an indecisive long standoff, the peasants will eventually win, since without buying or confiscating their food surplus, everyone else starves to death.
Both the Russian and the Chinese communists understood that nothing but the most extreme measures would suffice to break the resistance of the peasantry. When the peasants responded to confiscatory measures by withdrawing to subsistence agriculture, they knew they’d have to send the armed forces to confiscate their subsistence food and let them starve, and eventually force the survivors into state-run enterprises where they’d have no more capacity for autarky than the urban populations. (In the Russian case, this job was done very incompletely during the Revolution, which was followed by a decade of economic liberalization, after which the regime finally felt strong enough to finish the job.)
(Also, it’s simply untenable to claim that this was due to some special brutality of Stalin and Mao. Here is a 1918 speech by Trotsky that discusses the issue in quite frank terms. Now of course, he’s trying to present it as a struggle against the minority of rich “kulaks,” not the poorer peasants, but as Zinoviev admitted a few years later, “We [the Bolsheviks] are fond of describing any peasant who has enough to eat as a kulak.”)
Not directly relivant, but Mao seems to have known that his policies were causing mass starvation. Of course, with a tame AGI he could have achieved communism with a very different kind of Great Leap.
Oh yes, I see I’ve inadvertently fallen into that sordid old bromide about communism being a good idea that unfortunately failed to work, still- committing to an action that one knows will cause millions of deaths is quite different to learning about it as one is doing it. Certainly in the case of the British in India, their Malthusian rhetoric and victim-blaming was so at odds with their earlier talk of modernizing the continent that it sounds like a post-hoc rationalization of the genocide. I realize now though that I don’t know enough about the PRC to judge whether a similar phenomenon was at work there.
Well… That is hard to communicate now, as I will need to extricate the problems from the specifics that were communicated to me (in confidence)...
Let’s see...
1) That there is a dangerous political movement in the USA that seems to be preferring revealed knowledge to scientific understanding and investigation.
2) Poverty
3) Education
4) Hunger (I myself suffer from this problem—I am disabled, on a fixed income, and while I am in school again and doing quite well I still have to make choices sometimes between necessities… And, I am quite well off compared to some I know)
5) The lack of a political dialog and the preference for ideological certitude over pragmatic solutions and realistic uncertainty.
6) The fact that there exist a great amount of crime among the white collar crowd that goes both unchecked, and unpunished when it is exposed (Maddoff was a fluke in that regard).
7) The various “Wars” that we declare on things (Drugs, Terrorism, etc.) “War” is a poor paradigm to use, and it leads to more damage than it corrects (especially in the two instances I cited)
8) The real “Wars” that are happening right now (and not just those waged by the USA and allies)
Some of these were explicitly discussed.
Some will eventually be resolved, but that doesn’t mean that they should be ignored until that time. That would be akin to seeing a man dying of starvation, while one has the capacity to feed him, yet thinking “Oh, he’ll get some food eventually.”
And, some may just be perennial problems with which we will have to deal with for some time to come.
I misread you as saying that important ethical problems about FAI were being ignored, but yes, the idea that FAI is the most important thing in the world leaves quite a bit out, and not just great evils. There’s a lot of maintenance to be done along the way to FAI.
Madoff’s fraud was initiated by a single human being, or possibly Madoff and his wife. It was comprehensible without adding a lot of what used to be specialist knowledge. It’s a much more manageable sort of crime than major institutions becoming destructively corrupt.
What are the more important ethical problems?
Ben says:
http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html
That seems fairly reasonable. The SIAI are concerned that the engineers might screw up so badly that a bug takes over the world—and destroys everyone.
Another problem is if a Stalin or a Mao get hold of machine intelligence. The latter seems like a more obvious problem.
A psychotic egoist like Stalin or an non-humanist like Hitler is indeed terrifying but I’m not convinced that giving a great increase in power and intelligence to someone like a Mao or a Lord Lytton, who caused millions of deaths by doing something they thought would improve people’s lives, would lead to a worse outcome than we got in reality. Granted, for something like the cultural revolution these mistakes might be subtle enough to get into an AI, but it’s hard to imagine them getting a computer to say “yes, the peasants can live on 500 calories a day, increase the tariff” unless they were deliberately trying to be wrong, which they weren’t.
Moral considerations aside, the real causes of the mass famines under Mao and Stalin can be understood from a perspective of pure power and political strategy. From the point of view of a strong centralizing regime trying to solidify its power, the peasants are always the biggest problem.
Urban populations are easy to control for any regime that firmly holds the reins of the internal security forces: just take over the channels of food distribution, ration the food, and make obedience a precondition for eating. Along with a credible threat to meet any attempts at rioting with bayonets and live bullets, this is enough to ensure obedience of the urban dwellers. In contrast, peasants always have the option of withdrawing into an autarkic self-sufficient lifestyle, and they will do it if pressed hard by taxation and requisitioning. In addition, they are widely dispersed, making it hard for the security forces to coerce them effectively. And in an indecisive long standoff, the peasants will eventually win, since without buying or confiscating their food surplus, everyone else starves to death.
Both the Russian and the Chinese communists understood that nothing but the most extreme measures would suffice to break the resistance of the peasantry. When the peasants responded to confiscatory measures by withdrawing to subsistence agriculture, they knew they’d have to send the armed forces to confiscate their subsistence food and let them starve, and eventually force the survivors into state-run enterprises where they’d have no more capacity for autarky than the urban populations. (In the Russian case, this job was done very incompletely during the Revolution, which was followed by a decade of economic liberalization, after which the regime finally felt strong enough to finish the job.)
(Also, it’s simply untenable to claim that this was due to some special brutality of Stalin and Mao. Here is a 1918 speech by Trotsky that discusses the issue in quite frank terms. Now of course, he’s trying to present it as a struggle against the minority of rich “kulaks,” not the poorer peasants, but as Zinoviev admitted a few years later, “We [the Bolsheviks] are fond of describing any peasant who has enough to eat as a kulak.”)
Not directly relivant, but Mao seems to have known that his policies were causing mass starvation. Of course, with a tame AGI he could have achieved communism with a very different kind of Great Leap.
Oh yes, I see I’ve inadvertently fallen into that sordid old bromide about communism being a good idea that unfortunately failed to work, still- committing to an action that one knows will cause millions of deaths is quite different to learning about it as one is doing it. Certainly in the case of the British in India, their Malthusian rhetoric and victim-blaming was so at odds with their earlier talk of modernizing the continent that it sounds like a post-hoc rationalization of the genocide. I realize now though that I don’t know enough about the PRC to judge whether a similar phenomenon was at work there.
Well… That is hard to communicate now, as I will need to extricate the problems from the specifics that were communicated to me (in confidence)...
Let’s see...
1) That there is a dangerous political movement in the USA that seems to be preferring revealed knowledge to scientific understanding and investigation. 2) Poverty 3) Education 4) Hunger (I myself suffer from this problem—I am disabled, on a fixed income, and while I am in school again and doing quite well I still have to make choices sometimes between necessities… And, I am quite well off compared to some I know) 5) The lack of a political dialog and the preference for ideological certitude over pragmatic solutions and realistic uncertainty. 6) The fact that there exist a great amount of crime among the white collar crowd that goes both unchecked, and unpunished when it is exposed (Maddoff was a fluke in that regard). 7) The various “Wars” that we declare on things (Drugs, Terrorism, etc.) “War” is a poor paradigm to use, and it leads to more damage than it corrects (especially in the two instances I cited) 8) The real “Wars” that are happening right now (and not just those waged by the USA and allies)
Some of these were explicitly discussed.
Some will eventually be resolved, but that doesn’t mean that they should be ignored until that time. That would be akin to seeing a man dying of starvation, while one has the capacity to feed him, yet thinking “Oh, he’ll get some food eventually.”
And, some may just be perennial problems with which we will have to deal with for some time to come.
I misread you as saying that important ethical problems about FAI were being ignored, but yes, the idea that FAI is the most important thing in the world leaves quite a bit out, and not just great evils. There’s a lot of maintenance to be done along the way to FAI.
Madoff’s fraud was initiated by a single human being, or possibly Madoff and his wife. It was comprehensible without adding a lot of what used to be specialist knowledge. It’s a much more manageable sort of crime than major institutions becoming destructively corrupt.
I think major infrastructure rebuilding is probably closer to the case than “maintenance”