Do we think that effort spent trying to inhibit the development of AGI/bio/nano would be wasted because they are inevitable or at least so difficult to derail that “friendly” AI is our best shot? Where then has a detailed argument been made for this? Can someone point me to it?
Lastly, national measures that prohibit publication will not work in an international community, especially in the Internet age. If either Science or Nature had refused to publish the H5N1 papers, they would have been published somewhere else. Even if some countries stop funding—or ban—this sort of research, it will still happen in another country.
The U.S. cryptography community saw this in the 1970s and early 1980s. At that time, the National Security Agency (NSA) controlled cryptography research, which included denying funding for research, classifying results after the fact, and using export-control laws to limit what ended up in products. This was the pre-Internet world, and it worked for a while. In the 1980s they gave up on classifying research, because an international community arose (6). The limited ability for U.S. researchers to get funding for block-cipher cryptanalysis merely moved that research to Europe and Asia. The NSA continued to limit the spread of cryptography via export-control laws; the U.S.-centric nature of the computer industry meant that this was effective. In the 1990s they gave up on controlling software because the international online community became mainstream; this period was called “the Crypto Wars” (7). Export-control laws did prevent Microsoft from embedding cryptography into Windows for over a decade, but it did nothing to prevent products made in other countries from filling the market gaps.
Today, there are no restrictions on cryptography, and many U.S. government standards are the result of public international competitions.
Anyone know of anything more on deliberate relinquishment? I have seen some serious discussion by Bill McKibben in his book Enough but that’s about it.
In the linked post on the government controlling AGI development, the arguments say that it’s hard to narrowly tailor the development of specific technologies. Information technology was advancing rapidly and cryptography proved impossible to control. The government putting specific restrictions on “soft AI” amid otherwise advancing IT similarly seems far-fetched. But there are other routes. Instead we could enact policies that would deliberately slow growth in broad sectors like IT, biotechnology, and anything leading to self-replicating nanotechnology. Or maybe slow economic growth entirely and have the government direct resources at SIRCS. One can hardly argue that it is impossible to slow or even stop economic growth. We are in the middle of a worldwide economic slowdown as we type. The United States has seen little growth for at least the past ten years. I think broad relinquishment certainly cannot be dismissed without extensive discussion and to me it seems the natural way to deal with existential risk.
One can hardly argue that it is impossible to slow or even stop economic growth. We are in the middle of a worldwide economic slowdown as we type. The United States has seen little growth for at least the past ten years.
Yes, but most governments are doing their best to undo that slowdown: you’d need immense political power in order to make them encourage it.
Given some of today’s policy debates you might need less power than one might think. I think many governments, Europe being a clear case, are not doing their best to undo the slowdown. Rather, they are publicly proclaiming to be doing their best while actually setting very far from optimal policies. In a democracy you must always wear at least a cloak of serving the perceived public interest but that does not necessarily mean that you truly work in that perceived interest.
So when your Global Stasis Party wins 1% of the vote, you do not have 1% of people trying to bring about stasis and 99% trying to increase growth. Instead, already 50% of politicians may publicly proclaim to want increased growth but actually pursue growth reducing policies, and your 1% breaks the logjam and creates a 51% majority against growth. This assumes that you understand which parties are actually for and against growth, that is, you are wise enough to see through people’s facades.
I wonder how today’s policymakers would react to challengers seriously favoring no-growth economics. Would this have the effect of shifting the Overton Window? This position is so radically different from anything I’ve heard of that perhaps a small dose would have outsized effects.
You’re right about that. And there is already the degrowth movement, plus lately I’ve been hearing even some less radical politicians talking about scaling down economic growth (due to it not increasing well-being in the developed countries anymore). So perhaps something could in fact be done about that.
Here’s one such argument, which I find quite persuasive.
Also, look at how little success the environmentalists have had with trying to restrict carbon emissions, or how the US government eventually gave up its attempts to restrict cryptography:
Anyone know of anything more on deliberate relinquishment? I have seen some serious discussion by Bill McKibben in his book Enough but that’s about it.
In the linked post on the government controlling AGI development, the arguments say that it’s hard to narrowly tailor the development of specific technologies. Information technology was advancing rapidly and cryptography proved impossible to control. The government putting specific restrictions on “soft AI” amid otherwise advancing IT similarly seems far-fetched. But there are other routes. Instead we could enact policies that would deliberately slow growth in broad sectors like IT, biotechnology, and anything leading to self-replicating nanotechnology. Or maybe slow economic growth entirely and have the government direct resources at SIRCS. One can hardly argue that it is impossible to slow or even stop economic growth. We are in the middle of a worldwide economic slowdown as we type. The United States has seen little growth for at least the past ten years. I think broad relinquishment certainly cannot be dismissed without extensive discussion and to me it seems the natural way to deal with existential risk.
Yes, but most governments are doing their best to undo that slowdown: you’d need immense political power in order to make them encourage it.
Given some of today’s policy debates you might need less power than one might think. I think many governments, Europe being a clear case, are not doing their best to undo the slowdown. Rather, they are publicly proclaiming to be doing their best while actually setting very far from optimal policies. In a democracy you must always wear at least a cloak of serving the perceived public interest but that does not necessarily mean that you truly work in that perceived interest.
So when your Global Stasis Party wins 1% of the vote, you do not have 1% of people trying to bring about stasis and 99% trying to increase growth. Instead, already 50% of politicians may publicly proclaim to want increased growth but actually pursue growth reducing policies, and your 1% breaks the logjam and creates a 51% majority against growth. This assumes that you understand which parties are actually for and against growth, that is, you are wise enough to see through people’s facades.
I wonder how today’s policymakers would react to challengers seriously favoring no-growth economics. Would this have the effect of shifting the Overton Window? This position is so radically different from anything I’ve heard of that perhaps a small dose would have outsized effects.
You’re right about that. And there is already the degrowth movement, plus lately I’ve been hearing even some less radical politicians talking about scaling down economic growth (due to it not increasing well-being in the developed countries anymore). So perhaps something could in fact be done about that.
And of course there is Bill Joy’s essay. I forgot about that. But seems like small potatoes.