I think that I predict the opposite (conditional on what exactly is being predicted).
What exactly would count as a GPT-3 moment for fusion? How about an experiment demonstrating reactor-like conditions? This is roughly equivalent to what I referred to as ‘getting fusion’ in my book review.
My prediction is that, after Commonwealth Fusion Systems gets Q > 5 on SPARC, they will continue to supply or plan to supply HTS tape to at least 3 other fusion startups.
I agree that this is plausibly a real important difference, but I do not think that it is obvious.
The most recent augmentative technological change was the industrial revolution. It has reshaped virtually every every activity. It allowed for the majority of the population to not work in agriculture for the first time since the agricultural revolution.
The industrial revolution centered on energy. Having much cheaper, much more abundant energy allowed humans to use that energy for all sorts of things.
If fusion ends up being similar in cost to existing electricity production, it will be a substitutional technology. This is the thing that we are working on now (well, also making it work at all). People who work in fusion focus on this because it is the reasonable near/medium term projection. If fusion ends up being substantially cheaper, it will be an augmentative technology. It is not at all clear that this will happen, because we can’t know how the costs will change between the first and thousandth fusion power plant.
Notably, we don’t know if foom is going to be a thing either.
The narrative around the technology is at least as important as what has happened in the technology itself. The fusion community could frequently talk about how incredible the industrial revolution was, and how it powered Britain to global dominance for two centuries. A new source of energy might do the same thing ! But this is more hype than we feel we ought to offer, and the community’s goal is not to create a dominant superpower.
Even if foom is going to happen, things would look very different if the leaders credibly committed to helping others foom if they are first. I don’t know if this would be better or worse from a existential risk perspective, but it would change the nature of the race a lot.
Before. The 2022 survey responses were collected from June-August. ChatGPT came out at the end of November.
A few more thoughts on Ord’s paper:
Despite the similarities, I think that there is some difference between Ord’s notion of hyperbolation and what I’m describing here. In most of his examples, the extra dimension is given. In the examples I’m thinking of, what the extra dimension ought to be is not known beforehand.
There is a situation in which hyperbolation is rigorously defined: analytic continuation. This takes smooth functions defined on the real axis and extends them into the complex plane. The first two examples Ord gives in his paper are examples of analytic continuation, so his intuition that these are the simplest hyperbolations is correct.
More generally, solving a PDE from boundary conditions could be considered to be a kind of hyperbolation, although the result can be quite different depending on which PDE you’re solving. This feels like substantially less of a new ability than e.g. inventing language.
Climate change is not the only field to have defined words for specific probability ranges. The intelligence community has looked into this as well. They’re called words of estimative probability.
A lot of the emphasis is on climate change, which has become partisan than other environmental issues. But other environmental issues have become partisan as well. Here’s some data from a paper from 2013 by D.L. Guber, “A cooling climate for change? Party polarization and the politics of global warming.”
The poll you linked indicates that Republicans in the Mountain West are more concerned with the environmental than Republicans in the rest of the country. There is a 27 p.p. partisan gap on the energy vs environment question (p. 17) - much less than the 55 p.p. partisan gap for the country as a whole. The partisan gap for whether “a public official’s position on conservation issues will be an important factor in determining their support” is 22 p.p. (p. 13), with clear majorities in both parties. Climate change is somewhat less of a concern than other issues, which I would guess is because it is more partisan, but not by that much (p. 21).
In the Mountain West, it looks like there is some partisanship for environmental issues, but only the amount we would expect for a generic issue in the US, or for environmentalism in another country. This is consistent with environmentalism being extremely partisan on average over the entire country. The Mountain West is less than a tenth of the country’s population and has an unusually impressive natural environment.
Environmentalism started to became partisan around 1990. Nixon & Reagan both spoke of the environment in these terms.
I think that this is a coincidence. Japan has low partisanship for environmentalism and has less nuclear power than most developed countries (along with low overall partisanship). The association would be between three things: (1) low partisanship for environmentalism, (2) high overall partisanship, and (3) lots of nuclear power plants. There aren’t enough countries to do this kind of correlation.
From the introduction to the last post in this sequence:
Environmentalists were not the only people making significant decisions here. Fossil fuel companies and conservative think tanks also had agency in the debate – and their choices were more blameworthy than the choices of environmentalists. Politicians choose who they do and do not want to ally with. My focus is on the environmental movement itself, because that is similar to what other activist groups are able to control.
The motivation for this report was to learn what the AI safety movement should do to keep from becoming partisan. ‘Meta doesn’t lobby the government’ isn’t an action the AI safety movement can take.
Thank you !
The links to the report are now fixed.
The 4 blog posts cover most of the same ground as the report. The report goes into more detail, especially in sections 5 & 6.
I think this is true of an environmentalist movement that wants there to be a healthy environment for humans; I’m not sure this is true of an environmentalist movement whose main goal is to dismantle capitalism.
I talk about mission creep in the report, section 6.6.
Part of ‘making alliances with Democrats’ involved environmental organizations adopting leftist positions on other issues.
Different environmental organizations have seen more or less mission creep. The examples I give in the report are the women’s issues for the World Wildlife Fund:
In many parts of the developing world, women of all ages play a critical role in managing natural resources, which they rely on for food, water, medicine, and fuel wood for their families. Yet they often are excluded from participating in decisions about resource use.[1]
and the Sierra Club:
The Sierra Club is a pro-choice organization that endorses comprehensive, voluntary reproductive health care for all. Sexual and reproductive health and rights are inalienable human rights that should be guaranteed for all people with no ulterior motive. A human rights-based approach to climate justice centers a person’s bodily autonomy and individual choice.[2]
It’s hard to date exactly when many of this positions were adopted by major environmental organizations, but my impression is sometime in the 1990s or 2000s. That’s when the Sierra Club started making presidential endorsements and when several major environmental organizations started promoting environmental justice.
This mission creep is part of the story. Allowing mission creep into controversial positions that are not directly related to the movement’s core goals makes it harder to build bipartisan coalitions.
“Women and girls,” World Wildlife Fund, Accessed: March 28, 2024. https://www.worldwildlife.
org/initiatives/women-and-girls.
The Sierra Club and population issues,” Sierra Club, Accessed: March 28, 2024. https://www.sier
raclub.org/sierra-club-and-population-issues.
The title for this page is not explicitly about gender, but to get to this page from the “People & Justice” page, you click on “Read more” in the section: “And our future depends on gender equity.”
This is trying to make environmentalism become partisan, but in the other direction.
Environmentalists could just not have positions on most controversial issues, and instead focus more narrowly on the environment.
There is also the far right in France, which is not the same as the right wing in America, but is also not Joe Biden. From what I can tell, the far right in France supports environmentalism.[1]
Macron & Le Pen seem to have fairly similar climate policies. Both want France’s electricity to be mostly nuclear – Le Pen more so. Both are not going to raise fuel taxes – Macron reluctantly. Le Pen talks more about hydrogen and reshoring manufacturing from countries which emit more (and claims that immigration is bad for France’s environmental goals). Macron supports renewables in addition to nuclear power. The various leftists seem to be interested in phasing out nuclear & replacing it with renewables. None of the parties dismiss climate change as an issue and all are committed to following international climate agreements.
Kate Aronoff. Marine Le Pen’s Climate Policy Leans Ecofascist. The New Republic. (2022) https://newrepublic.com/article/166097/marine-le-pens-climate-policy-whiff-ecofascism.
I think it was possible for the environmental movement to form alliances with politicians in both parties, and for environmentalism to have remained bipartisan.
Comparing different countries and comparing the same country at different times is not the same thing as a counterfactual, but it can be very helpful for understanding counterfactuals. In this case, the counterfactual US is taken to be similar to the US in the 1980s or to the UK, France, or South Korea today.
I think you should ask the author of the song if it’s referring to someone using powerful AI to do something transformative to the sun.
This is extremely obvious to me. The song is opposed to how the sun currently is, calling it “wasteful” and “distasteful”—the second word is a quote from a fictional character, but the first is not. It later talks about when “the sun’s a battery,” so something about the sun is going to change. I really don’t know what “some big old computer” could be referring to if not powerful AI.
Thank you for responding! I am being very critical, both in foundational and nitpicky ways. This can be annoying and make people want to circle the wagons. But you and the other organizers are engaging constructively, which is great.
The distinction between Solstice representing a single coherent worldview vs. a series of reflections also came up in comments on a draft. In particular, the Spinozism of Songs Stay Sung feels a lot weirder if it is taken as the response to the darkness, which I initially did, rather than one response to the darkness.
Nevertheless, including something in Solstice solidly establishes it as a normal / acceptable belief for rationalists: within the local Overton Window. You might not be explicitly telling people that they ought to believe something, but you are telling that it is acceptable for high status people in their community to believe it. I am concerned that some of these beliefs are even treated as acceptable.
Take Great Transhumanist Future. It has “a coder” dismantling the sun “in another twenty years with some big old computer.” This is a call to accelerate AI development, and use it for extremely transformative actions. Some of the organizers believe that this is the sort of thing that will literally kill everyone. Even if it goes well, it would make life as it currently exists on the surface of the Earth impossible. Life could still continue in other ways, but some of us might want to still live here in 20 years.[1] I don’t think that reckless AI accelerationism should be treated as locally acceptable.
The line in Brighter Than Today points in the same way. It’s not only anti-religious. It is also disparaging towards people who warn about the destructive potential of a new technology. Is that an attitude we want to establish as normal?
If the main problem with changing the songs is in making them scan and rhyme, then I can probably just pay that cost. This isn’t a thing I’m particularly skilled at, but there are people who are who are adjacent to the community. I’m happy to ask them to rewrite a few lines, if the new versions will plausibly be used.
If the main problem with changing the songs is that many people in this community want to sing about AI accelerationism and want the songs to be anti-religious, then I stand by my criticisms.
Is this action unilateral? Unclear. There might be a global consensus building phase, or a period of reflection. They aren’t mentioned in the song. These processes can’t take very long given the timelines.
The London subway was private and returned enough profit to slowly expand while it was coal powered. Once it electrified, it became more profitable and expanded quickly.
The Baltimore tunnel was and is part of an intercity line that is mostly above ground. It was technologically similar to London, but operationally very different.
I chose the start date of 1866 because that is the first time the New York Senate appointed a committee to study rapid transit in New York, which concluded that New York would be best served by an underground railroad. It’s also the start date that Katz uses.
The technology was available. London opened its first subway line in 1863. There is a 1.4 mi railroad tunnel from 1873 in Baltimore that is still in active use today. These early tunnels used steam engines. This did cause ventilation challenges, but they were resolvable. The other reasonable pre-electricity option would be to have stationary steam engines at a few places, open to the air, that pulled cables that pulled the trains. There were also some suggestions of dubious power mechanisms, like the one you described here. None of the options were as good as electric trains, but some of them could have been made to work.
This is not a global technological overhang, because there continued to be urban railroad innovation in other cities. It would only be overhang for New York City. This is a more restrictive definition of overhang than I used in my previous post, but it might still be interesting to see what happened with local overhang.
The original version of the song reads to me as being deist or pantheist. You could replace ‘God’ with ‘Nature’ and the meaning would be almost the same. My view of Divinely Guided Evolution has a personal God fiddling with random mutations and randomly determined external factors to create the things He wants.
It is definitely anti-Young-Earth-Creationism, but it is also dismissive of the Bible. Even if you don’t think that Genesis 1 should be treated as a chronology, I think that you should take the Bible seriously. Its commentary on what it means to be human is important.
I would also make the same prediction for Q > 10. Or when CFS first sells electricity to the grid. These will be farther into the future, but I do not think that this culture will have changed by then.