Academic futurism has low status. This causes people interested in futurism to ignore those academics and instead listen to people who talk about futurism after gaining high status via focusing on other topics. As a result, the people who are listened to on the future tend to be amateurs, not specialists. And this is why “we” know a lot less about the future than we could.
Larry Carter, a UCSD emeritus professor of computer science, didn’t mince words. The first time he heard about Wells’s theories, he thought, “Oh my God, is this guy a crackpot?”
But persuaded by Well’s credentials, which include a PhD from Caltech in math and theoretical physics, a career that led him L-3 Photonics and the Caltech/Jet Propulsion Laboratory, and an invention under his belt, Carter gave the ideas a chance. And was intrigued.
For a taste of the book, here is Wells’ description of one specific risk:
When advanced robots arrive… the serious threat [will be] human hackers. They may deliberately breed a hostile strain of androids, which then infects normal ones with its virus. To do this, the hackers must obtain a genetic algorithm and pervert it, probably early in the robotic age before safeguards become sophisticated… Excluding hackers, it seems unlikely that androids will turn against us as they do in some movies… computer code for hostility is too complex… In the very long term, androids will become conscious for the same reasons humans did, whatever those reasons may be… In summary, the androids have powerful instincts to nurture humans, but these instincts will be unencumbered by concerns for human rights. Androids will feel free to impose a harsh discipline that saves us from ourselves while violating many of our so-called human rights.
Now, despite Larry Carter’s being “persuaded by Wells’ credentials” — which might have been exaggerated or made-up by the journalist, I don’t know — I suspect very few people have taken Wells seriously, for good reason. He’s clearly just making stuff up, with almost no study of the issue whatsoever. (On this topic, the only people he cites are Joy, Kurzweil, and Posner, despite the book being published in 2009.)
But reading that passage did drive home again what it must be like for most people to read FHI or MIRI on AI risk, or Robin Hanson on ems. They probably can’t tell the difference between someone who is making stuff up and an argument that has gone through a gauntlet of 15 years of heated debate and both theoretical and empirical research.
Yes by judging someone on their credentials in other fields, you can’t tell if they are just making stuff up on this subject vs. studied it for 15 years.
I took a quick skim through the book. Your focused criticism of Wells’s book is somewhat unfair. The majority of the book (ch. 1–4) is about a survival analysis of doomsday risks. The scenario you quoted is in the last chapter (ch. 5), which looks like an afterthought to the main intent of the book (i.e., providing the survival analysis), and is prepended by the following disclaimer:
This set serves as a foil to the balanced discussions by Rees, Leslie, Powell, and others. The choice of eight examples is purely arbitrary. Their purpose is not orderly coverage but merely examples that indicate a range of possibilities. The actual number of such complex unorthodox scenarios is virtually infinite, hence the high risk.
I think it is fair to criticize the crackpot scenario that he gave as an example, but your criticism seems to suggest that his entire book is of the same crackpot nature, which it is not. It is unfortunate that PR articles and public attention focuses on the insubstantial parts of the book, but I am sure you know what that is like as the same occurs frequently to MIRI/SIAI’s ideas.
Orthogonal notes on the book’s content: Wells seems unaware of Bostrom’s work on observation selection effects, and it appears that he implicitly uses SSA. (I have not carefully read enough of his book to form an opinion on his analysis, nor do I currently know enough about survival analysis to know whether what he does is standard.)
Yes, I’m an academic and I get a similar reaction from telling people I study the Singularity as when I say I’ve signed up for cryonics. Thankfully, I have tenure.
Do you actually say you “study the singularity” or give a more in depth explanation? I ask because the word study is usually used only in reference to things that do or have exisited, rather than to speculative future events.
It might be a worthwhile endeavor to modify our wiki such that it serves not only as a mostly local reference on current terms and jargon, but also as an independent guide to the various arguments for and against various concepts, where applicable. It could create a lot of credibility and exposure to establish a sort of neutral reference guide / an argument map / the history and iterations an idea has gone through, in a neutral voice. Ideally, neutrality regarding PoV works in favor of those with the balance of arguments in their favor.
This need not be entirely new material, but instead simply a few mandatory / recommended headers in each wiki entry, pertaining to history, counterarguments etc. Could be worth it lifting the wiki from relative obscurity, with a new landing page, and marketed potentially as a reference guide for journalists researching current topics. Kruel’s LW interview with Shane Legg got linked to in a NYTimes blog, why not a suitable LW wiki article, too?
Academic futurism has low status. This causes people interested in futurism to ignore those academics and instead listen to people who talk about futurism after gaining high status via focusing on other topics. As a result, the people who are listened to on the future tend to be amateurs, not specialists. And this is why “we” know a lot less about the future than we could.
I don’t think that’s the case. Most people who are listened to on the future don’t tend to speak to an audience primarily consisting of futurists.
There are think tanks who employee people to think about the future and those think tanks tend generally to be quite good at influencing the public debate.
I also don’t think that academic has any special claim to be specialists about the future.
When I think about specialists on futurism names like Stewart Brand or Bruce Sterling.
I don’t think that’s the case. Most people who are listened to on the future don’t tend to speak to an audience primarily
consisting of futurists.
This is a very important and general point. While it is important to communicate ideas to a general audience, generally excessive communication to general audiences at the expense of communication to peers should be “bad news” when it comes to evaluating experts. Folks like Witten mostly just get work done, they don’t write popular science books.
I mean Edward Witten, one of the most prominent physicists alive. The fact that his name does not ring a bell is precisely my point. The names that do ring a bell are the names of folks who are “good at the media,” not necessarily folks who are the best in their field.
Witten is one of the greatest physicists alive, if not the greatest. He is the one who unified the various string theories into M-theory. He is also the only physicist to receive a Fields Medal.
Robin Hanson on Facebook:
Consider the case of Willard Wells and his Springer-published book Apocalypse When?: Calculating How Long the Human Race Will Survive (2009). From a UCSD news story about a talk Wells gave about the book:
For a taste of the book, here is Wells’ description of one specific risk:
Now, despite Larry Carter’s being “persuaded by Wells’ credentials” — which might have been exaggerated or made-up by the journalist, I don’t know — I suspect very few people have taken Wells seriously, for good reason. He’s clearly just making stuff up, with almost no study of the issue whatsoever. (On this topic, the only people he cites are Joy, Kurzweil, and Posner, despite the book being published in 2009.)
But reading that passage did drive home again what it must be like for most people to read FHI or MIRI on AI risk, or Robin Hanson on ems. They probably can’t tell the difference between someone who is making stuff up and an argument that has gone through a gauntlet of 15 years of heated debate and both theoretical and empirical research.
Yes by judging someone on their credentials in other fields, you can’t tell if they are just making stuff up on this subject vs. studied it for 15 years.
Wells’s book: Apocalypse when.
I took a quick skim through the book. Your focused criticism of Wells’s book is somewhat unfair. The majority of the book (ch. 1–4) is about a survival analysis of doomsday risks. The scenario you quoted is in the last chapter (ch. 5), which looks like an afterthought to the main intent of the book (i.e., providing the survival analysis), and is prepended by the following disclaimer:
I think it is fair to criticize the crackpot scenario that he gave as an example, but your criticism seems to suggest that his entire book is of the same crackpot nature, which it is not. It is unfortunate that PR articles and public attention focuses on the insubstantial parts of the book, but I am sure you know what that is like as the same occurs frequently to MIRI/SIAI’s ideas.
Orthogonal notes on the book’s content: Wells seems unaware of Bostrom’s work on observation selection effects, and it appears that he implicitly uses SSA. (I have not carefully read enough of his book to form an opinion on his analysis, nor do I currently know enough about survival analysis to know whether what he does is standard.)
Ah, you’re right that I should have quoted the “This set serves as a foil” paragraph as well.
I found chs. 1-4 pretty unconvincing, too, though I’m still glad that analysis exists.
Yes, I’m an academic and I get a similar reaction from telling people I study the Singularity as when I say I’ve signed up for cryonics. Thankfully, I have tenure.
What happens when you say, “I study the economic implications of advanced artificial inteligence,” to people?
I don’t phrase it this way.
Do you actually say you “study the singularity” or give a more in depth explanation? I ask because the word study is usually used only in reference to things that do or have exisited, rather than to speculative future events.
I go into more depth, especially when I (unsuccessfully) came up for promotion for full professor.
It might be a worthwhile endeavor to modify our wiki such that it serves not only as a mostly local reference on current terms and jargon, but also as an independent guide to the various arguments for and against various concepts, where applicable. It could create a lot of credibility and exposure to establish a sort of neutral reference guide / an argument map / the history and iterations an idea has gone through, in a neutral voice. Ideally, neutrality regarding PoV works in favor of those with the balance of arguments in their favor.
This need not be entirely new material, but instead simply a few mandatory / recommended headers in each wiki entry, pertaining to history, counterarguments etc. Could be worth it lifting the wiki from relative obscurity, with a new landing page, and marketed potentially as a reference guide for journalists researching current topics. Kruel’s LW interview with Shane Legg got linked to in a NYTimes blog, why not a suitable LW wiki article, too?
I don’t think that’s the case. Most people who are listened to on the future don’t tend to speak to an audience primarily consisting of futurists.
There are think tanks who employee people to think about the future and those think tanks tend generally to be quite good at influencing the public debate.
I also don’t think that academic has any special claim to be specialists about the future. When I think about specialists on futurism names like Stewart Brand or Bruce Sterling.
This is a very important and general point. While it is important to communicate ideas to a general audience, generally excessive communication to general audiences at the expense of communication to peers should be “bad news” when it comes to evaluating experts. Folks like Witten mostly just get work done, they don’t write popular science books.
Witten doesn’t ring a bell with me. Googling the name might mean Edward Witten and Tarynn Madysyn Witten. Do you mean either or them or someone else?
I mean Edward Witten, one of the most prominent physicists alive. The fact that his name does not ring a bell is precisely my point. The names that do ring a bell are the names of folks who are “good at the media,” not necessarily folks who are the best in their field.
Okay, given that the subject is theoretical physics and I’m not much into that field I understand why I have no recognition.
When looking at his Wikipedia I see he made Time 100 so it still might be worth knowing the name.
Witten is one of the greatest physicists alive, if not the greatest. He is the one who unified the various string theories into M-theory. He is also the only physicist to receive a Fields Medal.