“Smarter than us” is out!
We’re pleased to announce the release of “Smarter Than Us: The Rise of Machine Intelligence”, commissioned by MIRI and written by Oxford University’s Stuart Armstrong, and available in EPUB, MOBI, PDF, and from the Amazon and Apple ebook stores.
Can we instruct AIs to steer the future as we desire? What goals should we program into them? It turns out this question is difficult to answer! Philosophers have tried for thousands of years to define an ideal world, but there remains no consensus. The prospect of goal-driven, smarter-than-human AI gives moral philosophy a new urgency. The future could be filled with joy, art, compassion, and beings living worthwhile and wonderful lives—but only if we’re able to precisely define what a “good” world is, and skilled enough to describe it perfectly to a computer program.
AIs, like computers, will do what we say—which is not necessarily what we mean. Such precision requires encoding the entire system of human values for an AI: explaining them to a mind that is alien to us, defining every ambiguous term, clarifying every edge case. Moreover, our values are fragile: in some cases, if we mis-define a single piece of the puzzle—say, consciousness—we end up with roughly 0% of the value we intended to reap, instead of 99% of the value.
Though an understanding of the problem is only beginning to spread, researchers from fields ranging from philosophy to computer science to economics are working together to conceive and test solutions. Are we up to the challenge?
Special thanks to all those at the FHI, MIRI and Less Wrong who helped with this work, and those who voted on the name!
What, if anything, should regular LessWrong readers expect to get from reading this book?
A collection of the usual arguments in a (hopefully) concise, discursive, and easy to follow format. Plus a short story of the battle between a Terminator and a true AI...
I didn’t get anything new. I did get the knowledge that there was existed a book accessible to very lay people about Existential Risk from AI.
I see you’ve released it under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license.
Are you encouraging widespread gratis distribution, or are you keeping that option low-key with the intention of selling more from intelligence.org and Amazon?
Mostly the former.
You should contact Luke about what MIRI’s distribution strategy is.
Kudos, Stuart. I’m not sold on all of MIRI’s arguments, but this is the briefest cogent summary I’ve seen, and is now definitely the thing I’d hand to people who asked me questions about MIRI.
Thanks!
For years I’ve been saying that MIRI needs some basic intro texts that people can be pointed to when they first hear about these ideas. Ideally, there should be multiple such texts: Blurb length, short article, short book, and weighty tome say 200 words, 2000, 20000, a 200,000 words.
In recent years, this has started to happen.
Stuart Armstrong’s book is the short-book-length overview we need—perfect. Nick Bostrom’s maybe the long-book overview.
Luke Muehlhauser’s Facing the Singularity and James Barrat’s The Final Invention can fill this role to some extent.
Several good intro articles have been written over the last two years. I’m not sure which is the article I’d point people to—MIRI should probably choose one and push it as the basic overview article, as they have done for Stuart’s book.
I’ll be sending people to Stuart’s book from now on.
Cool!
By the way, are there any plans to eventually publish some of these MIRI-related ebooks in print form? Paper books tend to convey greater credibility than ebook-only.
We don’t plan to put in the extra effort to make a paper copy available. But later this year, Superintelligence will be available, and it will be both hardback and emblazoned with the word “Oxford.”
How much extra effort are you imagining it would be to make paper copies available of any given book? You already have cover art. Although checking the “Look Inside” on Amazon suggests that this one has not been well-proofread, let alone prettily typeset, I have somebody willing to do those things for free (well, for copies of the books) and I’m not a charitable organization—MIRI won’t ping its volunteers or throw fifty bucks at somebody to finagle CreateSpace?
Amazon’s ‘Look Inside’ shows the Kindle version, using whatever typesetting Amazon chooses; the PDF is better typeset. The main cost is researching the different options for making it available as a paperback and then verifying that research, which probably costs several hours of staff time, and our operations staff are currently doing higher-value work. If somebody I trust has already analyzed the options recently, and found the best choice or shown that it doesn’t matter much, then it should only take Alex 1-2 hours of his time to make the paperback available, which is probably worth it.
This sounds like bad instrumental rationality. If your current option is “don’t publish it in paperback at all”, and you are presented with an option you would be willing to take, publishing at a certain quality, if that quality was the best quality, then the fact that there may be better options you haven’t explored should never return your “best choice to make” to “don’t publish it in paperback at all.” Your only viable candidates should be: “Publish using a suboptimal option” and “Do verified research about what is the best option and then do that.”
As they say, “The perfect is the enemy of the good.”
Sure, but I’m not even sure at this stage that publishing a paperback version with CreateSpace is a better use of 2 hours of Alex’s time than the other stuff he’s doing. Are there hidden gotchas which make publishing worse than not-publishing even if it was totally free? (I’ve encountered many examples of this while running MIRI.) Will it actually take 5 hours of time rather than 2? I don’t know the answers to these questions, and this isn’t a priority. Deciding whether to publish a paperback copy of Smarter Than Us is, like, the 20th most important decision I’ll make this week. I’m not even sure that explaining all the different considerations I’m weighing for such a minor decision is worth the time I’ve spent typing these sentences. Anyway, I don’t mean to be rude and I understand why you and Alicorn are engaging me about this, it’s just that the decision is more complicated and less important (relative to all the invisible-to-LWers things we’re doing) than you might realize, and I don’t have time to explain it all. Again: if somebody can save us time on the initial research to figure out what’s a good idea, it might become competitive with the other things Alex is doing with his MIRI time.
I’m not clear on what Alex in particular has to do with this. Aren’t there people with lower opportunity cost you could go “hey, investigate self-publishing options” to? They are marketed to publishing-non-experts and while they don’t require zero skill, perhaps it doesn’t call for your scarcest and thinnest-spread people. Are you sure you don’t want to ask me any questions about my experience self-publishing with Createspace...?
Alex is just the one who would work with the files and CreateSpace, not necessarily the one who has to do the research about which company to publish through.
Another thing Alex is doing, btw, is finding a scalable way to outsource “general internet research” projects, without needing to find new contractor hours, validate them, sign a contract, etc. There was some service that looked awesome that I encountered 6 months ago when we had less money to spend on such things but now I can’t find it.
EDIT: Oh, and yes, I’d be happy to hear of your own experiences with (and judgements about) CreateSpace.
I have been happy with Createspace. It produces cheap-for-trade-quality sleek paperbacks, faithfully renders my cover art, is relatively easy to interact with in all the ways I haven’t chosen to delegate (and easy enough in those other ways that the delegate-ee is willing to work for one signed copy each of the books in question and a frontmatter acknowledgment and nothing else), and doesn’t cost any money up until I actually tell them to send me a book. I will happily show you three different volumes I have had Createspaced if you would like to see a physical copy and arrange to be near mine.
Maybe https://www.fancyhands.com/ ?
I think it was a different one, but that’s the best match I’ve found so far, so maybe it is indeed FancyHands.
I’ve had a few people also asking about physical copies, btw.
Sounds expensive...
I second this question, not on credibility grounds but because I prefer reading physical paper books rather than ebooks.
FYI, Smarter Than Us is now available in print form. :-)
Amazon.com
Amazon.co.uk
Paper books provide greater credibility than ebook-only because paper books normally go through a publisher who, because of the constraints in publishing physical books, is forced to determine whether the paper book meets some minimal standards of quality before publishing it. Publishing paper books just to gain credibility when they did not actually go through this process and the reasons to give them more credibility don’t apply, is a form of dark arts.
I can’t tell whether you mean something much more trivial by ‘dark arts’ than I do, or whether you really think even PR on this small a scale is dangerously corrosive of our core epistemic standards. (Or is otherwise nefarious.)
As a toy example, suppose I don’t usually wear suits, but I decide to wear a suit to a job interview or important meeting. Causally, this is useful in part because it’s misleading; it signals more wealth, status, and professionalism than is entirely representative of who I am. But concluding that suit-wearing is ‘dark arts’ for that reason is rather melodramatic, given that it’s so weakly and indirectly misleading, that it’s perfectly normal for people to signal in this fashion, and that it isn’t purely dishonest signaling. (Going to the trouble of publishing a book or wearing a suit does mean that you have more commitment and resources than a random person.)
Ditto ‘going out of your way in any fashion to impress people on first dates is Dark Arts because it doesn’t show them the Real You’, ‘putting extra money into making fancy business cards for your small business is Dark Arts because it makes you look like a bigger deal than you are’, etc. There’s a line we shouldn’t cross, yes, but it concerns signaling that’s much more extreme and unusual than any of these examples.
Memetically, yes, but note that nowadays you can set up print-on-demand through Amazon and sell paper books without having to go through a publisher. And, of course, vanity publishers have existed for a long time.
And print-on-demand and vanity publishers don’t have credibility, for exactly that reason.
Publishers don’t, but they produce paper books which provide the credibility we’re talking about. The percentage of people who’ll be able to recognize the publisher as a vanity publisher is vanishingly small.
The same applies: paper books have credibility because publishers have to vet them. Self-publishing and relying on people’s inability to determine that the book is self-published is a form of dark arts, because it takes advantage of people’s ignorance to give the book credibility.
There can be a lot of valid reasons to self-publish besides”tak[ing] advantage of people’s ignorance”.
Traditional publishing is close to becoming an atavism, a vestigial remnant of the pre-digital age.
Only in the minds of “yay future!” sorts of people. My parents think self-publishing a non-fiction book is sketchy, and they have every valid reason for believing this. I agree with them, in fact.
Take the “outside view”: if someone handed you a book and said, “This describes the most important scientific and moral challenge of our times”, and you had never heard of MIRI/FHI/CFAR/LW in any way whatsoever, what are the questions you’re going to ask? They’ll almost definitely be credibility questions; you want to read a book that’s been edited well and comes from some kind of well-known intellectual expert on the topic being discussed, preferably one from very credible institutions like academia and government. You’re a non-expert, after all, and it’s not worth your time becoming an expert on some obscure, highly technical topic just to judge whether you should spend $30 on a book.
Luckily, information above indicates that FHI is publishing Superintelligence under the Oxford University label, with actual academic credentials and such, in hardback print. I’ll be waiting for that one before I recommend anything to “the normals”.
“Why do you think so?”
Not in my case.
Oh, certainly not. Government is pretty much the opposite of a “credible institution” and as to academia, it depends on the field.
Do you think your views on this matter are typical of the audience to whom the book is targeted?
Digital or not, the fact that an article or book has passed the filter of a reputable publisher provides useful evidence in estimating its quality.
That’s most obvious for scientific publishing, where publishers employ expert editors and peer review. Publication of material intended for non-technical audiences has to meet lower standards, but there is still some level of editorial oversight.
Yep, the word for that is curating and it is a common and valuable activity. Art museums, for example, play a similar role. On the web brand-name blog collections (e.g. the Gawker stable) is also basically about the same thing—in this particular case, though, I don’t know if we’re talking about quality… :-D
Publishers aren’t interested in a book’s credibility except insofar as it affects its ability to drive sales. Self-publishing or vanity publishing isn’t evidence of inaccuracy or lack of intellectual integrity; it’s evidence of being boring, niche, poorly written, or otherwise of limited potential. That’s definitely not good news but it doesn’t qualify as “Dark Arts” by any means.
(Things are a little different in the academic press, but I don’t think that applies here.)
Normal publishers filter books for many aspects that would lead a reader to not want to read the book. While it’s true that not much filtering is done on accuracy and integrity, the aspects that the books are filtered on are still ones that typical readers care about. Relying on the user’s lack of knowledge of the publisher to make him think such filtering has been done, when it has not, is dark arts, deceptive, or whatever other term for bad things you like to use.
In other words, “I’m not tricking a user into reading an inaccurate book, I’m just tricking the user into reading a boring and poorly written book” isn’t an excuse.
Your line of argument looks strange to me. Basically, you are saying that any writer MUST go through gatekeepers to reach his readers and if he bypasses the gatekeepers that’s fraud and deceit upon the readers. I don’t find this approach reasonable.
Choosing to self-publish your book with the intention of having your readers mistake it for a non-self-published book is deceptive.
I don’t think this situation can really be described as a trick.
The way I see it, the main services publishers provide are distribution, marketing, and to a lesser extent editing. Self-publish or go with a vanity publisher, and you’re going to have a harder time getting into bookstores or other content distributors, because you haven’t gone through their filters but also because you’re not playing the usual game. But that just means you need to establish the book’s worth yourself. The typical reader won’t be able to tell the difference, but in order to get your book to the typical reader, you need to jump through a lot of hoops that are more or less equivalent to what a publisher would be doing for you. And popularity of course is a vindication all its own (there have been successful self-published books, albeit not many).
Now, if the question was whether it’s ethical to claim the status you’d get from being picked up by a major publisher (“I’m a published author!”), then I’d be right there with you. But I don’t think that having a vanity-published book in the wild, or even pointing people to it, is equivalent to making that claim.
This isn’t true because publishers will often hire someone to go through all of your footnotes.
Sure, but that’s just editing. Which is one of the basic services of a publisher; they have an interest in catching your mistakes and generally helping you not look stupid, because that sells, but that’s not the same thing as being nontrivial evidence of intellectual credibility. Obvious bullshit and pseudoscience gets published all the time, because that sells.
It was a lot more than editing on my last book.
I was hoping the link would be to a description of what your publisher did for you.
Sorry. They hired someone to look at every footnote. If my footnote didn’t seem justified this fact checker asked me for justification or had me chance it.
Ben Goertzel’s review in H+ Magazine. Excerpt:
My response in the comment section:
What I expect from formal “analytic philosophy” methods:
1) A useful decomposition of the issue into problems and subproblems (eg AI goal stability, AI agency, reduced impact, correct physical models on the universe, correct models of fuzzy human concepts such as human beings, convergence or divergence of goals, etc...)
2) Full or partial solutions some of the subproblems, ideally of general applicability (so they can be added easily to any AI design).
3) A good understanding of the remaining holes.
and lastly:
4) Exposing the implicit assumptions in proposed (non-analytic) solutions to the AI risk problem, so that the naive approaches can be discarded and the better approaches improved.
Ben expanded his original article by editing a reply to your points into the end.
Sigh… I’ll have to get round to addressing that point (though I’ve already addressed it several times already).
Bug report:
I bought the download-package. When I got the email—the link is invalid (that’s the error message I got when I pasted it into the browser). Same with the “You can view your order details” link.
However—luckily I still had the payment-thankyou tab still open and could reload that to get a download link. That one worked.
Passed your bug report on to MIRI, thanks!
No probs. I also replied to the email. But I figure if anybody else tries to download in the meantime—at least they know what they can do to fix it. :)
Can I ask why this book is not provided for free, given that it is essentially promotional material for MIRI, commissioned by MIRI and published by MIRI?
EDIT:
It seems that it is released under a Creative Commons licence.
So the price is just the cost of hosting it on Amazon?
It is provided as “pay-what-you-want” package, and is under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license.
Even if this qualifies as “promotional material”, it’s not exactly uncommon to charge money for promotional material. For example, politicians who are trying to get elected will charge money for their autobiographies.
I suppose that most people who buy an autobiography have already decided to vote for that politician.