Thank you for your comments. I have included them in version 1.1 of the map, where I have swapped FRI and OpenAI/DeepMind, added Crystal Trilogy and corrected the spelling of Vernor Vinge.
SoerenE
Karma: 153
I have taken the survey.
I think difference in date of birth (1922 vs ~1960) is less important than difference of date of publication (2003 vs ~2015).
On the Outside View, is criticism 12 years after publication more likely to be valid than criticism levelled immediately? I do not know. On one hand, science generally improves over time. On the other hand, if a particular work get the first criticism after many years, it could mean that the work is of higher quality.
I should clarify that I am referring to the section David Chapman calls: “Historical appendix: Where did the confusion come from?”. I read it as a criticism of both Jaynes and his book.
No, I do know what Yudkowsky’s argument is. Truth be told, I probably would be able to evaluate the arguments, but I have not considered it important. Should I look into it?
I care about whether “The Outside View” works as a technique for evaluating such controversies.
Yes! From the Outside View, this is exactly what I would expect substantial, well-researched criticism to look like. Appears very scientific, contains plenty of references, is peer-reviewed and published in “Journal of Statistical Physics” and has 29 citations.
Friedman and Shimonys criticism of MAXENT is in stark contrast to David Chapmans criticism of “Probability Theory”.
Could you post a link to a criticism similar to David Chapman?
The primary criticism I could find was the errata. From the Outside View, the errata looks like a number of mathematically minded people found it to be worth their time to submit corrections. If they had thought that E. T. Jaynes was hopelessly confused, they would not have submitted corrections of this kind.
I don’t think it’s a good sign for a book if there isn’t anybody to be found that criticizes it.
I think it is a good sign for a Mathematics book that there isn’t anybody to be found that criticizes it except people with far inferior credentials.
Thank you for pointing this out. I did not do my background check far enough back in time. This substantially weakens my case.
I am still inclined to be skeptical, and I have found another red flag. As far as I can tell, E. T. Jaynes is generally very highly regarded, and the only person who is critical of his book is David Chapman. This is just from doing a couple of searches on the Internet.
There are many people studying logic and probability. I would expect some of them would find it worthwhile to comment on this topic if they agreed with David Chapman.
I do not know enough about logic to be able to evaluate the argument. But from the Outside View, I am inclined to be skeptical about David Chapman:
DAVID CHAPMAN
“Describing myself as a Buddhist, engineer, scientist, and businessman (...) and as a pop spiritual philosopher“
Web-book in progress: Meaningness
Tagline: Better ways of thinking, feeling, and acting—around problems of meaning and meaninglessness; self and society; ethics, purpose, and value.
EDWIN THOMPSON JAYNES
Professor of Physics at Washington University
Most cited works:
Information theory and statistical mechanics − 10K citations
Probability theory: The logic of science − 5K citations
The tone of David Chapman’s refutation:
E. T. Jaynes (...) was completely confused about the relationship between probability theory and logic. (...) He got confused by the word “Aristotelian”—or more exactly by the word “non-Aristotelian.” (...) Jaynes is just saying “I don’t understand this, so it must all be nonsense.”
My apologies for not being present. I did not put it into my calendar, and it slipped my mind. :(
You might also be interested in this article by Kaj Sotala: http://kajsotala.fi/2016/04/decisive-strategic-advantage-without-a-hard-takeoff/
Even though you are writing about the exact same subject, there is (as far as I can tell) no substantial overlap with the points you highlight. Kaj Sotala titled his blog post “(Part 1)” but never wrote a subsequent part.
Also, it looks like the last time slot is 2200 UTC. I can participate from 1900 and forward.
I will promote this in the AI Safety reading group tomorrow evening.
The title says 2017/6/27. Should it be 2017-05-27?
Good luck with meetup!
In the Skype-based reading group, we followed the “Ambitious” plan from MIRI’s reading guide: https://intelligence.org/wp-content/uploads/2014/08/Superintelligence-Readers-Guide-early-version.pdf We liked the plan. Among other things, the guide recommended splitting chapter 9 into two parts, and that was good advice.
Starting from chapter 7, I made slides appropriate for a 30 minute summary: http://airca.dk/reading_group.htm
Be sure to check out the comments from the Lesswrong reading group by Katja Grace: http://lesswrong.com/lw/kw4/superintelligence_reading_group/
I think I agree with all your assertions :).
(Please forgive me for a nitpick: The opposite statement would be “Many humans have the ability to kill all humans AND AI Safety is a good priority”. NOT (A IMPLIES B) is equivalent to A AND NOT B. )
There are no specific plans—at the end of each session we discuss briefly what we should read for next time. I expect it will remain a mostly non-technical reading group.
Do you think Leo Szilard would have had more success through through overt means (political campaigning to end the human race) or surreptitiously adding kilotons of cobalt to a device intended for use in a nuclear test? I think both strategies would be unsuccessful (p<0.001 conditional on Szilard wishing to kill all humans).
I fully accept the following proposition: IF many humans currently have the capability to kill all humans THEN worrying about long-term AI Safety is probably a bad priority. I strongly deny the antecedent.
I guess the two most plausible candidates would be Trump and Putin, and I believe they are exceedingly likely to leave survivors (p=0.9999).
The word ‘sufficiently’ makes your claim a tautology. A ‘sufficiently’ capable human is capable of anything, by definition.
Your claim that Leo Szilard probably could have wiped out the human race seems very far from the historical consensus.
Thank you for explaining.