what do you view as the purpose of SIAI’s publishing papers?
Grab the interest of smart people who won’t be grabbed by cheaper methods. This has worked before. Also: Many smart and productive people are extremely busy, and they use “Did they bother to pass peer review?” as a filter for what they choose to read. In addition, many smart people prefer to read papers over blog posts because papers are generally better organized, are more clearly written, helpfully cite related work, etc.
Reduce communication overhead. We don’t have time to have a personal conversation with every interested smart person, and blog posts are often too disorganized and ambiguous to help. Though for this, a scholarly AI risk wiki would probably be even better. Luckily, as I say in that post, there isn’t much additional cost involved in turning parts of papers into wiki articles, or combining wiki articles into papers.
Grab some prestige and credibility, because this matters to lots of the people we care about.
Show that we’re capable of doing serious research. “Eliezer did some work with Marcello that we can never tell you about” and “We wrote some blog posts this month” don’t quite show to most people that we can do research.
Be kinda-forced into writing more clearly, and in a way that is more thoroughly connected to the relevant empirical literatures, than we might otherwise be tempted to write.
Why not just publish them directly on the site, in (probably) a much more readable format?
As I said before, many people find papers more readable than ambiguous blog posts barely connected to the relevant literatures. Eliezer’s papers aren’t written in a different style than his blog posts, anyway. Also, peer review often improves the final product.
The problem with conformity in academia… is that a) it restricts the sorts of things you can say, b) restricts you, in many cases, to an awkward way of wording things, and c) it makes academia a less fertile ground for recruiting people.
Agree with (a) and somewhat with (b), but we’re only writing certain things in paper form. Like I said, the vast majority of FAI work and discussion happens outside papers. I don’t know what you mean by (c).
it seems… that we aren’t going to have that much prestige in academia anyway, given that the main prestige mechanism is elite university affiliations, and most of us don’t have those.
I don’t care about something like “average prestige in academia.” What I care about is some particular people thinking we have enough credibility to bother reading and engaging with. Many of the people I care about won’t bother to check whether the author of an article has elite university affiliation, but will care if we bothered to write up our ideas clearly and with references to related work. The Singularity and Machine Ethics looks much less crankish than Creating Friendly AI, even though none of the authors have elite university affiliation.
Which people have come through Eliezer and Bostrom’s papers?
Still gathering data, and I haven’t gathered permission to share it. I think two people who wouldn’t mind you knowing they came to x-risk through “Astronomical Waste” are Nick Beckstead and Jason Gaverick Matheny.
Using my own personal experiences is… very far from generalizing from a single example
Point taken.
How hard would it be to get someone to read a paper, vs. a single Sequence post of equal length, or a bunch of Sequence posts that sum to an equal length?
My intended point was that sometimes a paper has summed up the main points from something that Eliezer took 30 blog posts to write when he wrote The Sequences. But obviously you don’t have to write a paper to do this, so I drop the point.
If all new areas of research are developed through in-person conversations and mailing lists, that doesn’t imply that papers are a good way to do FAI research; it implies that papers are a bad way to do all those other kinds of research.
Remember: almost all FAI research is not done via papers. In my above list of reasons why SI publishes papers, I didn’t even think to mention “to produce original research” (and I won’t go back and add it now), though that sometimeshappens.
there are some instances of academic moderation being net good rather than net bad. However, to quote of your earlier arguments, “don’t generalize from one example”. I’m sure that there are some well-moderated journals, just as I’m sure there are Mafia bosses who are really nice helpful guys. However, that doesn’t imply that hanging out with Mafia bosses is a good idea.
If one journal is poorly moderated, then you jump to another one. Unlike Mafia bosses, a “problem” with journal moderators means “I wasted a few hours communicating with them and making revisions,” not “They decided to cut off my thumbs.”
For people who “are extremely busy, and they use “Did they bother to pass peer review?” as a filter for what they choose to read”, which specific examples are you thinking of, and how much any of them become nontrivial members of our community, or helped us out in nontrivial ways?
I’m sure there are people who a) are very smart, b) look impressive on paper, who c) we’ve contacted about FAI research, and d) have said “I’m not going to pay attention, since this isn’t peer reviewed” (or some equivalent). However, I think that for most of those people, that isn’t their true rejection (http://lesswrong.com/lw/wj/is_that_your_true_rejection/), and they aren’t going to take us seriously anyway. But I could be wrong—what evidence do you have in mind?
A lot of your points are criticisms of blog posts, like “a lot of them don’t have citations”, or “a lot of them are poorly organized”. These are true in many cases. However, if SIAI is considering whether to publish some given idea in paper or blog post form, they could simply spend the (fairly small) effort to write a blog post which was well organized and had citations, thereby making these problems moot.
Journal editors obviously aren’t perfectly analogous to mob bosses. However, I’ve heard many stories from academics of authors spending huge amounts of time and effort trying to get stuff published. In the most recent case, which I discussed with a grad student just a few hours ago, it took hundreds of hours, over a full year. If it’s usually easy to get around that sort of thing, by just publishing in a different journal, why don’t more academics do so?
Your first two questions ask about evidence that I already said I’m not in a position to share yet. I know that’s unsatisfying, but… are your priors on my claims being true really very low? Famous scientists, especially, are barraged with a few purported unifications of quantum theory and relativity every month, and “Did they bother to pass peer review?” is a pretty useful heuristic for them. When you visualize a busy academic receiving CFAI from one person, and The Singularity and Machine Ethics from somebody else, which one do you think they’re more likely to read and take seriously, and why? (Feel free to take this as a rhetorical question.)
A lot of your points are criticisms of blog posts, like “a lot of them don’t have citations”, or “a lot of them are poorly organized”. These are true in many cases. However, if SIAI is considering whether to publish some given idea in paper or blog post form, they could simply spend the (fairly small) effort to write a blog post which was well organized and had citations, thereby making these problems moot.
The effort required may be much larger than you think. Eliezer finds it very difficult to do that kind of work, for example. (Which is why his papers still read like long blog posts, and include very few citations. CEV even contains zero citations, despite re-treading ground that has been discussed by philosophers for centuries, as “The Singularity and Machine Ethics” shows.)
And if you’ve done all that work, then why not also tweak it for use in a scholarly AI risk wiki, and then combine it with a couple other wiki articles into a paper?
I’ve heard many stories from academics of authors spending huge amounts of time and effort trying to get stuff published. In the most recent case, which I discussed with a grad student just a few hours ago, it took hundreds of hours, over a full year. If it’s usually easy to get around that sort of thing, by just publishing in a different journal, why don’t more academics do so?
Because their career depends on satisfying their advisors, or on getting published in particular journals. SI researchers’ careers don’t depend on investing hundreds of hours making revisions. If publishing in a certain journal is going to require 30 hours of revisions that don’t actually improve the paper in our eyes, then we aren’t going to bother publishing in that journal.
The effort required may be much larger than you think. Eliezer finds it very difficult to do that kind of work, for example. (Which is why his papers still read like long blog posts, and include very few citations. CEV even contains zero citations, despite re-treading ground that has been discussed by philosophers for centuries, as “The Singularity and Machine Ethics” shows.)
If this is the case, then a significant benefit to Eliezer of trying to get papers published would be that it would be excellent discipline for Eliezer, and would make him an even better scholar.
A benefit that would follow on is that it would establish by example that nobody is above showing their work, acknowledging their debts and being current on the relevant literature. Conceivably Eliezer is such a talented guy that it is of no benefit to him to do these things, but if everyone who thought they were that talented were excused from showing their work and keeping current then progress would slow significantly.
It also avoids reinventing the wheel. No matter how smart Eliezer is, it’s always conceivable that someone else thought of something first and expressed it in rigorous detail with proper citations. A proper literature review avoids this waste of valuable research time.
It also avoids reinventing the wheel. No matter how smart Eliezer is, it’s always conceivable that someone else thought of something first and expressed it in rigorous detail with proper citations. A proper literature review avoids this waste of valuable research time.
Luke (and his remote research assistants) have this angle covered.
A lot of your points are criticisms of blog posts, like “a lot of them don’t have citations”, or “a lot of them are poorly organized”. These are true in many cases. However, if SIAI is considering whether to publish some given idea in paper or blog post form, they could simply spend the (fairly small) effort to write a blog post which was well organized and had citations, thereby making these problems moot.
… and if you’re going through the effort of writing a blog post that’s journal-quality anyway, you might as well go ahead and publish it as a full paper while you’re at it.
… and if you’re going through the effort of writing a blog post that’s journal-quality anyway, you might as well go ahead and publish it as a full paper while you’re at it.
If it’s usually easy to get around that sort of thing, by just publishing in a different journal, why don’t more academics do so?
Clearly the grad student (or more likely, their advisor) thought that getting published in journal X was worth enough status to spend over a year working on it. Fake utility functions and all that. Not even academics are perfectly rational.
Grab the interest of smart people who won’t be grabbed by cheaper methods. This has worked before. Also: Many smart and productive people are extremely busy, and they use “Did they bother to pass peer review?” as a filter for what they choose to read. In addition, many smart people prefer to read papers over blog posts because papers are generally better organized, are more clearly written, helpfully cite related work, etc.
Reduce communication overhead. We don’t have time to have a personal conversation with every interested smart person, and blog posts are often too disorganized and ambiguous to help. Though for this, a scholarly AI risk wiki would probably be even better. Luckily, as I say in that post, there isn’t much additional cost involved in turning parts of papers into wiki articles, or combining wiki articles into papers.
Grab some prestige and credibility, because this matters to lots of the people we care about.
Show that we’re capable of doing serious research. “Eliezer did some work with Marcello that we can never tell you about” and “We wrote some blog posts this month” don’t quite show to most people that we can do research.
Be kinda-forced into writing more clearly, and in a way that is more thoroughly connected to the relevant empirical literatures, than we might otherwise be tempted to write.
As I said before, many people find papers more readable than ambiguous blog posts barely connected to the relevant literatures. Eliezer’s papers aren’t written in a different style than his blog posts, anyway. Also, peer review often improves the final product.
Agree with (a) and somewhat with (b), but we’re only writing certain things in paper form. Like I said, the vast majority of FAI work and discussion happens outside papers. I don’t know what you mean by (c).
I don’t care about something like “average prestige in academia.” What I care about is some particular people thinking we have enough credibility to bother reading and engaging with. Many of the people I care about won’t bother to check whether the author of an article has elite university affiliation, but will care if we bothered to write up our ideas clearly and with references to related work. The Singularity and Machine Ethics looks much less crankish than Creating Friendly AI, even though none of the authors have elite university affiliation.
Still gathering data, and I haven’t gathered permission to share it. I think two people who wouldn’t mind you knowing they came to x-risk through “Astronomical Waste” are Nick Beckstead and Jason Gaverick Matheny.
Point taken.
My intended point was that sometimes a paper has summed up the main points from something that Eliezer took 30 blog posts to write when he wrote The Sequences. But obviously you don’t have to write a paper to do this, so I drop the point.
Remember: almost all FAI research is not done via papers. In my above list of reasons why SI publishes papers, I didn’t even think to mention “to produce original research” (and I won’t go back and add it now), though that sometimes happens.
If one journal is poorly moderated, then you jump to another one. Unlike Mafia bosses, a “problem” with journal moderators means “I wasted a few hours communicating with them and making revisions,” not “They decided to cut off my thumbs.”
This comment and your others in this thread have greatly improved my confidence in SI.
Re-replying:
For people who “are extremely busy, and they use “Did they bother to pass peer review?” as a filter for what they choose to read”, which specific examples are you thinking of, and how much any of them become nontrivial members of our community, or helped us out in nontrivial ways?
I’m sure there are people who a) are very smart, b) look impressive on paper, who c) we’ve contacted about FAI research, and d) have said “I’m not going to pay attention, since this isn’t peer reviewed” (or some equivalent). However, I think that for most of those people, that isn’t their true rejection (http://lesswrong.com/lw/wj/is_that_your_true_rejection/), and they aren’t going to take us seriously anyway. But I could be wrong—what evidence do you have in mind?
A lot of your points are criticisms of blog posts, like “a lot of them don’t have citations”, or “a lot of them are poorly organized”. These are true in many cases. However, if SIAI is considering whether to publish some given idea in paper or blog post form, they could simply spend the (fairly small) effort to write a blog post which was well organized and had citations, thereby making these problems moot.
Journal editors obviously aren’t perfectly analogous to mob bosses. However, I’ve heard many stories from academics of authors spending huge amounts of time and effort trying to get stuff published. In the most recent case, which I discussed with a grad student just a few hours ago, it took hundreds of hours, over a full year. If it’s usually easy to get around that sort of thing, by just publishing in a different journal, why don’t more academics do so?
Your first two questions ask about evidence that I already said I’m not in a position to share yet. I know that’s unsatisfying, but… are your priors on my claims being true really very low? Famous scientists, especially, are barraged with a few purported unifications of quantum theory and relativity every month, and “Did they bother to pass peer review?” is a pretty useful heuristic for them. When you visualize a busy academic receiving CFAI from one person, and The Singularity and Machine Ethics from somebody else, which one do you think they’re more likely to read and take seriously, and why? (Feel free to take this as a rhetorical question.)
The effort required may be much larger than you think. Eliezer finds it very difficult to do that kind of work, for example. (Which is why his papers still read like long blog posts, and include very few citations. CEV even contains zero citations, despite re-treading ground that has been discussed by philosophers for centuries, as “The Singularity and Machine Ethics” shows.)
And if you’ve done all that work, then why not also tweak it for use in a scholarly AI risk wiki, and then combine it with a couple other wiki articles into a paper?
Because their career depends on satisfying their advisors, or on getting published in particular journals. SI researchers’ careers don’t depend on investing hundreds of hours making revisions. If publishing in a certain journal is going to require 30 hours of revisions that don’t actually improve the paper in our eyes, then we aren’t going to bother publishing in that journal.
Both links go to the same place.
Fixed, thanks.
If this is the case, then a significant benefit to Eliezer of trying to get papers published would be that it would be excellent discipline for Eliezer, and would make him an even better scholar.
A benefit that would follow on is that it would establish by example that nobody is above showing their work, acknowledging their debts and being current on the relevant literature. Conceivably Eliezer is such a talented guy that it is of no benefit to him to do these things, but if everyone who thought they were that talented were excused from showing their work and keeping current then progress would slow significantly.
It also avoids reinventing the wheel. No matter how smart Eliezer is, it’s always conceivable that someone else thought of something first and expressed it in rigorous detail with proper citations. A proper literature review avoids this waste of valuable research time.
Luke (and his remote research assistants) have this angle covered.
… and if you’re going through the effort of writing a blog post that’s journal-quality anyway, you might as well go ahead and publish it as a full paper while you’re at it.
Research assistants!
Clearly the grad student (or more likely, their advisor) thought that getting published in journal X was worth enough status to spend over a year working on it. Fake utility functions and all that. Not even academics are perfectly rational.