Karma awards for proofreaders of the Less Wrong Sequences ebook
MIRI is gathering a bunch of Eliezer’s writings into a nicely-edited ebook, currently titled The Hard Part is Actually Changing Your Mind. This book will ultimately be released in various digital formats (Kindle MOBI, EPUB, and PDF). Much of the initial work for this project is complete. What we need now are volunteers to review the book’s chapters to:
verify that all the content has been correctly transferred (text, equations, and images),
proofread for any typographical errors (spelling, punctuation, layout, etc.),
verify all internal and external links,
and more.
This project has been added to Youtopia, MIRI’s volunteer system. (Click “Register as a Volunteer” here to sign up. Already signed up? Go here.)
LW Karma Bonus
For this special project, every point earned in Youtopia will also earn you 3 karma on LW!
Points are awarded based on the amount of time spent proofreading the book. For example, an hour of work logged in Youtopia earns you 10 points, which will also get you 30 LW karma. Karma is awarded by admins in a publicly-accountable way: all manual karma additions are listed here.
Questions about this project can be directed to alexv@intelligence.org or in the comments.
- 's comment on Bragging Thread, June 2014 by (8 Jun 2014 10:18 UTC; 97 points)
- 's comment on Open thread, 3-8 June 2014 by (4 Jun 2014 10:13 UTC; 6 points)
- 's comment on Open thread, Oct. 20 - Oct. 26, 2014 by (23 Oct 2014 21:18 UTC; 4 points)
- 's comment on Open thread, 3-8 June 2014 by (4 Jun 2014 9:01 UTC; 3 points)
- 's comment on Open thread, Nov. 3 - Nov. 9, 2014 by (6 Nov 2014 17:01 UTC; 0 points)
What, no bug bounty?
Just wanted to say: Whoever updated the texts from the original web version to the version used in the book, they did a very good job. So far I have compared only one chapter, but I am impressed. The changes are typically rephrasing a part of a sentence, or reordering some paragraphs, but the new version feels better. I am happy to have a more polished version of the Sequences, in addition to just another format.
(I am not proofreading. I translated some of the original articles to Slovak, and now I am updating my translations using the version from the book, with the goal of making a translation of the whole book one day. This is why I automatically noticed the differences between the original and the new versions.)
Proofreading is my calling and my reason to live. Count me in.
The book is not a blog; it is not published step by step, but as a whole, at the same time.
Therefore, I suggest removing comments like “Continued in next post” or “Followup to”; especially if it is a reference to the very next or very previous chapter, but also if it is two chapters behind. Specifically, chapters 3 and 4, 11 and 13, 20 and 22, 22 and 23, 28 and 29, 31 and 32, 31+32 and 34, 41 and 42, 49 and 50, 54 and 55, 56 and 57, 57 and 58, 59 and 60, 60 and 61, 62 and 63, 63 and 64, 64 and 65, 64 and 66… okay, I guess you get the idea. In a book, the next chapter being a followup to the previous chapter is pretty much expected.
On the other hand, chapters 16, 18, 28… are followups to chapters not included in the book. Maybe instead of “followup” it would be better to use some other words to express that this chapter is related to some online article.
Summary: the book is a different medium, so I suggest rewriting the standard “followup” notice, depending on situation:
it it’s a followup to the previous chapter (maybe also two chapter behind), just remove it;
if it’s a followup more chapters behind, use the full sentence to explain it;
if it’s a followup to something not included in the book, again use the full sentence, but a different one.
When is this expected to be released?
I really don’t know. The release date now depends almost entirely on how much volunteer effort gets put in.
I think this might be another way the karma system continues to be a feature of a map that doesn’t necessarily represent the territory.
High karma seems to me to be about being on board with LW orthodoxy, articulating that clearly in posts and comments and supporting the aims of the group.
I guess there is nothing wrong with that...though that is exactly how “karma points” are handed out in the church.
This is basically saying, “People will listen to your opinion X amount more inside LW for doing this volunteer work.” It is the exact same way in the church for those who evangelize a lot or display some other evidence of their commitment to the group.
The problem, I think, is that proofreading e-books or evangelizing the group’s message is not the slightest bit related to having a view that is in line with reality which is, as I understood it, one of the main aims of rationality.
The criticism you proffer has been made independently several times. The usual answer is that cogent, intelligent criticism that shows familiarity with the premises and conclusions of LW “orthodoxy” (and clearly locates its criticism as a problem with either the plausibility of a premise or the validity of an argument, or both) tends to get upvoted. You just have to show you’ve done your homework.
The problem is that exactly what qualifies as “homework” is determined by the in-group. And, as I said, this is exactly how it works in the church.
Nevermind that though. My point was really that karma isn’t tied to who is right, it is tied to who we like or who furthers our preferential ends. This karma-for-work deal is another example of that.
If karma is a popularity system, then fine. But there seems to be a lingering sentiment it is more about rationality, and how a given comment or commenter is in line with it. That’s not the case when you are giving people points to do tasks.
If we feel that we want to keep track of instrumentally useful contributions to the community but also want karma to remain a more-or-less pure representation of the reception of a user’s comments, then the obvious way to reflect that seems to be to create a metric other than karma to represent the former. This might also be a useful way of reflecting certain actions that have traditionally been rewarded by means of upvotes on comments bragging about them, like posting survey responses or donating to CFAR or MIRI. Granted, the development resources for this aren’t likely to appear in the near term.
Elsewhere in these comments I’ve mentioned the XP metric that the Everything2 community created to fill a karma-like niche. It was after my time, but I’m told they ended up creating a “GP” metric (RPG metaphor, yes) that worked similarly to this.
badges
Barring an objective method for telling what arguments are right, this is the way any human-run evaluation system (including e.g. formal peer review or university grades) has to work. You can try to eliminate the “who we like” part by trying to blind the identities of the people in question, but since one cannot assess degree-of-correctness directly, one has to rely on some other criteria, e.g. the extent to which the comment seems to take previous work into account. And those other criteria and their parameters, like what counts as “previous work”, are ultimately set by an in-group consensus. (I felt that James Paul Gee had a particularly good elaboration of this.)
But I do agree that awarding karma for work distances karma from correctness even further than would be necessary. (Not sure whether it’s a bad thing, though.)
At the very low end, a certain amount of karma must be earned for certain functionality to become available to a user account. But past a certain point—somewhere in the 200-500 range—more karma just doesn’t matter very much when assessing a user account. After that point, the meaning of a karma score attaches much more strongly to specific comments than to users.
The corollary is that the karma reward will incent newcomers a lot more than old hands. I don’t have a problem with that.
The “who we like” part seems to be how some people operate, especially vis-a-vis mass downvoting. I view that as counterproductive. What the phrase “furthers our preferential ends” denotes is not clear to me; if it means something like an upvote is meant to signal “I want to see more like this comment” and a downvote the opposite, then my understanding of LW consensus is, yes, that’s the idea.
Huh? If karma isn’t very meaningful past a certain count, why keep track of it at all? Why not just call everybody who reached 500 karma points “vetted” and leave it at that? (I suspect the answer is that karma does matter to some significant portion of the people here, but I’m open to hearing why you think otherwise.)
“Who we like” could include mass downvoting. More than it, it involves applause lights hanging above certain members who espouse popular views or who have done something, apart from making a rational comment, to garner favor in the group. This could be proofreading a LW-approved text, organizing a meetup, etc., etc.
The key characteristic of this earned karma is that is has zero to do making a direct contribution to a more accurate map.
It’s is just interesting to me because it’s a lot like the church from whence I came. Signaling devotion to the cause becomes more important than being right about the merits of the cause.
LW is a community specifically committed to the mission of “refining rationality” and, therefore, (you’d think) making sure things like karma systems work to incentivize members toward that end. I don’t see that happening.
LW is the best blog/forum I’ve seen—virtually troll free, consistent flow of interesting articles, thoughtful & well-written comments, lots of people waaaay smarter than me, etc. That is part of why it is so interesting that the karma system seems so...weak.
I didn’t say it was strictly meaningless or negligibly meaningful.
The reasons for not having an explicit “vetted” status in lieu of accumulating karma are to a certain extent historical—it wasn’t thought of when the LW karma system was implemented, since that was adapted whole-hog from the Reddit codebase. I think the reason why such a change hasn’t been made in the meantime is three-fold: (i) it would obviate the “Top Contributor, 30 Days” status incentive (the most feasible way to top that list is to write highly upvoted front page articles), (ii) it would obviate any loss-aversion-motivated engagement induced by each account’s “karma in the last 30 days” score; and (iii) on general “if it ain’t broke don’t fix it” grounds.
Yes, churches win at creating socially cohesive communities. If your complaint is that that karma system induces undue (in your view) social cohesiveness, my response is, “feature, not bug”. (Here’s some LW canon on the usefulness and pitfalls of social cohesiveness.) If you suspect that the harmfulness of the system outweighs the usefulness, set some standards for harmfulness and usefulness and then collect some evidence for and against that hypothesis. Keep in mind that by design, the biggest rewards go to high-quality front page posts (like this one criticizing time spent kibitzing on LW).
Nor did I say you said that. You said this:
And I replied with this:
My apologies if you feel I rephrased you inaccurately, or missed your meaning.
To the system being historical: “That’s the way we’ve always done it” isn’t a very good reason for any policy or behavior to continue.
To (i) and (ii): Keep 30-day karma and ditch cumulative karma. No need to obviate anything.
To (iii): Begging the question.
Social cohesive is fine, of course. I agree it is a feature. And it is great!
It (social cohesiveness) shouldn’t take priority over adherence to reality. When it does, it’s buggy. And it is what happens in the church. In my experience, they value “unity” over rationality. At LW, that is a clear no-no. (I agree it is a no-no. Make sure you are correct first; only then be unified. Lest dogma tends to ensue.)
This is way outside my level of interest in, or commitment to, LW. I’ve given what I believe to be a reasonable criticism of the karma system (one that you mention has been independently noted many times) and made what I believe to be an accurate and helpful analogy (i.e. how “karma” works in the church).
It’s on the record for anyone who is interested to do with it what they’d like, or ignore it altogether.
One of my favorite posts. Though I would say reading and interacting on LW is really good for rationality novices—like me.
At some point, it does become a bit of an anti-rational engagment. For me, in this thread, that time is now.
Tap.
So people who don’t post on LW for a month or more become indistinguishable from newbies?
What happens if old karma is only displayed as a percentage, rather than as a number? That way you know generally what the community thinks of their post quality, without more-prolific posters overwhelming less frequent posters?
I am still not quite sure what is the problem we are trying to solve here.
What exactly do you hope to gain by screwing around with the karma system?
I like this suggestion.
Are you sure that the other features you mention aren’t in part because of the “weak” karma system? Trolls get actively downvoted, for instance.
I’d imagine the existing system works pretty well towards several ends… troll deterence and the writing quality of posts/comments being a couple examples.
Will the ebook be offered for free?
It will probably be available as pay-what-you-want, similar to the Facing the Intelligence Explosion ebook.
The LW Karma idea makes me uncomfortable. And I’m saying this as a user who doesn’t have that much karma, and might be interested in doing this.
If you come by my office and shine my shoes, I will upvote your posts until you get an extra 500 karma points. Doing my laundry will net you a cool thousand karma points.
I think I’d rather do my own laundry than upvote a thousand posts. If the average user even has a thousand posts.
I suppose that’s what scripts are for, though.
Indeed, and they aren’t too hard to find and modify.
EDIT: Hi there! Since you’re downvoting everything I say at this very minute, and haven’t got to this post yet, could you please stop for a second when you do and say ‘hi’?
EDIT2: Guess not.
don’t feed.
What’s the current estimated time when the project will finish?
Hopefully by the end of the year.
Is this going to be released as a real book as well? I see only mentions of digital release in the post.
The plan is to do both.
Great! Thanks guys!
I recently read the sequences at a fairly rapid pace using the ebooks versions available on my ipad and there were certainly a few common things I found intensely irritating. I definitely found some of the sequences interesting enough to read again more closely. Which of the sequences are included in the ebook?
Basically all of them, with some modifications (e.g. a significantly reduced QM sequence), and with some reorganizing to improve flow.
A professional editor would tell you to cut, at least, 100k. You should strongly consider some hacking and slashing.
Which particular things did you find irritating?
Poor handling of images, maths sections and inter-sequence links were the most notable ones I remember
I’ve proofread three of the chapters. There are 340 of them in the version of the ebook that I was sent, and the whole thing runs over 2000 pages. At roughly 250 words per page, that’s 500k words.
So, basically what alexvermeer said—pretty much all of them.
Does anyone know what happened to the version that was supposed to be reviewed/edited down by a professional so it could be publishable length? There’s so much good stuff there I’d love to be able to send to friend and family but 500k worth of blog posts is much harder to send someone than a nicely published 200k version.
Quiz: what kind of organizations solicit hard work from their members offering only an immaterial reward as compensation?
Ah ok, keeping your browser logged in a website while attending to your own business is not hard work...
Wikipedia, YouTube, Reddit, any blogging service...
YouTube actually pays with real cash people who upload popular videos. AFAIK the other organizations you mentioned don’t give away “karma points” in exchange of work.
The point is that using social status to motivate people who provide content or do other useful work is very common; it’s arguably one of the defining features of what the press calls Web 2.0. Karma is just a (crude) way of quantifying that.
I first saw a similar quantification scheme on Everything2, an early competitor to Wikipedia that morphed into more of a writers’ platform when it became clear that Wikipedia was winning. They called it XP, though.
IIUC these reputation systems are based on user votes.
LessWrong karma is usually also generated that way, at least until now. But if MIRI starts to sell LessWrong karma in exchange of work, then what’s the point of having a karma system? What does ‘karma’ represent?
What did it represent until now?
If your answer is “user votes,” as your comment suggests, then I guess what it represents now is “a combination of user votes and MIRI work.” Is the latter less valuable a thing to represent than the former?
What percent of YouTube uploaders would you estimate get paid more than, say, $1/month?
I don’t know, but I expect something like a power-law distribution over payments.
volunteer organizations
Do they award “karma points”?
The closest example I can think of is the Catholic Church selling indulgences.
Volunteer organizations motivate and compensate their members in a variety of ways. Most of the ones I’m acquainted with do so via immaterial rewards, which was your original question.
With respect to the new goalpost of “karma points”—can you clarify what characterizes the set of “karma point” like reward structures? If the Catholic Church selling indulgences qualifies, I’m not sure why wikipedia granting wider edit privileges wouldn’t, for example… those things are certainly different from one another, but then again both are different from karma points. I’m not sure what differences are significant, on your view.
Lots. Scientific publications solicit hours of work from reviewers, for every paper published, without compensation or even much in the way of professional credit to the anonymous reviewers.
so they don’t offer any reward. My point.
Hrm? As a scientific reviewer, you get the reward of being able to list “reviewer for XYZ” on your CV. Which is an “immaterial reward”, probably about on a par with LW karma. Both are a mild point of prestige, but not really enough to boast about in isolation.
No one cares about that. The real point of reviewing papers is keeping up w/ your field.
No one ever lists that in any field I know.
So, if reviewers are allowed to mention the fact on their CV, and this provides a significant, even if small, professional advantage, then that’s a real reward in the sense that it leads to material benefits.
I doubt that LW karma is of much help in that direction.
You addressed this in a rude way, but I think it was good this was addressed. It’s a shame this got downvoted.