(Something genuinely amusing, given the context, about the above being at 3 points out of 2 votes after four hours, compared to its parent being at 30 points out of 7 votes after five.)
It’s bad that comments which are good along three different axes, and bad along none as far as I can see, are ranked way below comments that are much worse along those three axes and also have other flaws
I have an alternative and almost orthogonal interpretation for why the karma scores are the way they are.
Both in your orthonormal-Matt example, and now in this meta-example, the shorter original comments require less context to understand and got more upvotes, while the long meandering detail-oriented high-context responses were hardly even read by anyone.
This makes perfect sense to me—there’s a maximum comment length after which I get a strong urge to just ignore / skim a comment (which I initially did with your response here; and I never took the time to read Matt’s comments, though I also didn’t vote on orthonormal’s comment one way or another, nor vote in the jessicata post much at all), and I would be astonished if that only happened to me.
Also think about how people see these comments in the first place. Probably a significant chunk comes from people browsing the comment feed on the LW front page, and it makes perfect sense to scroll past a long sub-sub-sub-comment that might not even be relevant, and that you can’t understand without context, anyway.
So from my perspective, high-effort, high-context, lengthy sub-comments intrinsically incur a large attention / visibility (and therefore karma) penalty. Things like conciseness are also virtues, and if you don’t consider that it in your model of “good along three different axes, and bad along none as far as I can see”, then that model is incomplete.
(Also consider things like: How much time do you think the average reader spends on LW; what would be a good amount of time, relative to their other options; would you prefer a culture where hundreds of people take the opportunity cost to read sub-sub-sub-comments over one where they don’t; also people vary enormously in their reading speed; etc.)
Somewhat related: My post in this thread on some of the effects of the existing LW karma system: If we grant the above, one remaining problem is that the original orthonormal comment was highly upvoted but looked worse over time:
What if a comment looks correct and receives lots of upvotes, but over time new info indicates that it’s substantially incorrect? Past readers might no longer endorse their upvote, but you can’t exactly ask them to rescind their upvotes, when they might have long since moved on from the discussion.
First, some off the cuff impressions of matt’s post (in the interest of data gathering):
In the initial thread I believe that I read the first paragraph of matt’s comment, decided I would not get much out of it, and stopped reading without voting.
Upon revisiting the thread and reading matt’s comment in full, I find it difficult to understand and do not believe I would be able to summarize or remember its main points now, about 15 minutes after the fact.
This seems somewhat interesting to test, so here is my summary from memory. After this I’ll reread matt’s post and compare what I thought it said upon first reading with what I think it says upon a second closer reading:
[person who met geoff] is making anecdotal claims about geoff’s cult-leader-ish nature based on little data. People who have much more data are making contrary claims, so it is surprising that [person]’s post has so many upvotes. [commenter to person] is using deadpan in a particular way, which could mean multiple things depending on context but I lack that context. I believe that they are using it to communicate that geoff said so in a non-joking manner, but that is also hearsay.
Commentary before re-reading: I expect that I missed a lot, since it was a long post and it did not stick in my mind particularly well. I also remember a lot of hedging that confused me, and points that went into parentheticals within parentheticals. These parentheticals were long enough that I remember losing track of what point was being made. I also may have confabulated arguments in this thread about upvotes and some from matt’s post.
I wanted to keep the summary “pure” in the sense that it is a genuine recollection without re-reading, but for clarity [person] is othonormal and [commenter to person] is RyanCarey.
Second attempt at summarizing while flipping back and forth between editor and matt’s comment:
RyanCarey is either mocking orthonormal or providing further weak evidence, but I don’t know which.
One reading of orthonormal’s comment is that he had a strong first impression, has been engaging in hostile gossip about Geoff, and has failed to update since in the presence of further evidence. Some people might have different readings. Orthonormal’s post has lots of karma, they have 15k+ karma in general, and their post is of poor quality, therefore the karma system may be broken.
RyanCarey used deadpan in an unclear way, I believe the best reading of their comment is that Geoff made a joke about being a cult leader. Several other commenters and I, all of whom have much more contact with Geoff than orthonormal, do not think he is or wants to be a cult leader. It is out of character for Geoff to make a deadpan joke about wanting to be a cult leader and RyanCarey didn’t give confidence in their recollection of their memory, therefore people should be unimpressed with the anecdote.
I am explicitly calling out orthonormal’s comment as hostile gossip, which I will not back up here but will back up in a later post. You are welcome to downvote me because of this, but if you do it means that the discussion norms of LessWrong have corroded. Other reasons for downvotes might be appropriate, such as the length.
How about we ask Geoff? I hereby ask Geoff if he’s a cult leader, or if he has any other comment.
I talked with Geoff recently, which some might see as evidence of a conspiracy.
Editing that summary to be much more concise:
Orthnonormal has had little contact with Geoff, but is and continues to engage in hostile gossip. I and others with more substantive contact do not believe he is a cult leader. The people orthonormal has talked with, alluded to by the conversations that have incurred orthonormal reputational costs, have had much more contact with Geoff. Despite all of this, orthonormal refuses to believe that Geoff is not a cult leader. I believe we should base the likelihood of Geoff being a cult leader on those who have had more contact with Geoff, or even based on Geoff’s words on their own.
I notice that as I am re-reading matt’s post, I expect that the potential reading of orthonormal’s that he presents at the beginning (a reading that I find uncharitable) is in fact matt’s reading. But he doesn’t actually say this outright. Instead he says “An available interpretation of orthonormal’s comment is...”. Indeed, I initially had an author’s note in the summary that reflected the point that I was unsure if “an available interpretation” was matt’s interpretation. It is only much later (inside a parenthetical) that he says “I want to note that while readers may react negatively to me characterising orthonormal’s behaviour as “hostile gossip”...” to indicate that the uncharitable reading is in fact Matt’s reading.
Matt’s comment also included some comments that I read as sneering:
I wonder whether orthonormal has other evidence, or whether orthonormal will take this opportunity to reduce their confidence in their first impression… or whether orthonormal will continue to be spectacularly confident that they’ve been right all along.
I would have preferred his comment to start small with some questions about orthonormal’s experience rather than immediately accuse them of hostile gossip. For instance, matt might have asked about the extent of orthonormal’s contact with Geoff, how confident orthonormal is that Geoff is a cult leader, and whether orthonormal updated against Geoff being a cult leader in light of their friends believing Geoff wasn’t a cult leader, etc. Instead, those questions are assumed to have answers that are unsupportive of orthonormal’s original point (the answers assumed in matt’s comment in order: very little contact, extremely confident, anti-updates in the direction of higher confidence). This seems like a central example of an uncharitable comment.
Overall I find matt’s comment difficult to understand after multiple readings and uncharitable of those he is conversing with, although I do value the data it adds to the conversation. I believe this lack of charity is part of why matt’s comment has not done well in terms of karma. I still have not voted on matt’s comment and do not believe I will. There are parts of it that are valuable, but it is uncharitable and that is a value I hold above most others. In cases like these, where parts of a comment are valuable and other parts are the sort of thing that I would rather pruned from the gardens I spend my time in, I tend to withhold judgment.
How do my two summaries compare? I’m surprised by how close the first summary I gave was to the “much more concise” summary I gave later. I expected to have missed more, largely due to matt’s comment’s length. I also remember finding it distasteful, which I omitted from my summaries but likely stemmed from the lack of charity extended to orthonormal.
Do other readers find my summary, particularly my more concise summary, an accurate portrayal of matt’s comment? How would they react to that much more concise comment, as compared to matt’s comment?
Strong upvote for doing this process/experiment; this is outstanding and I separately appreciate the effort required.
Do other readers find my summary, particularly my more concise summary, an accurate portrayal of matt’s comment? How would they react to that much more concise comment, as compared to matt’s comment?
I find your summary at least within-bounds, i.e. not fully ruled out by the words on the page. I obviously had a different impression, but I don’t think that it’s invalid to hold the interpretations and hypotheses that you do.
I particularly like and want to upvote the fact that you’re being clear and explicit about them being your interpretations and hypotheses; this is another LW-ish norm that is half-reliable and I would like to see fully reliable. Thanks for doing it.
When it comes to assessing whether a long comment or post is hard to read, quality and style of writing matters, too. SSC’s Nonfiction Writing Advice endlessly hammers home the point of dividing text into increasingly smaller chunks, and e.g. here’s one very long post by Eliezer that elicited multiple comments of the form “this was too boring to finish” (e.g. this one), some of which got alleviated merely by adding chapter breaks.
And since LW makes it trivial to add headings even to comments (e.g. I used headings here), I guess that’s one more criterion for me to judge long comments by.
(One could even imagine the LW site nudging long comments towards including stuff like headings. One could imagine a good version of a prompt like this: “This comment / post is >3k chars but consists of only 3 paragraphs and uses no headings. Consider adding some level of hierarchy, e.g. via headings.”)
(Something genuinely amusing, given the context, about the above being at 3 points out of 2 votes after four hours, compared to its parent being at 30 points out of 7 votes after five.)
I have an alternative and almost orthogonal interpretation for why the karma scores are the way they are.
Both in your orthonormal-Matt example, and now in this meta-example, the shorter original comments require less context to understand and got more upvotes, while the long meandering detail-oriented high-context responses were hardly even read by anyone.
This makes perfect sense to me—there’s a maximum comment length after which I get a strong urge to just ignore / skim a comment (which I initially did with your response here; and I never took the time to read Matt’s comments, though I also didn’t vote on orthonormal’s comment one way or another, nor vote in the jessicata post much at all), and I would be astonished if that only happened to me.
Also think about how people see these comments in the first place. Probably a significant chunk comes from people browsing the comment feed on the LW front page, and it makes perfect sense to scroll past a long sub-sub-sub-comment that might not even be relevant, and that you can’t understand without context, anyway.
So from my perspective, high-effort, high-context, lengthy sub-comments intrinsically incur a large attention / visibility (and therefore karma) penalty. Things like conciseness are also virtues, and if you don’t consider that it in your model of “good along three different axes, and bad along none as far as I can see”, then that model is incomplete.
(Also consider things like: How much time do you think the average reader spends on LW; what would be a good amount of time, relative to their other options; would you prefer a culture where hundreds of people take the opportunity cost to read sub-sub-sub-comments over one where they don’t; also people vary enormously in their reading speed; etc.)
Somewhat related: My post in this thread on some of the effects of the existing LW karma system: If we grant the above, one remaining problem is that the original orthonormal comment was highly upvoted but looked worse over time:
First, some off the cuff impressions of matt’s post (in the interest of data gathering):
In the initial thread I believe that I read the first paragraph of matt’s comment, decided I would not get much out of it, and stopped reading without voting.
Upon revisiting the thread and reading matt’s comment in full, I find it difficult to understand and do not believe I would be able to summarize or remember its main points now, about 15 minutes after the fact.
This seems somewhat interesting to test, so here is my summary from memory. After this I’ll reread matt’s post and compare what I thought it said upon first reading with what I think it says upon a second closer reading:
Commentary before re-reading: I expect that I missed a lot, since it was a long post and it did not stick in my mind particularly well. I also remember a lot of hedging that confused me, and points that went into parentheticals within parentheticals. These parentheticals were long enough that I remember losing track of what point was being made. I also may have confabulated arguments in this thread about upvotes and some from matt’s post.
I wanted to keep the summary “pure” in the sense that it is a genuine recollection without re-reading, but for clarity [person] is othonormal and [commenter to person] is RyanCarey.
Second attempt at summarizing while flipping back and forth between editor and matt’s comment:
Editing that summary to be much more concise:
I notice that as I am re-reading matt’s post, I expect that the potential reading of orthonormal’s that he presents at the beginning (a reading that I find uncharitable) is in fact matt’s reading. But he doesn’t actually say this outright. Instead he says “An available interpretation of orthonormal’s comment is...”. Indeed, I initially had an author’s note in the summary that reflected the point that I was unsure if “an available interpretation” was matt’s interpretation. It is only much later (inside a parenthetical) that he says “I want to note that while readers may react negatively to me characterising orthonormal’s behaviour as “hostile gossip”...” to indicate that the uncharitable reading is in fact Matt’s reading.
Matt’s comment also included some comments that I read as sneering:
I would have preferred his comment to start small with some questions about orthonormal’s experience rather than immediately accuse them of hostile gossip. For instance, matt might have asked about the extent of orthonormal’s contact with Geoff, how confident orthonormal is that Geoff is a cult leader, and whether orthonormal updated against Geoff being a cult leader in light of their friends believing Geoff wasn’t a cult leader, etc. Instead, those questions are assumed to have answers that are unsupportive of orthonormal’s original point (the answers assumed in matt’s comment in order: very little contact, extremely confident, anti-updates in the direction of higher confidence). This seems like a central example of an uncharitable comment.
Overall I find matt’s comment difficult to understand after multiple readings and uncharitable of those he is conversing with, although I do value the data it adds to the conversation. I believe this lack of charity is part of why matt’s comment has not done well in terms of karma. I still have not voted on matt’s comment and do not believe I will. There are parts of it that are valuable, but it is uncharitable and that is a value I hold above most others. In cases like these, where parts of a comment are valuable and other parts are the sort of thing that I would rather pruned from the gardens I spend my time in, I tend to withhold judgment.
How do my two summaries compare? I’m surprised by how close the first summary I gave was to the “much more concise” summary I gave later. I expected to have missed more, largely due to matt’s comment’s length. I also remember finding it distasteful, which I omitted from my summaries but likely stemmed from the lack of charity extended to orthonormal.
Do other readers find my summary, particularly my more concise summary, an accurate portrayal of matt’s comment? How would they react to that much more concise comment, as compared to matt’s comment?
Strong upvote for doing this process/experiment; this is outstanding and I separately appreciate the effort required.
I find your summary at least within-bounds, i.e. not fully ruled out by the words on the page. I obviously had a different impression, but I don’t think that it’s invalid to hold the interpretations and hypotheses that you do.
I particularly like and want to upvote the fact that you’re being clear and explicit about them being your interpretations and hypotheses; this is another LW-ish norm that is half-reliable and I would like to see fully reliable. Thanks for doing it.
To add one point:
When it comes to assessing whether a long comment or post is hard to read, quality and style of writing matters, too. SSC’s Nonfiction Writing Advice endlessly hammers home the point of dividing text into increasingly smaller chunks, and e.g. here’s one very long post by Eliezer that elicited multiple comments of the form “this was too boring to finish” (e.g. this one), some of which got alleviated merely by adding chapter breaks.
And since LW makes it trivial to add headings even to comments (e.g. I used headings here), I guess that’s one more criterion for me to judge long comments by.
(One could even imagine the LW site nudging long comments towards including stuff like headings. One could imagine a good version of a prompt like this: “This comment / post is >3k chars but consists of only 3 paragraphs and uses no headings. Consider adding some level of hierarchy, e.g. via headings.”)