It is disappointing/confusing to me that of the two articles I recently wrote, the one that was much closer to reality got a lot less karma.
A new process for mapping discussions is a summary of months of work that I and my team did on mapping discourse around AI. We built new tools, employed new methodologies. It got 19 karma
Advice for journalists is a piece that I wrote in about 5 hours after perhaps 5 hours of experiences. It has 73 karma and counting
I think this is isn’t much evidence, given it’s just two pieces. But I do feel a pull towards coming up with theories rather than building and testing things in the real world. To the extent this pull is real, it seems bad.
If true, I would recommend both that more people build things in the real world and talk about them and that we find ways to reward these posts more, regardless of how alive they feel to us at the time.
(Aliveness being my hypothesis—many of us understand or have more live feelings about dealing with journalists than a sort of dry post about mapping discourse)
Your post on journalists is, as I suspected, a lot better.
I bounced off the post about mapping discussions because the post didn’t make clear what potential it might have to be useful to me. The post on journalists, which drew me in and quickly made clear what its use would be: informing me of how journalists use conversations with them.
The implied claims that theory should be worth less than building or that time on task equals usefulness are both wrong. We are collectively very confused, so running around building stuff before getting less confused isn’t always the best use of time.
Yeah but the mapping post is about 100x more important/well informed also. Shouldn’t that count for something? I’m not saying it’s clearer, I’m saying that it’s higher priority, probably.
As a reaction to this shortform I looked at the mapping post and immediately bounced off.
Echoing Seth I didnt understand the point.
The journalist post, by contrast, resonated strongly with me. Obviously it’s red meat for the commentariat here.
It made me feel like (1) other people care about this problem in a real, substantive, and potentially constructive way ; (2) there might be ways to make this genui ely better and this is potentially a very high impact lever.
You may think the post is far more important and well informed, but if it isn’t sufficiently clear, then maybe that didn’t come across to your audience.
I was relating pretty similar experiences here, albeit with more of a “lol post karma is stupid amiright?” vibe than a “this is a problem we should fix” vibe.
Your journalist thing is an easy-to-read blog post that strongly meshes with a popular rationalist tribal belief (i.e., “boo journalists”). Obviously those kinds of posts are going to get lots and lots of upvotes, and thus posts that lack those properties can get displaced from the frontpage. I have no idea what can be done about that problem, except that we can all individually try to be mindful about which posts we’re reading and upvoting (or strong-upvoting) and why.
(This is a pro tanto argument for having separate karma-voting and approval-voting for posts, I think.)
And simultaneously, we writers should try not to treat the lesswrong frontpage as our one and only plan for getting what we write in front of our target audience. (E.g. you yourself have other distribution channels.)
I haven’t read either post, but maybe this problem reduces partly to more technical posts getting less views, and thus less karma? One problem with even great technical posts is that very few readers can evaluate that such a post is indeed great. And if you can’t tell whether a post is accurate, then it can feel irresponsible to upvote it. Even if your technical post is indeed great, it’s not clear that a policy of “upvote all technical posts I can’t judge myself” would make great technical posts win in terms of karma.
A second issue I’m just noticing is that the first post contains lots of text-heavy screenshots, and that has a bunch of downsides for engagement. Like, the blue font in the first screenshot is very small and thus hard to read. I read stuff in a read-it-later app (called Readwise Reader), incl. with text-to-speech, and neither the app nor the TTS work great with such images. Also, such images usually don’t respect dark mode on either LW or other apps. You can’t use LW’s inline quote comments. And so on and so forth. Screenshots work better in a presentation, but not particularly well in a full essay.
Another potential issue is that the first post doesn’t end on “Tell me what you think” (= invites discussion and engagement), but rather with a Thanks section (does anyone ever read those?) and then a huge skimmable section of Full screenshots.
I’m also noticing that the LW version of the first post is lacking the footnotes from the Substack version.
EDIT: And the title for the second post seems way better. Before clicking on either post, I have an idea what the second one is about, and none whatsoever what the first one is about. So why would I even click on the latter? Would the readers you’re trying to reach with that post even know what you mean by “mapping discussions”?
EDIT2: And when I hear “process”, I think of mandated employee trainings, not of software solutions, so the title is misleading to me, too. Even “A new website for mapping discussions” or “We built a website for mapping discussions” would already sound more accurate and interesting to me, though I still wouldn’t know what the “mapping discussions” part is about.
Well, karma is not a perfect tool. It is good at keeping good stuff above zero and bad stuff below zero, by distributed effort. It is not good at quantifying how good or how bad the stuff is.
It is a good tool for removing homeopathy and ugly kittens. Without it, we would probably have more of those. So despite all the disadvantages, I want the karma system to stay. Until perhaps we invent something better.
I think we currently don’t have a formal tool for measuring “important” as a thing separate from “interesting” or “pleasant to read”. The best you can get is someone quoting you approvingly.
Tangential, but my immediate reaction to your example was “ugly kitten? All kittens are cute!”, so I searched specifically for “ugly kitten” on Google and it turns out that you were right! There are a lot of ugly kittens even though I never saw them! This probably says something about society..
Often, you write something short that ends up being valuable. That doesn’t mean you should despair about your longer and harder work being less valuable. Like if you could spend 40 hours a week writing quick 5-hour posts that are as well-received as the one you wrote, that would be amazing, but I don’t think anyone can do that because the circumstances have to line up just right, and you can’t count on that happening. So you have to spend most of your time doing harder and predictably-less-impactful work.
(I just left some feedback for the mapping discussion post on the post itself.)
Advice for journalists was a bit more polemic which I think naturally leads to more engagement. But I’d like to say that I strongly upvoted the mapping discussions post and played around with the site quite a bit when it was first posted—it’s really valuable to me.
Karma’s a bit of a blunt tool—yes I think it’s good to have posts with broad appeal but some posts are going to be comparatively more useful to a smaller group of people, and that’s OK too.
I think we should put less faith in the karma system on this site as a ranking system. I agree ranking systems are good to have, but I think short-term upvotes by readers with feedback winner-takes-most mechanisms is inherently ill-suited to this.
If we want a better ranking system, I think we’d need something like a set of voting options at the bottom of the post like:
does this seem to have enduring value?
is this a technical post which reports on a substantial amount of work done?
would I recommend that a researcher in the field this post is in familiarize themself with this post?
and then also a score based on citations, as is done with academia.
Karma, as it stands, is something more like someone wandering by a poster and saying ‘nice!’. LessWrong is a weird inbetween zone with aspects of both social media and academic publishing.
Separate feedback / food for thought: You mention that your post on mapping discussions is a summary of months of work, and that the second post took 5h to write and received far more karma. But did you spend at least 5h on writing the first one, too?
I see. Then I’ll point to my feedback in the other comment and say that the journalism post was likely better written despite your lower time investment. And that if you spend a lot of time on a post, I recommend spending more of that time on the title in particular, because of the outsized importance of a) getting people to click on your thing and b) having people understand what the thing you’re asking them to click on even is. Here are two long comments on this topic.
Separately, when it comes to the success of stuff like blog posts, I like the framing in Ben Kuhn’s post Searching for outliers, about the implications for activities (like blogging) whose impacts are dominated by heavy-tailed outcomes.
Why does it confuse you? The attention something gets doesn’t depend strongly on its quality, but on how accessible it is.
If I get a lot of karma/upvotes/thumbs/hearts/whatever online, then I feel bad, because I would have written something poor. My best comments are usually ignored, with the occasional reply from somebody who misunderstands me entirely, and the even more rare even that somebody understands me (this type is usually so aligned with what I wrote that they have nothing to add).
The nature of the normal distribution makes it so that popularity and quality never correlate very strongly. This is discouraging to people who do their best in some field with the hope that they will be recognized for it. I’ve seen many artists troubled by this as well, everything they consider a “masterpiece” is somewhat obscure, while most popular things go against their taste. An example that many can agree with is probably pop music, but I don’t think any examples exists which more than 50% of people agree with, because then said example wouldn’t exist in the first place.
This is far too cynical. Great writers (e.g. gwern, Scott Alexander, Matt Yglesias) can write excellent, technical posts and comments while still getting plenty attention.
Gwern and Scott are great writers, which is different from writing great things. It’s like high-purity silver rather than rough gold, if that makes sense.
I do think they write a lot of great things, but not excellent things. Posts like “Maybe Your Zoloft Stopped Working Because A Liver Fluke Tried To Turn Your Nth-Great-Grandmother Into A Zombie” are probably around the limit of how difficult of an idea somebody can communicate while retaining some level of popularity. Somebody wanting to communicate ideas one or two standard deviations about this would find themselves in obscurity. I think there’s more intelligent people out there sharing ideas which don’t really reach anyone. Of course, it’s hard for me to provide examples, as obscure things are hard to find, and I won’t be able to prove that said ideas are good, for if it was easy to recognize as such, then they’d already be popular. And once you get abstract enough, the things you say will basically be indistinguishable from nonsense to anyone below a certain threshold of intelligence.
Of course, it may just be that high levels of abstraction aren’t useful, leading intelligent people towards width and expertise with the mundane, rather than rabbit holes. Or it may be that people give up attempting to communicate certain concepts in language, and just make the attempt at showing them instead.
I saw a biologist on here comparing people to fire (as chemical processes) and immediately found the idea familiar as I had made the same connection myself before. To most people, it probably seems like a weird idea?
The idea that popularity must be a sign of shallowness, and hence unpopularity or obscurity a sign of depth, sounds rather shallow to me. My attitude here is more like, if supposedly world-shattering insights can’t be explained in relatively simple language, they either aren’t that great, or we don’t really understand them. Like in this Feynman quote:
Once I asked him to explain to me, so that I can understand it, why spin-1/2 particles obey Fermi-Dirac statistics. Gauging his audience perfectly, he said, “I’ll prepare a freshman lecture on it.” But a few days later he came to me and said: “You know, I couldn’t do it. I couldn’t reduce it to the freshman level. That means we really don’t understand it.”
I think it’s necessarily truth given the statistical distribution of things. If I say “There’s necessarily less people with PhDs than with masters, and necessarily less masters than college graduates” you’d probably agree.
The theory that “If you understand something, you can explain it simply” is mostly true, but this does not make it easy to understand, as simplicity is not ease (Just try to explain enlightenment / the map-territory distinction to a stupid person). What you understand will seem trivial to you, and what you don’t understand will seem difficult. This is just the mental representation of things getting more efficienct and us building mental shortcuts for things and getting used to patterns.
Proof: There’s people who understand high level mathematics, so they must be able to explain these concepts simply. In theory, they should be able to write a book of these simple concepts, which even 4th graders can read. Thus, we should already have plenty of 4th graders who understand high level mathematics. But this is not the case, most 4th graders are still 10 years of education away from understanding things on a high level. Ergo, either the initial claim (that what you understand can be explained simply) is false, or else “explained simply” does not imply “understood easily”
The excessive humility is a kind og signaling or defense mechanism against criticism and excessive expectations from other people, and it’s rewarded because of its moralistic nature. It’s not true, it’s mainly pleasant-sounding nonsense originating in herd morality.
It is disappointing/confusing to me that of the two articles I recently wrote, the one that was much closer to reality got a lot less karma.
A new process for mapping discussions is a summary of months of work that I and my team did on mapping discourse around AI. We built new tools, employed new methodologies. It got 19 karma
Advice for journalists is a piece that I wrote in about 5 hours after perhaps 5 hours of experiences. It has 73 karma and counting
I think this is isn’t much evidence, given it’s just two pieces. But I do feel a pull towards coming up with theories rather than building and testing things in the real world. To the extent this pull is real, it seems bad.
If true, I would recommend both that more people build things in the real world and talk about them and that we find ways to reward these posts more, regardless of how alive they feel to us at the time.
(Aliveness being my hypothesis—many of us understand or have more live feelings about dealing with journalists than a sort of dry post about mapping discourse)
Your post on journalists is, as I suspected, a lot better.
I bounced off the post about mapping discussions because the post didn’t make clear what potential it might have to be useful to me. The post on journalists, which drew me in and quickly made clear what its use would be: informing me of how journalists use conversations with them.
The implied claims that theory should be worth less than building or that time on task equals usefulness are both wrong. We are collectively very confused, so running around building stuff before getting less confused isn’t always the best use of time.
Yeah but the mapping post is about 100x more important/well informed also. Shouldn’t that count for something? I’m not saying it’s clearer, I’m saying that it’s higher priority, probably.
Why is it more important ?
As a reaction to this shortform I looked at the mapping post and immediately bounced off. Echoing Seth I didnt understand the point.
The journalist post, by contrast, resonated strongly with me. Obviously it’s red meat for the commentariat here. It made me feel like (1) other people care about this problem in a real, substantive, and potentially constructive way ; (2) there might be ways to make this genui ely better and this is potentially a very high impact lever.
You may think the post is far more important and well informed, but if it isn’t sufficiently clear, then maybe that didn’t come across to your audience.
I was relating pretty similar experiences here, albeit with more of a “lol post karma is stupid amiright?” vibe than a “this is a problem we should fix” vibe.
Your journalist thing is an easy-to-read blog post that strongly meshes with a popular rationalist tribal belief (i.e., “boo journalists”). Obviously those kinds of posts are going to get lots and lots of upvotes, and thus posts that lack those properties can get displaced from the frontpage. I have no idea what can be done about that problem, except that we can all individually try to be mindful about which posts we’re reading and upvoting (or strong-upvoting) and why.
(This is a pro tanto argument for having separate karma-voting and approval-voting for posts, I think.)
And simultaneously, we writers should try not to treat the lesswrong frontpage as our one and only plan for getting what we write in front of our target audience. (E.g. you yourself have other distribution channels.)
I haven’t read either post, but maybe this problem reduces partly to more technical posts getting less views, and thus less karma? One problem with even great technical posts is that very few readers can evaluate that such a post is indeed great. And if you can’t tell whether a post is accurate, then it can feel irresponsible to upvote it. Even if your technical post is indeed great, it’s not clear that a policy of “upvote all technical posts I can’t judge myself” would make great technical posts win in terms of karma.
A second issue I’m just noticing is that the first post contains lots of text-heavy screenshots, and that has a bunch of downsides for engagement. Like, the blue font in the first screenshot is very small and thus hard to read. I read stuff in a read-it-later app (called Readwise Reader), incl. with text-to-speech, and neither the app nor the TTS work great with such images. Also, such images usually don’t respect dark mode on either LW or other apps. You can’t use LW’s inline quote comments. And so on and so forth. Screenshots work better in a presentation, but not particularly well in a full essay.
Another potential issue is that the first post doesn’t end on “Tell me what you think” (= invites discussion and engagement), but rather with a Thanks section (does anyone ever read those?) and then a huge skimmable section of Full screenshots.
I’m also noticing that the LW version of the first post is lacking the footnotes from the Substack version.
EDIT: And the title for the second post seems way better. Before clicking on either post, I have an idea what the second one is about, and none whatsoever what the first one is about. So why would I even click on the latter? Would the readers you’re trying to reach with that post even know what you mean by “mapping discussions”?
EDIT2: And when I hear “process”, I think of mandated employee trainings, not of software solutions, so the title is misleading to me, too. Even “A new website for mapping discussions” or “We built a website for mapping discussions” would already sound more accurate and interesting to me, though I still wouldn’t know what the “mapping discussions” part is about.
Well, karma is not a perfect tool. It is good at keeping good stuff above zero and bad stuff below zero, by distributed effort. It is not good at quantifying how good or how bad the stuff is.
Solving alignment = positive karma. Cute kitten = positive karma. Ugly kitten = negative karma. Promoting homeopathy = negative karma.
It is a good tool for removing homeopathy and ugly kittens. Without it, we would probably have more of those. So despite all the disadvantages, I want the karma system to stay. Until perhaps we invent something better.
I think we currently don’t have a formal tool for measuring “important” as a thing separate from “interesting” or “pleasant to read”. The best you can get is someone quoting you approvingly.
Tangential, but my immediate reaction to your example was “ugly kitten? All kittens are cute!”, so I searched specifically for “ugly kitten” on Google and it turns out that you were right! There are a lot of ugly kittens even though I never saw them! This probably says something about society..
Fwiw I loved your journalist post and I never even saw your other post (until now).
Often, you write something short that ends up being valuable. That doesn’t mean you should despair about your longer and harder work being less valuable. Like if you could spend 40 hours a week writing quick 5-hour posts that are as well-received as the one you wrote, that would be amazing, but I don’t think anyone can do that because the circumstances have to line up just right, and you can’t count on that happening. So you have to spend most of your time doing harder and predictably-less-impactful work.
(I just left some feedback for the mapping discussion post on the post itself.)
Advice for journalists was a bit more polemic which I think naturally leads to more engagement. But I’d like to say that I strongly upvoted the mapping discussions post and played around with the site quite a bit when it was first posted—it’s really valuable to me.
Karma’s a bit of a blunt tool—yes I think it’s good to have posts with broad appeal but some posts are going to be comparatively more useful to a smaller group of people, and that’s OK too.
Sure but shouldn’t the karma system be a prioritisation ranking, not just “what is fun to read?”
I think we should put less faith in the karma system on this site as a ranking system. I agree ranking systems are good to have, but I think short-term upvotes by readers with feedback winner-takes-most mechanisms is inherently ill-suited to this.
If we want a better ranking system, I think we’d need something like a set of voting options at the bottom of the post like:
does this seem to have enduring value?
is this a technical post which reports on a substantial amount of work done?
would I recommend that a researcher in the field this post is in familiarize themself with this post?
and then also a score based on citations, as is done with academia.
Karma, as it stands, is something more like someone wandering by a poster and saying ‘nice!’. LessWrong is a weird inbetween zone with aspects of both social media and academic publishing.
But isn’t the point of karma to be a ranking system? Surely its bad if it’s a suboptimal one?
Separate feedback / food for thought: You mention that your post on mapping discussions is a summary of months of work, and that the second post took 5h to write and received far more karma. But did you spend at least 5h on writing the first one, too?
I would say I took at least 10 hours to write it. I rewrote it about 4 times.
I see. Then I’ll point to my feedback in the other comment and say that the journalism post was likely better written despite your lower time investment. And that if you spend a lot of time on a post, I recommend spending more of that time on the title in particular, because of the outsized importance of a) getting people to click on your thing and b) having people understand what the thing you’re asking them to click on even is. Here are two long comments on this topic.
Separately, when it comes to the success of stuff like blog posts, I like the framing in Ben Kuhn’s post Searching for outliers, about the implications for activities (like blogging) whose impacts are dominated by heavy-tailed outcomes.
Why does it confuse you? The attention something gets doesn’t depend strongly on its quality, but on how accessible it is.
If I get a lot of karma/upvotes/thumbs/hearts/whatever online, then I feel bad, because I would have written something poor.
My best comments are usually ignored, with the occasional reply from somebody who misunderstands me entirely, and the even more rare even that somebody understands me (this type is usually so aligned with what I wrote that they have nothing to add).
The nature of the normal distribution makes it so that popularity and quality never correlate very strongly. This is discouraging to people who do their best in some field with the hope that they will be recognized for it. I’ve seen many artists troubled by this as well, everything they consider a “masterpiece” is somewhat obscure, while most popular things go against their taste. An example that many can agree with is probably pop music, but I don’t think any examples exists which more than 50% of people agree with, because then said example wouldn’t exist in the first place.
This is far too cynical. Great writers (e.g. gwern, Scott Alexander, Matt Yglesias) can write excellent, technical posts and comments while still getting plenty attention.
Gwern and Scott are great writers, which is different from writing great things. It’s like high-purity silver rather than rough gold, if that makes sense.
I do think they write a lot of great things, but not excellent things. Posts like “Maybe Your Zoloft Stopped Working Because A Liver Fluke Tried To Turn Your Nth-Great-Grandmother Into A Zombie” are probably around the limit of how difficult of an idea somebody can communicate while retaining some level of popularity. Somebody wanting to communicate ideas one or two standard deviations about this would find themselves in obscurity. I think there’s more intelligent people out there sharing ideas which don’t really reach anyone. Of course, it’s hard for me to provide examples, as obscure things are hard to find, and I won’t be able to prove that said ideas are good, for if it was easy to recognize as such, then they’d already be popular. And once you get abstract enough, the things you say will basically be indistinguishable from nonsense to anyone below a certain threshold of intelligence.
Of course, it may just be that high levels of abstraction aren’t useful, leading intelligent people towards width and expertise with the mundane, rather than rabbit holes. Or it may be that people give up attempting to communicate certain concepts in language, and just make the attempt at showing them instead.
I saw a biologist on here comparing people to fire (as chemical processes) and immediately found the idea familiar as I had made the same connection myself before. To most people, it probably seems like a weird idea?
The idea that popularity must be a sign of shallowness, and hence unpopularity or obscurity a sign of depth, sounds rather shallow to me. My attitude here is more like, if supposedly world-shattering insights can’t be explained in relatively simple language, they either aren’t that great, or we don’t really understand them. Like in this Feynman quote:
I think it’s necessarily truth given the statistical distribution of things. If I say “There’s necessarily less people with PhDs than with masters, and necessarily less masters than college graduates” you’d probably agree.
The theory that “If you understand something, you can explain it simply” is mostly true, but this does not make it easy to understand, as simplicity is not ease (Just try to explain enlightenment / the map-territory distinction to a stupid person). What you understand will seem trivial to you, and what you don’t understand will seem difficult. This is just the mental representation of things getting more efficienct and us building mental shortcuts for things and getting used to patterns.
Proof: There’s people who understand high level mathematics, so they must be able to explain these concepts simply. In theory, they should be able to write a book of these simple concepts, which even 4th graders can read. Thus, we should already have plenty of 4th graders who understand high level mathematics. But this is not the case, most 4th graders are still 10 years of education away from understanding things on a high level. Ergo, either the initial claim (that what you understand can be explained simply) is false, or else “explained simply” does not imply “understood easily”
The excessive humility is a kind og signaling or defense mechanism against criticism and excessive expectations from other people, and it’s rewarded because of its moralistic nature. It’s not true, it’s mainly pleasant-sounding nonsense originating in herd morality.