(I’m still getting a handle on exactly what an A-grade tag should feel like though, honestly).
Me too. Maybe I’ll get a chance to write some of my own today or tomorrow and we can compare notes.
Probably the hard thing is that what counts as A-Class for one posts isn’t for enough for another. For Heuristics & Biases, man, it’s such a central tag for LW so it’s gotta be really good. Probably we should hold off declaring this one A-Class until we’re ready to say it really doesn’t merit more work, which could be a while.
One thought is that this a topic with some amazing posts introducing it and the tag description should lean on them, maybe even quoting them a lot and pushing them to read them. The current tag description is alright, but it feels like it doesn’t get at the heart of the topic in the few paragraphs or make me feel like I should care much. Contrast with the engagingness of …What’s a bias, again?
In Conservation of Expected Evidence, I basically thought to myself “there’s no way I’m gonna write a better explanation than Eliezer on this”, should just quote him. Though sometime’s Eliezer’s explanations are too long to quote and it makes sense to rewrite them.
Other thoughts: - I think posts in this class should heavily mention and describe top posts close to the top (including pretty related ones like “Your Intuitions Aren’t Magic”’). - Relatedly, it should guide your reading of the topic much more explicitly. What is Predictably Wrong? What does it contain? - I think it’d be good to have a more explicit list of different heuristics and biases. As kind of a parent tag, it should maybe even have a nice table of all “sub-tags” - The way “fallacies” is used on LW isn’t about explicit logic, really. I think it’s more that a fallacy is a bad inference/step of reasoning, whereas heuristics and biases are properties of the algorithm that does the reasoning.
[I also moved the Related Tags to the top because I think it’s good if someone can immediately see whether a related tag is what they’re after (even in the hover-over which you can hackily include if you shift-enter)]
I’ve updated the Heuristics and Biases tag again btw. I don’t think it’s A-grade based on “I’d like to see more work done on it”, but I think it’s about as good as I personally am going to be able to get it. I’d really like somebody (yes you, fellow user reading this) to have a read through and make any adjustments that make sense and/or make it more comprehensive.
re: fallacies, I thought about it, and I think they’re actually used pretty similarly, at least here on LW. Planning fallacy could easily be described as a bias generated by an ‘imagine your ideal plan going correctly (and maybe add, say, 10%)’ heuristic. At the very least, there’s plenty overlap. Really what I envisioned for that section was making the point that a heuristic can be good (or just ok), because that was something that I didn’t realise for a long time.
OK. So you see the grading as being more of a “neglected-o-meter” in the sense that it describes the gap between how a tag currently is and how it would be in an ideal world? (i.e. a more important tag would have a higher bar for being A-grade than a less important one?)
I think that makes more sense than an absolute-quality stamp, but I think the tag grading post as is currently written should make that clear (if it is the case)-- currently it implies almost the opposite, at least as I read it. For instance phrases like “It covers a valuable topic” in A-grade, and “tagged posts may not be especially good.” in C-grade. To me these read as “quality/importance of topic and of posts are as important for grading as description”.
I think actually the way you’re describing tags now is more useful (for e.g. directing peoples attention for improving tags), but I’m not sure if it came across that way (to me) in the initial post. I would be interested to hear how other people read it.
Yes, “neglected-o-meter” is a good way of putting it. The idea was a bit tricky to convey, I guess I didn’t have it super well-articulated in my own head.
The idea was that:
Tag grades identify tags in need of work.
For each grade, there’s a set of standard things to do to improve them. (This seemed better than individually marking tags as “needs more posts” or “needs better description”)
And also additionally that tags reflect absolute quality as well, such that if you only want the best tags, you can filter for that. I didn’t realize that what I’d consider A-Grade for an obscure topic with limited content would be different for a major topic where there’s lot to be said. Another difference is how fundamental and introductory a topic is, where topics that a person is early on in someone’s LW journey need extra polish.
Now that people are writing more tag descriptions, the gaps in the system are coming out. I’ve felt somewhat that I should give any tag that seems to meet the criteria a grade, but then in some cases there’s still more I’d want. This might be solved by making the criteria better and clearer.
I apologize for the confusion. We’re about to go on team retreat, but when we get back maybe taggers in this thread and the LW team can refine the system/schema.
Me too. Maybe I’ll get a chance to write some of my own today or tomorrow and we can compare notes.
Probably the hard thing is that what counts as A-Class for one posts isn’t for enough for another. For Heuristics & Biases, man, it’s such a central tag for LW so it’s gotta be really good. Probably we should hold off declaring this one A-Class until we’re ready to say it really doesn’t merit more work, which could be a while.
One thought is that this a topic with some amazing posts introducing it and the tag description should lean on them, maybe even quoting them a lot and pushing them to read them. The current tag description is alright, but it feels like it doesn’t get at the heart of the topic in the few paragraphs or make me feel like I should care much. Contrast with the engagingness of …What’s a bias, again?
In Conservation of Expected Evidence, I basically thought to myself “there’s no way I’m gonna write a better explanation than Eliezer on this”, should just quote him. Though sometime’s Eliezer’s explanations are too long to quote and it makes sense to rewrite them.
Other thoughts:
- I think posts in this class should heavily mention and describe top posts close to the top (including pretty related ones like “Your Intuitions Aren’t Magic”’).
- Relatedly, it should guide your reading of the topic much more explicitly. What is Predictably Wrong? What does it contain?
- I think it’d be good to have a more explicit list of different heuristics and biases. As kind of a parent tag, it should maybe even have a nice table of all “sub-tags”
- The way “fallacies” is used on LW isn’t about explicit logic, really. I think it’s more that a fallacy is a bad inference/step of reasoning, whereas heuristics and biases are properties of the algorithm that does the reasoning.
[I also moved the Related Tags to the top because I think it’s good if someone can immediately see whether a related tag is what they’re after (even in the hover-over which you can hackily include if you shift-enter)]
I’ve updated the Heuristics and Biases tag again btw. I don’t think it’s A-grade based on “I’d like to see more work done on it”, but I think it’s about as good as I personally am going to be able to get it. I’d really like somebody (yes you, fellow user reading this) to have a read through and make any adjustments that make sense and/or make it more comprehensive.
re: fallacies, I thought about it, and I think they’re actually used pretty similarly, at least here on LW. Planning fallacy could easily be described as a bias generated by an ‘imagine your ideal plan going correctly (and maybe add, say, 10%)’ heuristic. At the very least, there’s plenty overlap. Really what I envisioned for that section was making the point that a heuristic can be good (or just ok), because that was something that I didn’t realise for a long time.
OK. So you see the grading as being more of a “neglected-o-meter” in the sense that it describes the gap between how a tag currently is and how it would be in an ideal world? (i.e. a more important tag would have a higher bar for being A-grade than a less important one?)
I think that makes more sense than an absolute-quality stamp, but I think the tag grading post as is currently written should make that clear (if it is the case)-- currently it implies almost the opposite, at least as I read it. For instance phrases like “It covers a valuable topic” in A-grade, and “tagged posts may not be especially good.” in C-grade. To me these read as “quality/importance of topic and of posts are as important for grading as description”.
I think actually the way you’re describing tags now is more useful (for e.g. directing peoples attention for improving tags), but I’m not sure if it came across that way (to me) in the initial post. I would be interested to hear how other people read it.
Yes, “neglected-o-meter” is a good way of putting it. The idea was a bit tricky to convey, I guess I didn’t have it super well-articulated in my own head.
The idea was that:
Tag grades identify tags in need of work.
For each grade, there’s a set of standard things to do to improve them. (This seemed better than individually marking tags as “needs more posts” or “needs better description”)
And also additionally that tags reflect absolute quality as well, such that if you only want the best tags, you can filter for that. I didn’t realize that what I’d consider A-Grade for an obscure topic with limited content would be different for a major topic where there’s lot to be said. Another difference is how fundamental and introductory a topic is, where topics that a person is early on in someone’s LW journey need extra polish.
Now that people are writing more tag descriptions, the gaps in the system are coming out. I’ve felt somewhat that I should give any tag that seems to meet the criteria a grade, but then in some cases there’s still more I’d want. This might be solved by making the criteria better and clearer.
I apologize for the confusion. We’re about to go on team retreat, but when we get back maybe taggers in this thread and the LW team can refine the system/schema.
Thanks for your patience.