A major goal I had for the LessWrong Review was to be “the intermediate metric that let me know if LW was accomplishing important things”, which helped me steer.
I think it hasn’t super succeeded at this.
I think one problem is that it just… feels like it generates stuff people liked reading, which is different from “stuff that turned out to be genuinely important.”
I’m now wondering “what if I built a power-tool that is designed for a single user to decide which posts seem to have mattered the most (according to them), and, then, figure out which intermediate posts played into them.” What would the lightweight version of that look like?
Another thing is, like, I want to see what particular other individuals thought mattered, as opposed to a generate aggregate that doesn’t any theory underlying it. Making the voting public veers towards some kind of “what did the cool people think?” contest, so I feel anxious about that, but, I do think the info is just pretty useful. But like, what if the output of the review is a series of individual takes on what-mattered-and-why, collectively, rather than an aggregate vote?
So Alasdair MacIntyre, says that all enquiry into truth and practical rationality takes place within a tradition, sometimes capital-t Tradition, that provides standards for things like “What is a good argument” and “What things can I take for granted” and so on. You never zoom all the way back to simple self-evident truths or raw-sense data—it’s just too far to go. (I don’t know if I’d actually recommend MacIntyre to you, he’s probably not sufficiently dense / interesting for your projects, he’s like a weird blend of Aquinas and Kuhn and Lakatos, but he is interesting at least, if you have a tolerance for.… the kind of thing he is.)
What struck me with a fair number of reviews, at this point, was that they seemed… kinda resigned to a LW Tradition, if it ever existed, no longer really being a single thing? Like we don’t have shared standards any more for what is a good argument or what things can be taken for granted (maybe we never did, and I’m golden-age fallacying). There were some reviews saying “idk if this is true, but it did influence people” and others being like “well I think this is kinda dumb, but seems important” and I know I wrote one being like “well these are at least pretty representative arguments of the kind of things people say to each other in these contexts.”
Anyhow what I’m saying is that—if we operate in a MacIntyrean frame—it makes sense to be like “this is the best work we have” within a Tradition, but humans start to spit out NaNs / operation not defined if you try to ask them “is this the best work we have” across Traditions. I don’t know if this is true of ideal reasoners but it does seem to be true of… um, any reasoners we’ve ever seen, which is more relevant.
Skimming the review posts for 2022, I think about 5⁄50 taught me something reasonably substantial and useful. I think another 10⁄50 provide a useful short idea and a label/pointer for that idea, but don’t really provide a large valuable lesson. Perhaps 20⁄50 are posts I might end up refering to at some point or recommending someone read.
Overall, I think I tend to learn way more in person talking to people than from LW posts, but I think LW posts are useful to reference reasonably often.
Those numbers sound reasonable to me (i.e. I might give similar numbers, although I’d probably list different posts than you)
Another angle I’ve had here: in my preferred world, the “Best of LessWrong” page leaves explicit that, in some sense, very few (possibly zero?) posts actually meet the bar we’d ideally aspire to. The Best of LessWrong page highlights the best stuff so far, but I think it’d be cool if there was a deliberately empty, aspirational section.
But, then I feel a bit stuck on “what counts for that tier?”
Here’s another idea:
Open Problems
(and: when voting on Best of LessWrong, you can ‘bet’ that a post will contribute to solving an Open Problem)
Open Problems could be a LessWrong feature which is basically a post describing an important, unsolved problem. They’d each be owned by a particular author or small group, who get to declare when they consider the problem “solved.” (If you want people to trust/care about the outcome of particular Open Problem, you might choose two co-owners who are sort of adversarial collaborators, and they have to both agree it was solved)
Two use-cases for Open Problems could be:
As a research target for an individual researcher (or team), i.e. setting the target they’re ultimately aiming for.
As a sort of X-Prize, for others to attempt to contribute to.
So we’d end up with problem statements like:
“AI Alignment for superintelligences is solved” (maybe Eliezer and Paul cosign a problem statement on that)
You (Ryan) and Buck could formulate some kind of Open Problem on AI Control
I’d like to be some kind of “we have a rationality training program that seems to demonstrably work”
And then there’s a page that highlights “these are the open problems people on LessWrong have upvoted the most as ‘important’”, and “here are the posts that people are betting will turn out to be relevant to the final solution.” (maybe this is operationalized as, like, a manifold market bet about whether the problem-author will eventually declare a given post to be an important contribution)
I don’t think that a solution to open problems being posted on LW would indicate that LW (the website and org, not the surrounding community) was accomplishing something useful.
E.g., imagine using the same metric for arXiv. (This case is more extreme, but I think it corresponds somewhat.)
Awkwardly, I think the existence of good posts is unlikely to track LW’s contribution. This seems especially true for posts about solutions to technical problems. The marginal contribution of LW is more in making it more likely that better posts are read and in making various conversations happen (with a variety of other diffuse potential advantages).
I think (but am unsure) that what I care about is more like a metric for “is useful intellectual progress getting made” (whether or not LessWrong-the-website was causal in that progress).
The point here is not to evaluate the Lightcone team’s work, but for the community to have a better benchmark for it’s collective progress (which then hopefully, like, improves credit-assignment which then hopefully improves our ability to collectively focus on useful stuff as the community scales)
This point does seem interesting though and maybe a different frame than I had previously been thinking in:
The marginal contribution of LW is more in making it more likely that better posts are read and in making various conversations happen (with a variety of other diffuse potential advantages).
I think (but am unsure) that what I care about is more like a metric for “is useful intellectual progress getting made” (whether or not LessWrong-the-website was causal in that progress).
Seems reasonable. From my perspective LW review is very bad for measuring overall (human) progress on achieving good things, though plausibly better than any other specific review or ranking process that has a considerable amount of buy in.
Do you think replacing (or at least combining) LW Review with the Open Problems frame would be an improvement on that axis?
Also: does it seem useful to you to measure overall progress on [the cluster of good things that the rationality and/or alignment community are pointed at?]?
Do you think replacing (or at least combining) LW Review with the Open Problems frame would be an improvement on that axis?
Uh, maybe for combining? I think my main complaint with LW review as a metric is more just that I disagree with the preferences of other people and think that a bunch of work is happening on places other than LW. I don’t really think Open Problems helps much with this from my perspective. (In many cases I can’t name a clear and operationalized open problem and more just think “more progress here would be good.)
A major goal I had for the LessWrong Review was to be “the intermediate metric that let me know if LW was accomplishing important things”, which helped me steer.
I think it hasn’t super succeeded at this.
I think one problem is that it just… feels like it generates stuff people liked reading, which is different from “stuff that turned out to be genuinely important.”
I’m now wondering “what if I built a power-tool that is designed for a single user to decide which posts seem to have mattered the most (according to them), and, then, figure out which intermediate posts played into them.” What would the lightweight version of that look like?
Another thing is, like, I want to see what particular other individuals thought mattered, as opposed to a generate aggregate that doesn’t any theory underlying it. Making the voting public veers towards some kind of “what did the cool people think?” contest, so I feel anxious about that, but, I do think the info is just pretty useful. But like, what if the output of the review is a series of individual takes on what-mattered-and-why, collectively, rather than an aggregate vote?
So Alasdair MacIntyre, says that all enquiry into truth and practical rationality takes place within a tradition, sometimes capital-t Tradition, that provides standards for things like “What is a good argument” and “What things can I take for granted” and so on. You never zoom all the way back to simple self-evident truths or raw-sense data—it’s just too far to go. (I don’t know if I’d actually recommend MacIntyre to you, he’s probably not sufficiently dense / interesting for your projects, he’s like a weird blend of Aquinas and Kuhn and Lakatos, but he is interesting at least, if you have a tolerance for.… the kind of thing he is.)
What struck me with a fair number of reviews, at this point, was that they seemed… kinda resigned to a LW Tradition, if it ever existed, no longer really being a single thing? Like we don’t have shared standards any more for what is a good argument or what things can be taken for granted (maybe we never did, and I’m golden-age fallacying). There were some reviews saying “idk if this is true, but it did influence people” and others being like “well I think this is kinda dumb, but seems important” and I know I wrote one being like “well these are at least pretty representative arguments of the kind of things people say to each other in these contexts.”
Anyhow what I’m saying is that—if we operate in a MacIntyrean frame—it makes sense to be like “this is the best work we have” within a Tradition, but humans start to spit out NaNs / operation not defined if you try to ask them “is this the best work we have” across Traditions. I don’t know if this is true of ideal reasoners but it does seem to be true of… um, any reasoners we’ve ever seen, which is more relevant.
I wonder if dramatically shrinking the review’s winners’ circle would help? Right now it feels huge to me.
What do you mean by winner’s circle? Like top 10 instead of top 50, or something else?
yeah, top 10 or even just top 5.
Skimming the review posts for 2022, I think about 5⁄50 taught me something reasonably substantial and useful. I think another 10⁄50 provide a useful short idea and a label/pointer for that idea, but don’t really provide a large valuable lesson. Perhaps 20⁄50 are posts I might end up refering to at some point or recommending someone read.
Overall, I think I tend to learn way more in person talking to people than from LW posts, but I think LW posts are useful to reference reasonably often.
Those numbers sound reasonable to me (i.e. I might give similar numbers, although I’d probably list different posts than you)
Another angle I’ve had here: in my preferred world, the “Best of LessWrong” page leaves explicit that, in some sense, very few (possibly zero?) posts actually meet the bar we’d ideally aspire to. The Best of LessWrong page highlights the best stuff so far, but I think it’d be cool if there was a deliberately empty, aspirational section.
But, then I feel a bit stuck on “what counts for that tier?”
Here’s another idea:
Open Problems
(and: when voting on Best of LessWrong, you can ‘bet’ that a post will contribute to solving an Open Problem)
Open Problems could be a LessWrong feature which is basically a post describing an important, unsolved problem. They’d each be owned by a particular author or small group, who get to declare when they consider the problem “solved.” (If you want people to trust/care about the outcome of particular Open Problem, you might choose two co-owners who are sort of adversarial collaborators, and they have to both agree it was solved)
Two use-cases for Open Problems could be:
As a research target for an individual researcher (or team), i.e. setting the target they’re ultimately aiming for.
As a sort of X-Prize, for others to attempt to contribute to.
So we’d end up with problem statements like:
“AI Alignment for superintelligences is solved” (maybe Eliezer and Paul cosign a problem statement on that)
You (Ryan) and Buck could formulate some kind of Open Problem on AI Control
I’d like to be some kind of “we have a rationality training program that seems to demonstrably work”
And then there’s a page that highlights “these are the open problems people on LessWrong have upvoted the most as ‘important’”, and “here are the posts that people are betting will turn out to be relevant to the final solution.” (maybe this is operationalized as, like, a manifold market bet about whether the problem-author will eventually declare a given post to be an important contribution)
I don’t think that a solution to open problems being posted on LW would indicate that LW (the website and org, not the surrounding community) was accomplishing something useful.
E.g., imagine using the same metric for arXiv. (This case is more extreme, but I think it corresponds somewhat.)
Awkwardly, I think the existence of good posts is unlikely to track LW’s contribution. This seems especially true for posts about solutions to technical problems. The marginal contribution of LW is more in making it more likely that better posts are read and in making various conversations happen (with a variety of other diffuse potential advantages).
I don’t know what a good metric for LW is.
I’m not 100% sure I got your point.
I think (but am unsure) that what I care about is more like a metric for “is useful intellectual progress getting made” (whether or not LessWrong-the-website was causal in that progress).
The point here is not to evaluate the Lightcone team’s work, but for the community to have a better benchmark for it’s collective progress (which then hopefully, like, improves credit-assignment which then hopefully improves our ability to collectively focus on useful stuff as the community scales)
This point does seem interesting though and maybe a different frame than I had previously been thinking in:
Seems reasonable. From my perspective LW review is very bad for measuring overall (human) progress on achieving good things, though plausibly better than any other specific review or ranking process that has a considerable amount of buy in.
I wasn’t quite sure from your phrasings:
Do you think replacing (or at least combining) LW Review with the Open Problems frame would be an improvement on that axis?
Also: does it seem useful to you to measure overall progress on [the cluster of good things that the rationality and/or alignment community are pointed at?]?
Uh, maybe for combining? I think my main complaint with LW review as a metric is more just that I disagree with the preferences of other people and think that a bunch of work is happening on places other than LW. I don’t really think Open Problems helps much with this from my perspective. (In many cases I can’t name a clear and operationalized open problem and more just think “more progress here would be good.)