I don’t think that a solution to open problems being posted on LW would indicate that LW (the website and org, not the surrounding community) was accomplishing something useful.
E.g., imagine using the same metric for arXiv. (This case is more extreme, but I think it corresponds somewhat.)
Awkwardly, I think the existence of good posts is unlikely to track LW’s contribution. This seems especially true for posts about solutions to technical problems. The marginal contribution of LW is more in making it more likely that better posts are read and in making various conversations happen (with a variety of other diffuse potential advantages).
I think (but am unsure) that what I care about is more like a metric for “is useful intellectual progress getting made” (whether or not LessWrong-the-website was causal in that progress).
The point here is not to evaluate the Lightcone team’s work, but for the community to have a better benchmark for it’s collective progress (which then hopefully, like, improves credit-assignment which then hopefully improves our ability to collectively focus on useful stuff as the community scales)
This point does seem interesting though and maybe a different frame than I had previously been thinking in:
The marginal contribution of LW is more in making it more likely that better posts are read and in making various conversations happen (with a variety of other diffuse potential advantages).
I think (but am unsure) that what I care about is more like a metric for “is useful intellectual progress getting made” (whether or not LessWrong-the-website was causal in that progress).
Seems reasonable. From my perspective LW review is very bad for measuring overall (human) progress on achieving good things, though plausibly better than any other specific review or ranking process that has a considerable amount of buy in.
Do you think replacing (or at least combining) LW Review with the Open Problems frame would be an improvement on that axis?
Also: does it seem useful to you to measure overall progress on [the cluster of good things that the rationality and/or alignment community are pointed at?]?
Do you think replacing (or at least combining) LW Review with the Open Problems frame would be an improvement on that axis?
Uh, maybe for combining? I think my main complaint with LW review as a metric is more just that I disagree with the preferences of other people and think that a bunch of work is happening on places other than LW. I don’t really think Open Problems helps much with this from my perspective. (In many cases I can’t name a clear and operationalized open problem and more just think “more progress here would be good.)
I don’t think that a solution to open problems being posted on LW would indicate that LW (the website and org, not the surrounding community) was accomplishing something useful.
E.g., imagine using the same metric for arXiv. (This case is more extreme, but I think it corresponds somewhat.)
Awkwardly, I think the existence of good posts is unlikely to track LW’s contribution. This seems especially true for posts about solutions to technical problems. The marginal contribution of LW is more in making it more likely that better posts are read and in making various conversations happen (with a variety of other diffuse potential advantages).
I don’t know what a good metric for LW is.
I’m not 100% sure I got your point.
I think (but am unsure) that what I care about is more like a metric for “is useful intellectual progress getting made” (whether or not LessWrong-the-website was causal in that progress).
The point here is not to evaluate the Lightcone team’s work, but for the community to have a better benchmark for it’s collective progress (which then hopefully, like, improves credit-assignment which then hopefully improves our ability to collectively focus on useful stuff as the community scales)
This point does seem interesting though and maybe a different frame than I had previously been thinking in:
Seems reasonable. From my perspective LW review is very bad for measuring overall (human) progress on achieving good things, though plausibly better than any other specific review or ranking process that has a considerable amount of buy in.
I wasn’t quite sure from your phrasings:
Do you think replacing (or at least combining) LW Review with the Open Problems frame would be an improvement on that axis?
Also: does it seem useful to you to measure overall progress on [the cluster of good things that the rationality and/or alignment community are pointed at?]?
Uh, maybe for combining? I think my main complaint with LW review as a metric is more just that I disagree with the preferences of other people and think that a bunch of work is happening on places other than LW. I don’t really think Open Problems helps much with this from my perspective. (In many cases I can’t name a clear and operationalized open problem and more just think “more progress here would be good.)