A core value of LessWrong is to be timeless and not news-driven.
I do really like the simplicity and predictability of the Hacker News algorithm. More karma means more visibility, older means less visibility.
Our current goal is to produce a recommendations feed that both makes people feel like they’re keeping up to date with what’s new (something many people care about) and also suggest great reads from across LessWrong’s entire archive.
I hope that we can avoid getting swallowed by Shoggoth for now by putting a lot of thought into our optimization targets
(Emphasis mine.)
Here’s an idea[1] for a straightforward(?) recommendation algorithm: Quantilize over all past LessWrong posts by using inflation-adjusted karma as a metric of quality.
The advantage is that this is dogfooding on some pretty robust theory. I think this isn’t super compute-intensive, since the only thing one has to do is to compute the cumulative distribution function once a day (associating it with the post), and then inverse transform sampling from the CDF.
Recommending this way has the disadvantage of not being recency-favoring (which I personally like), and not personalized (which I also like).
By default, it also excludes posts below a certain karma threshold. That could be solved by exponentially tilting the distribution instead of cutting it off (θ>0, otherwise to be determined (experimentally?)). Such a recommendation algorithm wouldn’t be as robust against very strong optimizers, but since we have some idea what high-karma LessWrong posts look like (& we’re not dealing with a superintelligent adversary… yet), that shouldn’t be a problem.
Personalization is easy to achieve while keeping the algorithm transparent. Just rank your own viewed/commented posts by most frequent tags, then score past posts based on the tags and pick a quantile based on the mixed upvotes/tags score, possibly with a slider parameter that allows you to adjust which of the two things you want to matter most.
(Emphasis mine.)
Here’s an idea[1] for a straightforward(?) recommendation algorithm: Quantilize over all past LessWrong posts by using inflation-adjusted karma as a metric of quality.
The advantage is that this is dogfooding on some pretty robust theory. I think this isn’t super compute-intensive, since the only thing one has to do is to compute the cumulative distribution function once a day (associating it with the post), and then inverse transform sampling from the CDF.
Recommending this way has the disadvantage of not being recency-favoring (which I personally like), and not personalized (which I also like).
By default, it also excludes posts below a certain karma threshold. That could be solved by exponentially tilting the distribution instead of cutting it off (θ>0, otherwise to be determined (experimentally?)). Such a recommendation algorithm wouldn’t be as robust against very strong optimizers, but since we have some idea what high-karma LessWrong posts look like (& we’re not dealing with a superintelligent adversary… yet), that shouldn’t be a problem.
If I was more virtuous, I’d write a pull request instead of a comment.
Personalization is easy to achieve while keeping the algorithm transparent. Just rank your own viewed/commented posts by most frequent tags, then score past posts based on the tags and pick a quantile based on the mixed upvotes/tags score, possibly with a slider parameter that allows you to adjust which of the two things you want to matter most.