For this year’s LessWrong Review, we’re building UI to make it much easier to import linkposts from other blogs, since a lot of important rationalsphere or AI Safety content lives in other places, and backdate it such that it’s eligible for the Review.
It’s actually pretty easy to automatically import all the text from a url in most cases (We’re looking into auto-importing PDFs of papers, which I suspect is doable but haven’t checked), and in many cases I think this would basically be preferred, but it’s also kinda exploitable in ways I don’t know that I’d endorse. (i.e. some authors are probably happy to have people crosspost stuff while nominating it for Best of LessWrong, other authors might feel violated)
Three options are:
only auto-import the first few paragraphs, ending with a load more
have an LLM extract some important highlights. (I’m ignoring “have an LLM summarize it” because they suck at that, but I think they’re decent at identifying key paragraphs)
start off by auto-importing the whole post, and then wait until anybody complains.
I’d probably be limiting this to users who are otherwise eligible to nominate (i.e. their account is at least two years old, and maybe they have like 100 karma), so randos can’t go crazy with it. Admins will be seeing all posts imported this way so we can be sanity checking things.
Tentative support for only auto-importing the first few paragraphs, if not that then start by auto-importing the whole post and waiting until anybody complains. My guess (~65%?) is that somebody will. Against having an LLM extract some important highlights- if doing highlights is the way to go I think whoever nominated the piece for the review can find the highlights?
I’d love it if I could use LessWrong as a central place to read rationalsphere content, and since more and more rationalist sphere writers are writing elsewhere this seems like it’s worth trying.
Against having an LLM extract some important highlights- if doing highlights is the way to go I think whoever nominated the piece for the review can find the highlights?
This changes it from a 10 second operation to a several minute operation, which makes it prohibitively expensive to do it for a lot of posts.
Curious to hear more about what feels off about LLM extract. I do think this is something they’re actually pretty good at (and you can always edit it afterwards)
I imagine two people are talking and one says “oh, I think you should read this essay, here’s the link!” and the second asks “oh, what’s it about? Any good quotes?”
If the first doesn’t have an answer to that, then it feels like a weird recommendation? I guess that’s the second stage of, where people review them.
Yeah. It needs a review to pass to the third stage so this should have come up by then. The first stage is “are there a number of people who are like ‘oh yeah that post, that was important’ and upvote it?”
I lean towards an opt-out system for whole post imports? I’d expect the vast majority of relevant authors to be happy with it, and it would offer less inconvenience to readers. Letting an author easily register as “no whole text imports please” seems worthwhile, and maybe if people aren’t happy with that switching to opt-in?
well a lot of the things-imported may be from people who don’t think of themselves as centrally LW members, or who wouldn’t notice.
(medium-difficulty case: Robin Hanson. Harder-difficulty-case: some academic who wrote something relevant to x-risk but isn’t actually very involved in our ecosystem)
Cool, in that case probably opt-in to full-post makes more sense, maybe with the ability to switch modes for all posts by an author if they give permission?
For this year’s LessWrong Review, we’re building UI to make it much easier to import linkposts from other blogs, since a lot of important rationalsphere or AI Safety content lives in other places, and backdate it such that it’s eligible for the Review.
It’s actually pretty easy to automatically import all the text from a url in most cases (We’re looking into auto-importing PDFs of papers, which I suspect is doable but haven’t checked), and in many cases I think this would basically be preferred, but it’s also kinda exploitable in ways I don’t know that I’d endorse. (i.e. some authors are probably happy to have people crosspost stuff while nominating it for Best of LessWrong, other authors might feel violated)
Three options are:
only auto-import the first few paragraphs, ending with a load more
have an LLM extract some important highlights. (I’m ignoring “have an LLM summarize it” because they suck at that, but I think they’re decent at identifying key paragraphs)
start off by auto-importing the whole post, and then wait until anybody complains.
I’d probably be limiting this to users who are otherwise eligible to nominate (i.e. their account is at least two years old, and maybe they have like 100 karma), so randos can’t go crazy with it. Admins will be seeing all posts imported this way so we can be sanity checking things.
Curious what people think.
Tentative support for only auto-importing the first few paragraphs, if not that then start by auto-importing the whole post and waiting until anybody complains. My guess (~65%?) is that somebody will. Against having an LLM extract some important highlights- if doing highlights is the way to go I think whoever nominated the piece for the review can find the highlights?
I’d love it if I could use LessWrong as a central place to read rationalsphere content, and since more and more rationalist sphere writers are writing elsewhere this seems like it’s worth trying.
This changes it from a 10 second operation to a several minute operation, which makes it prohibitively expensive to do it for a lot of posts.
Curious to hear more about what feels off about LLM extract. I do think this is something they’re actually pretty good at (and you can always edit it afterwards)
I imagine two people are talking and one says “oh, I think you should read this essay, here’s the link!” and the second asks “oh, what’s it about? Any good quotes?”
If the first doesn’t have an answer to that, then it feels like a weird recommendation? I guess that’s the second stage of, where people review them.
Yeah. It needs a review to pass to the third stage so this should have come up by then. The first stage is “are there a number of people who are like ‘oh yeah that post, that was important’ and upvote it?”
I lean towards an opt-out system for whole post imports? I’d expect the vast majority of relevant authors to be happy with it, and it would offer less inconvenience to readers. Letting an author easily register as “no whole text imports please” seems worthwhile, and maybe if people aren’t happy with that switching to opt-in?
well a lot of the things-imported may be from people who don’t think of themselves as centrally LW members, or who wouldn’t notice.
(medium-difficulty case: Robin Hanson. Harder-difficulty-case: some academic who wrote something relevant to x-risk but isn’t actually very involved in our ecosystem)
Cool, in that case probably opt-in to full-post makes more sense, maybe with the ability to switch modes for all posts by an author if they give permission?
I think basically nobody is going to really opt-in-or-out, so I think the question is “what actually is a reasonable default?”
If it’s easy for submitters to check a box which says “I asked them and they said full post imports are fine”, maybe?
No strong takes on default, just obvious considerations you’ll have thought of.
Mmm, I kinda like that.