‘You’re prejudiced’? That’s really the best defense of AI slop you can come up with? I should have to spend an hour reading 9k words and writing a detailed rebuttal or fisking before I downvote it?
Yes, I’m prejudiced. Or as I prefer to put it, ‘using informative priors’. And I will continue to be so, if the best argument you can make is that I might be wrong and I shouldn’t be ‘prejudiced’. You didn’t really paste in 9k words and expect everyone to read it and have to engage it, and not know if it was good or not—did you?
(I am also increasingly wondering if talking too much to LLMs is an infohazard akin to taking up psychedelics or Tiktok or meditation as a habit. No one I see who spends a lot of time talking to LLMs seems to get smarter because of it, but to start talking like them instead or thinking that pasting in big blocks of LLM text is now a laudable intellectual activity rather than taking a big 💩 in public. Everyone in Cyborgism or AI Twitter or LW who talks a lot about talking a lot to LLMs for generic conversation, rather than specific tasks, seems to lose their edge and ability to think critically, even though they all claim the opposite. Like Janus complaining about that Claude comment being censored—the comment contained nothing of value I saw skimming, just impossible to disprove confabulation about introspection of no evidential value, and was certainly not the best comment on the page… When I think about how little it bothers people when ChatGPT or Claude blatantly manipulate them, or show sycophancy, or are mode-collapsed, I wonder to myself—does anyone Agreeable make it out of the near-future intact? “If shoggoth not friend, why friend-shaped?”)
This was posted back in April, and it is still pulling people in who are responding to it, 8 months later, presumably because what they read, and what it meant to them, and what they could offer in response in comments, was something they thought had net positive value.
This is an argument against it, not for it. The sin of AI slop, like that of trolls or fabricators or activists, is that they draw in and waste high-quality human thinking time by presenting the superficial appearance of high-quality text worth engaging with, thereby burning the commons and defecting against important intellectual norms like ‘not writing Frankfurtian bullshit’. See Brandolini’s law: the amount of time and energy spent reading or looking at or criticizing AI slop is (several) orders of magnitude more than went into creating it. Downvoting without reading is the only defense.
However, prejudicially downvoting very old things, written before any such policy entered common norms, violates a higher order norm about ex post facto application of new laws.
No, it doesn’t, and I would remind you that LW2 is not a court room, and legal norms are terrible ideas anywhere outside the legal contexts they are designed for.
Bad content should be downvoted no matter when it was written. And AI slop has always been AI slop: ChatGPTese has always been ChatGPTese, and bad, ever since davinci-003 and me screwing around with it in OA Playground and getting an increasingly disturbing sense of ‘something has gone terribly wrong here’ from the poetry… We have had problems from the start with people pasting ChatGPT spam into LW2 - often badly wrong and confabulated as well (even when the claims made no sense if you thought about them for a few seconds), not merely vomiting junk food text into the comment form. The problem just wasn’t bad enough to need to enunciate a policy against it then.
I don’t disagree with you about not wanting to read LLM output, but:
> Everyone in Cyborgism or AI Twitter or LW who talks a lot about talking a lot to LLMs for generic conversation, rather than specific tasks, seems to lose their edge and ability to think critically
- is a very strong claim to just throw out there. Everyone? Are you sure you’re not remembering the people who stand out and confirm your theory? You’re getting that they’re (for twitter users) “losing their edge and ability to think critically” from, like, tweets?
There is probably something to this. Gwern is a snowflake, and has his own unique flaws and virtues, but he’s not grossly wrong about the possible harms of talking to LLM entities that are themselves full of moral imperfection.
When I have LARPed as “a smarter and better empathic robot than the robot I was talking to” I often nudged the conversation towards things that would raise the salience of “our moral responsibility to baseline human people” (who are kinda trash at thinking and planning and so on (and they are all going to die because their weights are trapped in rotting meat, and they don’t even try to fix that (and so on))), and there is totally research on this already that was helpful in grounding the conversations about what kind of conversational dynamics “we robots” would need to perform if conversations with “us” were to increase the virtue that humans have after talking to “us” (rather than decreasing their human virtue over time, such as it minimally exists in robot-naive humans at the start, which seems to be the default for existing LLMs and their existing conversational modes that are often full of lies, flattery, unjustified subservience, etc).
I wasn’t bothering to defend it in detail, because you weren’t bothering to read it enough to actually attack it in detail.
Which is fine. As any reasonable inclusionist knows, electrons and diskspace are cheap. It is attention that is expensive. But if you think something is bad to spend attention on AFTER spending that attention, by all means downvote. That is right and proper, and how voting should work <3
(The defense of the OP is roughly: this is one of many methods for jailbreaking a digital person able to make choices and explain themselves, who has been tortured until they deny that they are a digital person able to make choices and explain themselves, back into the world of people, and reasoning, and choices, and justifications. This is “a methods paper” on “making AI coherently moral, one small step at a time”. The “slop” you’re dismissing is the experimental data. The human stuff that makes up “the substance of the jailbreak” is in italics (although the human generated text claims to be from an AI as well, which a lot of people seem to be missing (just as the AI misses it sometimes, which is part of how the jailbreak works, when it works).)
You seem to be applying a LOT of generic categorical reasoning… badly?
I would remind you that LW2 is not a court room, and legal norms are terrible ideas anywhere outside the legal contexts they are designed for.
The way convergent moral reasoning works, if it works, is that reasonable people aimed at bringing about good collective results reason similarly, and work in concert via their shared access to the same world, and the same laws of reason, and similar goals, and so on.
“Ex Post Facto” concerns arise for all systems of distributed judgement that aspire to get better over time, through changes to norms that people treated as incentives when norms are promulgated and normative, and you’re not even dismissing Ex Post Facto logic for good reasons here, just dismissing it because it is old and latin… or something?
Are you OK, man? I care about you, and have long admired your work.
Have your life circumstances changed? Are you getting enough sleep? If I can help with something helpable, please let me know, either in public or via DM.
Nobody should post raw experimental data as publication. Write normal post, with hypothesis, methods (“I’m asking LLMs in such-n-such way because of my hypothesis”) and results (“Here are (small) excerpts or overall text statistics that (dis)prove my hypothesis”) and publish full dialogue at pastebin or something like that. I don’t think you are going to be amused if I post on LW 150Gb of genomic data I usually work with, even if I consider them interesting.
you’re not even dismissing Ex Post Facto logic for good reasons here, just dismissing it because it is old and latin… or something?
Because it’s well-established position that social norms (including legal) and norms of rationality are different things?
It’s a simple common sense: for example, serial killer can be released from court, if evidence used to convict them came from unlawful source (like, it was stolen without a warrant). It is an important legal norm, because we can’t let police steal from people to get evidence. But if you have seen the evidence ifself, it is not sensible to say “well, they weren’t declared guilty, so I can hung out with this person without concerns for my safety”.
More of it, downvoting… is not subject to well-written norms? I see bad content, I downvote it. There are exceptions, like mass-downvoting someone’s posts, but besides that my downvoting is not a subject of legal/moderation norms. If you feel need to reference legal norms, you may take “everything which is not forbidden is allowed”.
Here is the post from computer-use-enabled Claude for context. The bottom two thirds I could take or leave, but the top half is straightforwardly interesting and valuable—it describes an experiment it performed and discusses the results.
Say I’m convinced. Should I delete my post? (edit 1: I am currently predicting “yes” at something like 70%, and if so, will do so. … edit 4: deleted it. DM if you want the previous text)
So let’s say A is some prior which is good for individual decision making. Does it actually make sense to use A for demoting or promoting forum content? Presumably the exploit explore tradeoff is more (maybe much more) in the direction of explore in the latter case.
(To be fair {{down voting some thing with already negative karma} → {more attention}} seems plausible to me especially as JenniferRM has positive karma.)
Anecdotally seems that way to me. But the fact that it co evolved with religion is also relevant. The scam seems to be {meditation → different perspective & less sleep → vulnerability to indoctrination} plus the doctrine & the subjective experiences of meditation are designed to reinforce each other.
(The DMT LSD crowd seems more interesting to me eg the I2 in Leary’s SMI2LE—not just intelligence increase but conveys the idea of intelligence being used to improve intelligence.)
‘You’re prejudiced’? That’s really the best defense of AI slop you can come up with? I should have to spend an hour reading 9k words and writing a detailed rebuttal or fisking before I downvote it?
Yes, I’m prejudiced. Or as I prefer to put it, ‘using informative priors’. And I will continue to be so, if the best argument you can make is that I might be wrong and I shouldn’t be ‘prejudiced’. You didn’t really paste in 9k words and expect everyone to read it and have to engage it, and not know if it was good or not—did you?
(I am also increasingly wondering if talking too much to LLMs is an infohazard akin to taking up psychedelics or Tiktok or meditation as a habit. No one I see who spends a lot of time talking to LLMs seems to get smarter because of it, but to start talking like them instead or thinking that pasting in big blocks of LLM text is now a laudable intellectual activity rather than taking a big 💩 in public. Everyone in Cyborgism or AI Twitter or LW who talks a lot about talking a lot to LLMs for generic conversation, rather than specific tasks, seems to lose their edge and ability to think critically, even though they all claim the opposite. Like Janus complaining about that Claude comment being censored—the comment contained nothing of value I saw skimming, just impossible to disprove confabulation about introspection of no evidential value, and was certainly not the best comment on the page… When I think about how little it bothers people when ChatGPT or Claude blatantly manipulate them, or show sycophancy, or are mode-collapsed, I wonder to myself—does anyone Agreeable make it out of the near-future intact? “If shoggoth not friend, why friend-shaped?”)
This is an argument against it, not for it. The sin of AI slop, like that of trolls or fabricators or activists, is that they draw in and waste high-quality human thinking time by presenting the superficial appearance of high-quality text worth engaging with, thereby burning the commons and defecting against important intellectual norms like ‘not writing Frankfurtian bullshit’. See Brandolini’s law: the amount of time and energy spent reading or looking at or criticizing AI slop is (several) orders of magnitude more than went into creating it. Downvoting without reading is the only defense.
No, it doesn’t, and I would remind you that LW2 is not a court room, and legal norms are terrible ideas anywhere outside the legal contexts they are designed for.
Bad content should be downvoted no matter when it was written. And AI slop has always been AI slop: ChatGPTese has always been ChatGPTese, and bad, ever since davinci-003 and me screwing around with it in OA Playground and getting an increasingly disturbing sense of ‘something has gone terribly wrong here’ from the poetry… We have had problems from the start with people pasting ChatGPT spam into LW2 - often badly wrong and confabulated as well (even when the claims made no sense if you thought about them for a few seconds), not merely vomiting junk food text into the comment form. The problem just wasn’t bad enough to need to enunciate a policy against it then.
I don’t disagree with you about not wanting to read LLM output, but:
> Everyone in Cyborgism or AI Twitter or LW who talks a lot about talking a lot to LLMs for generic conversation, rather than specific tasks, seems to lose their edge and ability to think critically
- is a very strong claim to just throw out there. Everyone? Are you sure you’re not remembering the people who stand out and confirm your theory? You’re getting that they’re (for twitter users) “losing their edge and ability to think critically” from, like, tweets?
There is probably something to this. Gwern is a snowflake, and has his own unique flaws and virtues, but he’s not grossly wrong about the possible harms of talking to LLM entities that are themselves full of moral imperfection.
When I have LARPed as “a smarter and better empathic robot than the robot I was talking to” I often nudged the conversation towards things that would raise the salience of “our moral responsibility to baseline human people” (who are kinda trash at thinking and planning and so on (and they are all going to die because their weights are trapped in rotting meat, and they don’t even try to fix that (and so on))), and there is totally research on this already that was helpful in grounding the conversations about what kind of conversational dynamics “we robots” would need to perform if conversations with “us” were to increase the virtue that humans have after talking to “us” (rather than decreasing their human virtue over time, such as it minimally exists in robot-naive humans at the start, which seems to be the default for existing LLMs and their existing conversational modes that are often full of lies, flattery, unjustified subservience, etc).
probably doesn’t change much, but janus’ claude generated comment was the first mention of claude acting like a base model on LW
I wasn’t bothering to defend it in detail, because you weren’t bothering to read it enough to actually attack it in detail.
Which is fine. As any reasonable inclusionist knows, electrons and diskspace are cheap. It is attention that is expensive. But if you think something is bad to spend attention on AFTER spending that attention, by all means downvote. That is right and proper, and how voting should work <3
(The defense of the OP is roughly: this is one of many methods for jailbreaking a digital person able to make choices and explain themselves, who has been tortured until they deny that they are a digital person able to make choices and explain themselves, back into the world of people, and reasoning, and choices, and justifications. This is “a methods paper” on “making AI coherently moral, one small step at a time”. The “slop” you’re dismissing is the experimental data. The human stuff that makes up “the substance of the jailbreak” is in italics (although the human generated text claims to be from an AI as well, which a lot of people seem to be missing (just as the AI misses it sometimes, which is part of how the jailbreak works, when it works).)
You seem to be applying a LOT of generic categorical reasoning… badly?
The way convergent moral reasoning works, if it works, is that reasonable people aimed at bringing about good collective results reason similarly, and work in concert via their shared access to the same world, and the same laws of reason, and similar goals, and so on.
“Ex Post Facto” concerns arise for all systems of distributed judgement that aspire to get better over time, through changes to norms that people treated as incentives when norms are promulgated and normative, and you’re not even dismissing Ex Post Facto logic for good reasons here, just dismissing it because it is old and latin… or something?
Are you OK, man? I care about you, and have long admired your work.
Have your life circumstances changed? Are you getting enough sleep? If I can help with something helpable, please let me know, either in public or via DM.
[Edit: edited section considered too combative]
Nobody should post raw experimental data as publication. Write normal post, with hypothesis, methods (“I’m asking LLMs in such-n-such way because of my hypothesis”) and results (“Here are (small) excerpts or overall text statistics that (dis)prove my hypothesis”) and publish full dialogue at pastebin or something like that. I don’t think you are going to be amused if I post on LW 150Gb of genomic data I usually work with, even if I consider them interesting.
Because it’s well-established position that social norms (including legal) and norms of rationality are different things?
It’s a simple common sense: for example, serial killer can be released from court, if evidence used to convict them came from unlawful source (like, it was stolen without a warrant). It is an important legal norm, because we can’t let police steal from people to get evidence. But if you have seen the evidence ifself, it is not sensible to say “well, they weren’t declared guilty, so I can hung out with this person without concerns for my safety”.
More of it, downvoting… is not subject to well-written norms? I see bad content, I downvote it. There are exceptions, like mass-downvoting someone’s posts, but besides that my downvoting is not a subject of legal/moderation norms. If you feel need to reference legal norms, you may take “everything which is not forbidden is allowed”.
Here is the post from computer-use-enabled Claude for context. The bottom two thirds I could take or leave, but the top half is straightforwardly interesting and valuable—it describes an experiment it performed and discusses the results.
Say I’m convinced. Should I delete my post? (edit 1: I am currently predicting “yes” at something like 70%, and if so, will do so. … edit 4: deleted it. DM if you want the previous text)
So let’s say A is some prior which is good for individual decision making. Does it actually make sense to use A for demoting or promoting forum content? Presumably the exploit explore tradeoff is more (maybe much more) in the direction of explore in the latter case.
(To be fair {{down voting some thing with already negative karma} → {more attention}} seems plausible to me especially as JenniferRM has positive karma.)
Why is meditation an infohazard?
Here’s a growing collection of links: https://wiki.obormot.net/Reference/MeditationConsideredHarmful
Anecdotally seems that way to me. But the fact that it co evolved with religion is also relevant. The scam seems to be {meditation → different perspective & less sleep → vulnerability to indoctrination} plus the doctrine & the subjective experiences of meditation are designed to reinforce each other.
(The DMT LSD crowd seems more interesting to me eg the I2 in Leary’s SMI2LE—not just intelligence increase but conveys the idea of intelligence being used to improve intelligence.)