LessWrong team member / moderator. I’ve been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I’ve been interested in improving my own epistemic standards and helping others to do so as well.
Raemon
Karma: 53,281
I don’t feel very hopeful about the conversation atm, but fwiw I feel like you are missing a fairly important point while being pretty overconfident about not having missed it.Putting a different way: is there a percent of people who could disagree with you about what consciousness means, which would convince you that you it’s not as straightforward as assuming you have the correct definition of consciousness, and that you can ignore everyone else? If <50% of people agreed with you? If <50% of the people with most of the power?
(This is not about whether your definition is good, or the most useful, or whatnot – only that, if lots of people turned out to be mean different things by it, would it still particularly matter whether your definition was the “right” one?)
I think many of the things Critch has listed as definitions of consciousness are not “weak versions of some strong version”, they’re just different things.
You bring up a few times that LLMs don’t “experience” [various things Critch lists here]. I agree, they pretty likely don’t (in most cases). But, part of what I interpreted Critch’s point here to be was that there are many things that people mean by “consciousness” that aren’t actually about “experience” or “qualia” or whatnot.
For example, I’d bet (75%) that when Critch says they have introspection, he isn’t making any claims about them “experiencing” anything at all – I think he’s instead saying “in the same way that their information processing system knows facts about Rome and art and biology and computer programming, and can manipulate those facts, it can also know and manipulate facts about it’s thoughts and internal states.” (whereas other ML algorithms may not be able to know and manipulate their thoughts and internal states)
Purposefulness: Not only irrelevant to consciousness but...
A major point Critch was making in previous post is that when people say “consciousness”, this is one of the things they sometimes mean. The point is not that LLMs are conscious the way you are using the word, but that when you see debates about whether they are conscious, it will include some people who think it means “purposefulness.”
I think LessWrong used to be more like this. It certainly had some upsides to “have culture”. There are also downsides. I certainly miss it but not sure whether I’d hit a button to change it.
I’m not sure what past-you meant here, but, one thing you might think is “the amount of hurdles you have to jump through to profit off drugs is ‘hard’, i.e. you (unnecessarily) need to be very well funded and well connected company that can navigate bureaucratic hurdles”, and it’s not that you can’t do it. It’s just, like, “hard”, ya know?
Oh to be clear I don’t think it was bad for you to post this as-is. Just that I’d like to see more followup
This post seems important-if-right. I get a vibe from it of aiming to persuade more than explain, and I’d be interested in multiple people gathering/presenting evidence about this, preferably at least some of them who are (currently) actively worried about China.
I’ve recently made a pull-request (not quite ready to merge yet) that gives LessWrong Fatebook hoverovers (which are different from embeds. I’m considering also making embeds, although I think the UI takes up a bit too much space by default).
I am into “more Fatebook integration everywhere”.
(I think individual FB questions can toggle whether to show/hide predictions before you’ve made your own)
This seems right to me, but the discussion of “scaling will plateau” feels like it usually comes bundled with “and the default expectation is that this means LLM-centric-AI will plateau”, which seems like the wrong-belief-to-have, to me.
Noting, this doesn’t really engage with any of the particular other claims in the previous comment’s link, just makes a general assertion.
Curated. This was one of the more inspiring things I read this year (in a year that had a moderate number of inspiring things!)
I really like how Sarah lays out the problem and desiderata for neutrality in our public/civic institutional spaces.
LessWrong’s strength is being a fairly opinionated ”university[1]” about how to do epistemics, which the rest of the world isn’t necessarily bought into. Trying to make LW a civic institution would fail. But, this post has me more excited to revisit “what would be necessary to build good, civic infrastructure” (where “good” requires both “be ‘good’ in some kind of deep sense,” but also “be memetically fit enough to compete with Twitter et all.” One solution might be convincing Musk of specific policies rather than building a competitor)
- ^
I.e. A gated community with epistemic standards, a process for teaching people, and a process for some of those people going on to do more research.
- ^
You can make a post or shortform discussing it and see what people think. I recommend front loading the main arguments, evidence or takeaways so people can easily get a sense of it—people often bounce off long worldview posts from newcomers
Fwiw I didn’t find the post hostile.
I’m assuming “natural abstraction” is also a scalar property. Reading this paragraph, I refactored the concept in my mind to “some abstractions tend to be cheaper to abstract than others. agents will converge to using cheaper abstractions. Many cheapness properties generalize reasonably well across agents/observation-systems/environments, but, all of those could in theory come apart.”
And the Strong NAH would be “cheap-to-abstract-ness will be very punctuated, or something” (i.e. you might expect less of a smooth gradient of cheapnesses across abstractions)
How would you solve the example legal situation you gave?
Thanks, this gave me the context I needed.
Put another way: this post seems like it’s arguing with someone but I’m not sure who.
I think I care a bunch about the subject matter of this post, but something about the way this post is written leaves me feeling confused and ungrounded.
Before reading this post, my background beliefs were:
Rationality doesn’t (quite) equal Systemized Winning. Or, rather, that focusing on this seems to lead people astray more than helps them.
There’s probably some laws of cognition to be discovered, about what sort of cognition will have various good properties, in idealized situations.
There’s probably some messier laws of cognition that apply to humans (but those laws are maybe more complicated).
Neither sets of laws necessarily have a simple unifying framework that accomplishes All the Things (although I think the search for simplicity/elegance/all-inclusiveness is probably a productive search, i.e. it tends to yield good stuff along the way. “More elegance” is usually achievable on the margin.
There might be heuristics that work moderately well for humans much of the time, which approximate those laws.
there are probably Very Rough heuristics you can tell an average person without lots of dependencies, and somewhat better heuristics you can give to people who are willing to learn lots of subskills.
Given all that… is there anything in-particular I am meant to take from this post? (I have right now only skimmed it, it felt effortful to comb for the novel bits). I can’t tell whether the few concrete bits are particularly important, or just illustrative examples.
This is not very practically useful to me but dayumn it is cool
An individual Social Psychology lab (or lose collection of labs) can choose who to let in.
Frontier Lab AI companies can decide who to hire, and what sort of standards they want internally (and maybe, in a lose alliance with other Frontier Lab companies).
The Immoral Mazes outlines some reasons that you might think large institutions are dramatically worse than smaller ones (see: Recursive Middle Manager Hell for a shorter intro, although I don’t spell out the part argument about how mazes are sort of “contagious” between large institutions)
But the simpler argument is “the fewer people you have, the easier it is for a few leaders to basically make personal choices based on their goals and values,” rather than selection effects resulting in the largest institutions being better modeled as “following incentives” rather than “pursuing goals on purpose.” (If an organization didn’t follow the incentives, they’d be outcompeted by one that does)
(My own answer is that if like >75% of people agreed on what consciousness means, I’d be like “okay yeah Critch’s point isn’t super compelling”. If it was between like 50 − 75% of people I’d like “kinda edge case.” If it’s <50% of people agreeing on consciousness, I don’t think it matters much what definition is “correct.”)