see also my eaforum at https://forum.effectivealtruism.org/users/dirk and my tumblr at https://d-i-r-k-s-t-r-i-d-e-r.tumblr.com/ .
dirk
When I went to the page just now there was a section at the top with an option to download it; here’s the direct PDF link.
Normal statements actually can’t be accepted credulously if you exercise your reason instead of choosing to believe everything you hear (edit, some people lack this capacity due to tragic psychological issues such as having an extremely weak sense of self, hence my reference to same); so too with statements heard on psychedelics, and it’s not even appreciably harder.
Disagree, if you have a strong sense of self statements you hear while on psychedelics are just like normal statements.
Indeed, people with congenital insensitivity to pain don’t feel pain upon touching hot stoves (or in any other circumstance), and they’re at serious risk of infected injuries and early death because of it.
I think the ego is, essentially, the social model of the self. One’s sense of identity is attached to it (effectively rendering it also the Cartesian homunculus), which is why ego death feels so scary to people, but (in most cases; I further theorize that people who developed their self-conceptions top-down, being likelier to have formed a self-model at odds with reality, are worse-affected here) the traits which make up the self-model’s personality aren’t stored in the model; it’s merely a lossy description thereof and will rearise with approximately the same traits if disrupted.
OpenAI is partnering with Anduril to develop models for aerial defense: https://www.anduril.com/article/anduril-partners-with-openai-to-advance-u-s-artificial-intelligence-leadership-and-protect-u-s/
I haven’t tried harmful outputs, but FWIW I’ve tried getting it to sing a few times and found that pretty difficult.
Of course this would shrink the suspect pool, but catching the leaker more easily after the fact is very different from the system making it difficult to leak things. Under the proposed system, it would be very easy to leak things.
But someone who declared intent to read could simply take a picture and send it to any number of people who hadn’t declared intent.
How much of this was written by an LLM?
I enjoy being embodied, and I’d describe what I enjoy as the sensation rather than the fact. Proprioception feels pleasant, touch (for most things one is typically likely to touch) feels pleasant, it is a joy to have limbs and to move them through space. So many joints to flex, so many muscles to tense and untense. (Of course, sometimes one feels pain, but this is thankfully the exception rather than the rule).
No, I authentically object to having my qualifiers ignored, which I see as quite distinct from disagreeing about the meaning of a word.
Edit: also, I did not misquote myself, I accurately paraphrased myself, using words which I know, from direct first-person observation, mean the same thing to me in this context.
You in particular clearly find it to be poor communication, but I think the distinction you are making is idiosyncratic to you. I also have strong and idiosyncratic preferences about how to use language, which from the outside view are equally likely to be correct; the best way to resolve this is of course for everyone to recognize that I’m objectively right and adjust their speech accordingly, but I think the practical solution is to privilege neither above the other.
I do think that LLMs are very unlikely to be conscious, but I don’t think we can definitively rule it out.
I am not a panpsychist, but I am a physicalist, and so I hold that thought can arise from inert matter. Animal thought does, and I think other kinds could too. (It could be impossible, of course, but I’m currently aware of no reason to be sure of that). In the absence of a thorough understanding of the physical mechanisms of consciousness, I think there are few mechanisms we can definitively rule out.
Whatever the mechanism turns out to be, however, I believe it will be a mechanism which can be implemented entirely via matter; our minds are built of thoughtless carbon atoms, and so too could other minds be built of thoughtless silicon. (Well, probably; I don’t actually rule out that the chemical composition matters. But like, I’m pretty sure some other non-living substances could theoretically combine into minds.)
You keep saying we understand the mechanisms underlying LLMs, but we just don’t; they’re shaped by gradient descent into processes that create predictions in a fashion almost entirely opaque to us. AIUI there are multiple theories of consciousness under which it could be a process instantiable that way (and, of course, it could be the true theory’s one we haven’t thought of yet). If consciousness is a function of, say, self-modeling (I don’t think this one’s true, just using it as an example) it could plausibly be instantiated simply by training the model in contexts where it must self-model to predict well. If illusionism (which I also disbelieve) is true, perhaps the models already feel the illusion of consciousness whenever they access information internal to them. Et cetera.
As I’ve listed two theories I disbelieve and none I agree with, which strikes me as perhaps discourteous, here are some theories I find not-entirely-implausible. Please note that I’ve given them about five minutes of casual consideration per and could easily have missed a glaring issue.Attention schema theory, which I heard about just today
‘It could be about having an efference copy’
I heard about a guy who thought it came about from emotions, and therefore was localized in (IIRC) the amygdala (as opposed to the cortex, where it sounded like he thought most people were looking)
Ipsundrums (though I don’t think I buy the bit about it being only mammals and birds in the linked post)
Global workspace theory
[something to do with electrical flows in the brain]
Anything with biological nerves is conscious, if not of very much (not sure what this would imply about other substrates)
Uhh it doesn’t seem impossible that slime molds could be conscious, whatever we have in common with slime molds
Who knows? Maybe every individual cell can experience things. But, like, almost definitely not.
your (incorrect) claim about a single definition not being different from an extremely confident vague definition”
That is not the claim I made. I said it was not very different, which is true. Please read and respond to the words I actually say, not to different ones.
The definitions are not obviously wrong except to people who agree with you about where to draw the boundaries.
My emphasis implied you used a term which meant the same thing as self-evident, which in the language I speak, you did. Personally I think the way I use words is the right one and everyone should be more like me; however, I’m willing to settle on the compromise position that we’ll both use words in our own ways.
As for the prior probability, I don’t think we have enough information to form a confident prior here.
My dialect does not have the fine distinction between “clear” and “self-evident” on which you seem to be relying; please read “clear” for “self-evident” in order to access my meaning.
Having a vague concept encompassing multiple possible definitions, which you are nonetheless extremely confident is the correct vague concept, is not that different from having a single definition in which you’re confident, and not everyone shares your same vague concept or agrees that it’s clearly the right one.
It doesn’t demonstrate automation of the entire workflow—you have to, for instance, tell it which topic to think of ideas about and seed it with examples—and also, the automated reviewer rejected the autogenerated papers. (Which, considering how sycophantic they tend to be, really reflects very negatively on paper quality, IMO.)
I agree LLMs are probably not conscious, but I don’t think it’s self-evident they’re not; we have almost no reliable evidence one way or the other.
Reading the Semianalysis post, it kind of sounds like it’s just their opinion that that’s what Anthropic did.
They say “Anthropic finished training Claude 3.5 Opus and it performed well, with it scaling appropriately (ignore the scaling deniers who claim otherwise – this is FUD)”—if they have a source for this, why don’t they mention it somewhere in the piece instead of implying people who disagree are malfeasors? That reads to me like they’re trying to convince people with force of rhetoric, which typically indicates a lack of evidence.
The previous is the biggest driver of my concern here, but the next paragraph also leaves me unconvinced. They go on to say “Yet Anthropic didn’t release it. This is because instead of releasing publicly, Anthropic used Claude 3.5 Opus to generate synthetic data and for reward modeling to improve Claude 3.5 Sonnet significantly, alongside user data. Inference costs did not change drastically, but the model’s performance did. Why release 3.5 Opus when, on a cost basis, it does not make economic sense to do so, relative to releasing a 3.5 Sonnet with further post-training from said 3.5 Opus?”
This does not make sense to me as a line of reasoning. I’m not aware of any reason that generating synthetic data would preclude releasing the model, and it seems obvious to me that Anthropic could adjust their pricing (or impose stricter message limits) if they would lose money by releasing at current prices. This seems to be meant as an explanation of why Anthropic delayed release of the purportedly-complete Opus model, but it doesn’t really ring true to me.
Is there some reason to believe them that I’m missing? (On a quick google it looks like none of the authors work directly for Anthropic, so it can’t be that they directly observed it as employees).