I’m an independent researcher currently working on a sequence of posts about consciousness. You can send me anonymous feedback here: https://www.admonymous.co/rafaelharth. If it’s about a post, you can add [q] or [nq] at the end if you want me to quote or not quote it in the comment section.
Rafael Harth
Karma: 4,411
Schindler’s List: we can talk about specific rationality lessons all day, but we all know the biggest bottleneck is trying in the first place. This movie is the transformation of an ethical egoist into a utilitarian.
It also shows the value of Money: the Unit of Caring.
I have my own benchmark of tasks that I think measure general reasoning to decide when I freak out about LLMs, and they haven’t improved on them. I was ready to be cautiously optimistic that LLMs can’t scale to AGI (and would have reduced by p(doom) slightly) even if they keep scaling by conventional metrics, so the fact that scaling itself also seems to break down (maybe, possibly, partially, to whatever extent it does in fact break down, I haven’t looked into it much) and we’re reaching physical limits are all good things.
I’m not particularly more optimistic about alignment working anytime soon, just about very long timelines.
(You did respond to all the important parts, rest of my comment is very much optional.)
I’m sure you’re aware that people feel like they have a broader continuous awareness of their visual field than they actully do. There are lots of demonstrations of this—e.g. change blindness, selective attention test, the fact that peripheral vision has terrible resolution and terrible color perception and makes faces look creepy. There’s a refrigerator light illusion thing—if X is in my peripheral vision, then maybe it’s currently active as just a little pointer in a tiny sub-area of my cortex, but as soon as I turn my attention to X it immediately unfolds in full detail across the global workspace.
Yes—and my point was that appealing to these phenomena is the kind of thing you will probably have to do to explain the meta problem of seeing. Which raises all kinds of issues—for example, change blindness by itself doesn’t logically prove anything, since it’s possible not to notice that something changed even if it was represented. Only the reverse conclusion is valid—if a subject can tell that X changed, then X was in awareness, but if they can’t tell, X may or may not have been in awareness. So teasing out exactly how much information is really present in awareness, given the positive and negative evidence, is a pretty big rabbit hole. (Poor resolution in peripheral vision does prove absence of information, but as with the memory example I’ve complained about in post #2, this is an example of something people don’t endorse under reflection anyway, so it doesn’t get you very far. Like, there is a very, very big difference between arguing that peripheral resolution is poor, which people will agree with as soon as they actually pay attention to their peripheral vision for the first time, and arguing that the continuous visual image they think they see is not really there, which most people will stubbornly disagree with regardless of how much attention they pay to it.)
Anyway, that’s the only claim I was making—I was only trying to go as far as “this is why I think the problem is nontrivial and you haven’t solved it yet”, not “and that’s why you can’t solve it”.
The contents of IT are really truly different from the contents of LIP [I didn’t check where the visual information gets to the cortex in blindsight, I’m just guessing LIP for concreteness]. Querying IT is a different operation than querying LIP. IT holds different types of information than LIP does, and does different things with that information, including leading to different visceral reactions, motivations, semantic knowledge, etc., all of which correspond to neuroscientific differences in how IT versus LIP is wired up.
All these differences between IT vs LIP are in the territory, not the map. So I definitely agree that “the distinction [between seeing and vague-sense-of-presence] isn’t just that we happen to call them by different labels”. They’re different like how the concept “hand” is different from the concept “foot”—a distinction on the map downstream of a distinction in the territory.
Right, and I agree that this makes it apriori plausible that they could account for the differences in how people talk about, e.g., vivid seeing vs. intangible intuitions. But it doesn’t prove that they do, it only shows that this is the kind of explanation that, on first glance, looks like it could work. To actually solve the meta problem, you still have to do the work of explaining all the properties of introspective reports, which requires going into a lot of detail.
As of above, this is the only claim I was making—I’m not saying any of these issues are provably impossible with your approach, I’m only saying that your approach hasn’t provided a full solution yet. (And that I genuinely think most of the difficulty happens to be in these still unaddressed details; this was the point of the carrot/plant analogy.)
I think that’s compatible with my models, because those meditators still have a cortex, in which patterns of neurons can be firing or not firing at any particular time. And that’s the core aspect of the “territory” which corresponds to “conscious awareness” in the “map”. No amount of meditation, drugs, etc., can change that.
Fair enough, but I think it does show that free will isn’t that central of a piece.
The way intuitive models work (I claim) is that there are concepts, and associations / implications / connotations of those concepts. There’s a core intuitive concept “carrot”, and it has implications about shape, color, taste, botanical origin, etc. And if you specify the shape, color, etc. of a thing, and they’re somewhat different from most normal carrots, then people will feel like there’s a question “but now is it really a carrot?” that goes beyond the complete list of its actual properties. But there isn’t, really. Once you list all the properties, there’s no additional unanswered question. It just feels like there is. This is an aspect of how intuitive models work, but it doesn’t veridically correspond to anything of substance.
Mhhhmhh. Let me see if I can work with the carrot example to where it fits my view of the debate.
A botanist is charged with filling a small field with plants, any plants. A chemist hands him a perfect plastic replica of a carrot, perfect in shape, color, texture, and (miraculously) taste. The botanist says that it’s not a plant. The chemist, who has never seen plants other than carrots, points out the matching qualities to the plants he knows. The botanist says okay but those are just properties that a particular kind of plant happens to have, they’re not the integral property of what makes something a plant. “The core intuitive concept ‘plant’ has implications about shape, color, texture, taste, et cetera”, says the chemist. “If all those properties are met, people may think there’s an additional question about the true plant-ness of the object, but [...].” The botanist points out that he is not talking about an intangible, immeasurable, or non-physical property but rather about the fact that this carrot won’t grow and spread seeds when planted into the earth. The chemist, having conversed extensively with people who define plants primarily by their shape, color, texture, and taste (which are all those of carrots because they’ve also not seen other plants) just sighs, rolling his eyes at the attempt to redefine plant-ness to be entirely about this one obscure feature that also just happens to be the most difficult one to test.
Which is to say that I get—or at least I think I get—the sense that we’re successfully explaining important features of consciousness and the case for linking it to anything special is clearly diminishing—but I don’t think it’s correct. When I say that the hard meta problem of seeing probably contains ~90% of the difficulty of the hard meta problem of consciousness whereas the meta problem of free will contains 0% and the problem of awareness ~2%, then I’m not changing my model in response to new evidence. I’ve always thought Free Will was nonsense!
(The botanist separately points out that there in fact other plants with different shape, texture, and taste, although they all do have green leaves, to which the chemist replies that ?????. This is just to come back to the point that people report advanced meditative states that lose many of the common properties of consciousness, including Free Will, the feeling of having a self (I’ve experienced that one!) and even the presence of any information content whatsoever, and afaik they tend to be more “impressed”, roughly speaking, with consciousness as a result of those experiences, not less.)
[seeing stuff]
Attempt to rephrase: the brain has several different intuitive models in different places. These models have different causal profiles, which explains how they can correspond to different introspective reports. One model corresponds to the person talking about smelling stuff. Another corresponds to the person talking about seeing stuff. Yet another corresponds to the person talking about obtaining vague intuitions about the presence and location of objects. The latter two are triggered by visual inputs. Blindsight turns off the second but not the third.
If this is roughly correct, my response to it is that proposing different categories isn’t enough because the distinction between visually vivid experience and vague intuitions isn’t just that we happen to call them by different labels. (And the analogous thing is true for every other sensory modality, although the case is the least confusing with vision.) Claiming to see a visual image is different from claiming to have a vague intuition in all the ways that it’s different; people claim to see something made out of pixels, which can look beautiful or ugly, seems to have form, depth, spatial location, etc. They also claim to perceive a full visual image constantly, which presumably isn’t possible(?) since it would contain more information than can actually be there, so a solution has to explain how this illusion of having access to so much information is possible. (Is awareness really a serial processor in any meaningful way if it can contain as much information at once as a visual image seems to contain?)
(I didn’t actually intend to get into a discussion about any of this though, I was just using it as a demonstration of why I think the hard metaproblem of consciousness has at least one real subset and hence isn’t empty.)
Hard Problem
Yeah, I mean, since I’m on board with reducing everything to the meta problem, the hard problem itself can just be sidestepped entirely.
But since you brought it up, I’ll just shamelessly use this opportunity to make a philosophical point that I’ve never seen anyone else make, which is that imo the common belief that no empirical data can help distinguish an illusionist from a realist universe… is actually false! The reason is that consciousness is a high-level phenomenon in the illusionist universe and a low phenomenon in at least some versions of the realist universe, and we have different priors for how high-level vs. low-level phenomena behave.
The analogy I like is, imagine there’s a drug that makes people see ghosts, and some think these ghosts tap into the fundamental equations of physics, whereas others think the brain is just making stuff up. One way you can go about this is to have a thousand people describe their ghosts in detail. If you find that the brightness of hallucinated ghosts is consistently proportional to their height, then you’ve pretty much disproved the “the brain is just making stuff up hypothesis”. (Whereas if you find no such relationships, you’ve strengthened the hypothesis.) This is difficult to operationalize for consciousness, but I think determining the presence of absence of elegant mathematical structure within human consciousness is, at least in principle, an answer to the question of “[w]hat would progress on the ‘breathes fire’ question even look like”.
I think this post fails as an explanation of equanimity. Which, of course, is dependent on my opinion about how equanimity works, so you have a pretty easy response of just disputing that the way I think equanimity works is correct. But idk what to do about this, so I’ll just go ahead with a critique based on how I think equanimity works. So I’d say a bunch of things:
-
Your mechanism describes how PNSE or equanimity leads to a decrease in anxiety via breaking the feedback loop. But equanimity doesn’t actually decrease the severity of an emotion, it just increases the valence! It’s true that you can decrease the emotion (or reduce the time during which you feel it), but imE this is an entirely separate mechanism. So between the two mechanisms of (a) decreasing the duration of an emotion (presumably by breaking the feedback loop) and (b) applying equanimity to make it higher valence, I think you can vary each one freely independent of the other. You could do a ton of (a) with zero (b), a ton of (b) with zero (a), a lot of both, or (which is the default state) neither.
-
Your mechanism mostly applies to mental discomfort, but equanimity is actually much easier to apply to physical pain. You can also apply it to anxiety, but it’s very hard. I can reduce suffering from moderately severe physical pain on demand (although there is very much a limit) and ditto with itching sensations, but I’m still struggling a lot with mental discomfort.
-
You can apply equanimity to positive sensations and it makes them better! This is a point I’d emphasize the most because imo it’s such a clear and important aspect of how equanimity works. One of the ways to feel really really good is to have a pleasant sensation, like listening to music you love, and then applying maximum equanimity to it. I’m pretty sure you can enter the first jhana this way (although to my continuous disappointment I’ve never managed to reach the first jhana with music, so I can’t guarantee it.)
… actually, you can apply equanimity to literally any conscious percept. Like literally anything; you can apply equanimity to the sense of space around you, or to the blackness in your visual field, or to white noise (or any other sounds), or to the sensation of breathing. The way to do this is hard to put into words (similar to how an elementary motor command like lifting a finger is hard to put into words); the way it’s usually described is by trying to accept/not fight a sensation. (Which imo is problematic because it sounds like equanimity means stopping to do something, when I’m pretty sure it’s actively doing something. Afaik there are ~zero examples of animals who learn to no longer care about pain, so it very much seems like the default is that pain is negative valence, and applying equanimity is an active process that increases valence.)
I mean again, you can just say you’ve talked about something else using the same term, but imo all of the above are actually not that difficult to verify. At least for me, it didn’t take me that long to figure out how to apply equanimity to minor physical pain, and from there, everything is just a matter of skill to do it more—it’s very much a continuous scale of being able to apply more and more equanimity, and I think the limit is very high—and of realizing that you can just do same thing wrt sensations that don’t have negative valence in the first place.
-
After finishing the sequence, I’m in the odd position where most of my thoughts aren’t about the sequence itself, but rather about why I think you didn’t actually explain why people claim to be conscious. So it’s strange because it means I’m gonna talk a whole bunch about what you didn’t write about, rather than what you did write about. I do think it’s still worth writing this comment, but with the major disclaimer/apology that I realize most of this isn’t actually a response to the substance of your arguments.
First to clarify, the way I think about this is that there’s two relevant axes along which to decompose the problem of consciousness:
the easy vs. hard axis, which is essentially about the describing the coarse functional behavior vs. why it exists at all; and
the [no-prefix] vs. meta axis, which is about explaining the thing itself vs. why people talk about the thing. So for every , the meta problem of is “explain why people talk about ”
(So this gives four problems: the easy problem, the hard problem, the easy meta problem, and the hard meta problem.)
I’ve said in this comment that I’m convinced the meta problem is sufficient to solve the entire problem. And I very much stand by that, so I don’t think you have to solve the hard problem—but you do have to solve the hard meta problem! Like, you actually have to explain why people claim to be conscious, not just why they report the coarse profile of functional properties! And (I’m sure you see where this is going), I think you’ve only addressed the easy meta problem throughout this sequence.
Part of the reason why this is relevant is because you’ve said in your introductory post that you want to address this (which I translate to the meta problem in my terminology):
STEP 1: Explain the chain-of-causation in the physical universe that leads to self-reports about consciousness, free will, etc.—and not just people’s declarations that those things exist at all, but also all the specific properties that people ascribe to those things.
Imo you actually did explain why people talk about free will,[1] so you’ve already delivered on at least half of this. Which is just to say that, again, this is not really a critique, but I do think it’s worth explaining why I don’t think you’ve delivered on the other half.
Alright, so why do I think that you didn’t address the hard meta problem? Well, post #2 is about conscious awareness so it gets the closest, but you only really talk about how there is a serial processing stream in the brain whose contents roughly correspond to what we claim is in awareness—which I’d argue is just the coarse functional behavior, i.e., the macro problem. This doesn’t seem very related to the hard meta problem because I can imagine either one of the problems not existing without the other. I.e., I can imagine that (a) people do claim to be conscious but in a very different way, and (b) people don’t claim to be conscious, but their high-level functional recollection does match the model you describe in the post. And if that’s the case, then by definition they’re independent.
A possible objection to the above would be that the hard and easy meta problem aren’t really distinct—like, perhaps people do just claim to be conscious because they have this serial processing stream, and attempts to separate the two are conceptually confused...
… but I’m convinced that this isn’t true. One reason is just that, if you actually ask camp #2 people, I think they’ll tell you that the problem isn’t really about the macro functional behavior of awareness. But the more important reason is the hard meta problem can be considered in just a single sensory modality! So for example, with vision, there’s the fact that people don’t just obtain intangible information about their surroundings but claim to see continuous images.
Copying the above terminology, we could phrase the hard problem of seeing as explaining why people see images, and the hard meta problem of seeing as explaining why people claim to see images.[2] (And once again, I’d argue it’s fine/sufficient to only answer the meta problem—but only if you do, in fact, answer the meta problem!) Then since the hard meta problem of seeing is a subset of the hard meta problem of consciousness, and since the contents of your post very much don’t say anything about this, it seems like they can’t really have conclusively addressed the hard meta problem in general.
Again, not really a critique of the actual posts; the annoying thing for me is just that I think the hard meta problem is where all the juicy insights about the brain are hidden, so I’m continuously disappointed that no one talks about it. ImE this is a very consistent pattern where whenever someone says they’ll talk about it, they then end up not actually talking it, usually missing it even more than you did here (cough Dennett cough). Actually there is at least one phenomenon you do talk about that I think is very interesting (namely equanimity), but I’ll make a separate comment for that.
- ↩︎
Alas I don’t view Free Will as related to consciousness. I understand putting them into the same bucket of “intuitive self-models with questionable veridicality”. But the problem is that people who meditate—which arguably is like paying more attention—tend to be less likely to think Free Will is real, but I’d strongly expect that they’re more likely to say that consciousness is real, rather than less. (GPT-4 says there’s no data on this; would be very interesting to make a survey correlating camp#1 vs. camp#2 views by how much someone has meditated, though proving causation will be tricky.) If this is true, imo they don’t seem to belong into the same category.
- ↩︎
Also, I think the hard meta problem of seeing has the major advantage that people tend to agree it’s real—many people claim not to experience any qualia, but everyone seems to agree that they seem to see images. Basically I think talking about seeing is just a really neat way to reduce conceptual confusion while retaining the hard part of the problem. And then there’s also blindsight where people claim not to see and retain visual processing capabilities—but much very much reduced capabilities! -- so there’s some preliminary evidence that it’s possible to tease out the empirical/causal effects of the hard meta problem.
Feeling better about this prediction now fwiw. (But I still don’t want to justify this any further since I think progress toward AGI bad and LLMs little progress toward AGI, and hence more investment into LLMs probably good.)
You should be good (though I have only bet once; haven’t withdrawn yet, so can’t guarantee it). I think the gist of it is that Polymarket uses layer 2 and so is cheaper.
Feels empirically true. I remember cases where I thought about a memory and was initially uncertain about some aspect of it, but then when I think about it later it feels either true or false in my memory, so I have to be like, “well no, I know that I was 50⁄50 on this when the memory was more recent, so that’s what my probability should be now, even though it doesn’t feel like it anymore”.
Seems like the fact that I put about a 50% probability on the thing survived (easy/clear enough to remember), but the reasons did not, so the probability no longer feels accurate.
That’s what I mean (I’m talking about the input/output behavior of individual neurons).
Ah, I see. Nvm then. (I misunderstood the previous comment to apply to the entire brain—idk why, it was pretty clear that you were talking about a single neuron. My bad.)
Nice; I think we’re on the same page now. And fwiw, I agree (except that I think you need just a little more than just “fire at the same time”). But yes, if the artificial neurons affect the electromagnetic field in the same way—so not only fire at the same time, but with precisely the same strength, and also have the same level of charge when they’re not firing—then this should preserve both communication via synaptic connections and gap junctions, as well as any potential non-local ephaptic coupling or brain wave shenanigans, and therefore, the change to the overall behavior of the brain will be so minimal that it shouldn’t affect its consciousness. (And note that concerns the brain’s entire behavior, i.e., the algorithm it’s running, not just its input/output map.)
If you want to work more on this topic, I would highly recommend trying to write a proof for why simulations of humans on digital computers must also be conscious—which, as I said in the other thread, I think is harder than the proof you’ve given here. Like, try to figure out exactly what assumptions you do and do not require—both assumptions about how consciousness works and how the brain works—and try to be as formal/exact as possible. I predict that actually trying to do this will lead to genuine insights at unexpected places. No one has ever attempted this on LW (or at least there were no attempts that are any good),[1] so this would be a genuinely novel post.
- ↩︎
I’m claiming this based on having read every post with the consciousness tag—so I guess it’s possible that someone has written something like this and didn’t tag it, and I’ve just never seen it.
- ↩︎
The fact that it’s so formalized is part of the absurdity of IIT. There are a bunch of equations that are completely meaningless and not based in anything empirical whatsoever. The goal of my effort with this proof, regardless of whether there is a flaw in the logic somewhere, is that I think if we can take a single inch forward based on logical or axiomatic proofs, this can begin to narrow down our sea of endless speculative hypotheses, then those inches matter.
I’m totally on board with everything you said here. But I didn’t bring up IIT as a rebuttal to anything you said in your post. In fact, your argument about swapping out neurons specifically avoids the problem I’m talking about in this above comment. The formalism of IIT actually agrees with you that swapping out neurons in a brain doesn’t change consciousness (given the assumptions I’ve mentioned in the other comment)!
I’ve brought up IIT as a response to a specific claim—which I’m just going state again since I feel like I keep getting misunderstood as making more vague/general claims than I’m in fact making. The claim (which I’ve seen made on LW before) is that we know for a fact that a simulation of a human brain on a digital computer is conscious because of the Turing thesis. Or at least, that we know this for a fact if we assume some very basic things about the universe like laws of physics are complete and functionalism is true. So like, the claim is that every theory of consciousness that agrees with these two premises also states that a simulation of a human brain has the same consciousness as that human brain.
Well, IIT is a theory that agrees with both of these premises—it’s a functionalist proposal that doesn’t postulate any violation to the laws of physics—and it says that simulations of human brains have completely different consciousness than human brains themselves. Therefore, the above claim doesn’t seem true. This is my point; no more, no less. If there is a counter-example to an implication , then the implication isn’t true; it doesn’t matter if the counter-example is stupid.
Again, does not apply to your post because you talked about swapping neurons in a brain, which is different—IIT agrees with your argument but disagrees with green_leaf’s argument.
I don’t see how it’s an assumption. Are we considering that the brain might not obey the laws of physics?
If you consider the full set of causal effects of a physical object, then the only way to replicate those exactly is with the same object. This is just generally true; if you change anything about an object, this has changes to the particle structure, and that comes with measurable changes. An artificial neuron is not going to have exactly 100% the same behavior as a biological neuron.
This is why I made the comment about the plank of wood—it’s just to make the point that, in general, across all physical processes, substrate is causally relevant. This is a direct implication of the laws of physics; every particle has a continuous effect that depends on its precise location, any two objects have particles in different places, so there is no such thing as having a different object that does exactly the same thing.
So any step like “we’re going to take out this thing and then replace it with a different thing that has the same behavior” makes assumptions about the structure of the process. Since the behavior isn’t literally the same, you’re assuming that the system as a whole is such that the differences that do exist “fizzle out”. E.g., you might assume that it’s enough to replicate the changes to the flow of current, whereas the fact the new neurons have a different mass will fizzle out immediately and not meaningfully affect the process. (If you read my initial post, this is what I was getting at with the abstraction description thing; I was not just making a vague appeal to complexity.)
it seemed that part of your argument is that the neuron black box is unimplementable
Absolutely not; I’m not saying that any of these assumptions are wrong or even hard to justify. I’m just pointing out that this is, in fact, an assumption. Maybe this is so pedantic that it’s not worth mentioning? But I think if you’re going to use the word proof, you should get even minor assumptions right. And I do think you can genuinely prove things; I’m not in the “proof is too strong a word for anything like this” camp. So by analogy, if you miss a step in a mathematical proof, you’d get points deducted even if the thing you’re proving is still true, and even if the step isn’t difficult go get right. I really just want people to be more precise when they discuss this topic.
Also, here’s a sufficient reason why this isn’t true. As far as I know, Integrated Information Theory is currently the only highly formalized theory of consciousness in the literature. It’s also a functionalist theory (at least according to my operationalization of the term.) If you apply the formalism of IIT, it says that simulations on classical computers are minimally conscious at best, regardless of what software is run.
Now I’m not saying IIT is correct; in fact, my actual opinion on IIT is “100% wrong, no relation how consciousness actually works”. But nonetheless, if the only formalized proposal for consciousness doesn’t have the property that simulations preserve consciousness, then clearly the property is not guaranteed.
So why does IIT not have this property? Well because IIT analyzes the information flow/computational steps of a system—abstracting away the physical details, which is why I’m calling it functionalist—and a simulation of a system performs completely different computational steps than the original system. I mean it’s the same thing I said in my other reply; a simulation does not do the same thing as the thing it’s simulating, it only arrives at the same outputs, so any theory looking at computational steps will evaluate them differently. They’re two different algorithms/computations/programs, which is the level of abstraction that is generally believed to matter on LW. Idk how else to put this.
Well, this isn’t the assumption, it’s the conclusion (right or wrong). It appears from what I can tell is that the substrate is the firing patterns themselves.
You say “Now, replace one neuron with a functionally identical unit, one that takes the same inputs and fires the same way” and then go from there. This step is where you make the third assumption, which you don’t justify.
I think focusing on the complexity required by the replacement neurons is missing the bigger picture.
Agreed—I didn’t say anything that complexity itself is a problem, though, I said something much more specific.
(that means a classical computer can run software that acts the same way).
No. Computability shows that you can have a classical computer that has the same input/output behavior, not that you can have a classical computer that acts the same way. Input/Output behavior is generally not considered to be enough to guarantee same consciousness, so this doesn’t give you what you need. Without arguing about the internal workings of the brain, a simulation of a brain is just a different physical process doing different computational steps that arrives at the same result. A GLUT (giant look-up table) is also a different physical process doing different computational steps that arrives at the same result, and Eliezer himself argued that GLUT isn’t conscious.
The “let’s swap neurons in the brain with artificial neurons” is actually a much better argument than “let’s build a simulation of the human brain on a different physical system” for this exact reason, and I don’t think it’s a coincidence that Eliezer used the former argument in his post.
As other people have said, this is a known argument; specifically, it’s in The Generalized Anti-Zombie Principle in the Physicalism 201 series. From the very early days of LessWrong
Albert: “Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules.”
I think this proof relies on three assumptions. The first (which you address in the post) is that consciousness must happen within physics. (The opposing view would be substance dualism where consciousness causally acts on physics from the outside.) The second (which you also address in the post) is that consciousness and reports about consciousness aren’t aligned by chance. (The opposing view would be epiphenomenalism, which is also what Eliezer trashes extensively in this sequence.)
The third assumption is one you don’t talk about, which is that switching the substrate without affecting behavior is possible. This assumption does not hold for physical processes in general; if you change the substrate of a plank of wood that’s thrown into a fire, you will get a different process. So the assumption is that computation in the brain is substrate-independent, or to be more precise, that there exists a level of abstraction in which you can describe the brain with the property that the elements in this abstraction can be implemented by different substrates. This is a mouthful, but essentially the level of abstraction would be the connectome—so the idea is that you can describe the brain by treating each neuron as a little black box about which you just know its input/output behavior, and then describe the interactions between those little black boxes. Then, assuming you can implement the input/output behavior of your black boxes with a different substrate (i.e., an artificial neuron), you can change the substrate of the brain while leaving its behavior intact (because both the old and the new brain “implement” the abstract description, which by assumption captures the brain’s behavior).
So essentially you need the neuron doctirne to be true. (Or at least the neuron doctrine is sufficient for the argument to work.)
If you read the essay, Eliezer does mostly acknowledge this assumption. E.g., he talks about neuron’s local behavior, implying that the function of a neuron in the brain is entirely about its local behavior (if not, this makes the abstract description more difficult at least, may or may not still be possible).
He also mentions the quantum gravity bit with Penrose, which is one example of how the assumption would be false, although probably a pretty stupid one. Something more concerning may be ephaptic coupling, which are non-local effects of neurons. Are those copied by artificial neurons as well? If you want to improve upon the argument, you could discuss the validity of those assumptions, i.e., why/how we are certain that the brain can be fully described as a graph of modular units.
(Also, note that the argument as you phrase it only proves that you can have a brain-shaped thing with different neurons that’s still conscious, which is slightly different from the claim that a simulation of a human on a computer would be conscious. So if the shape of the brain plays a role for computation (as e.g. this paper claims), then your argument still goes through but the step to simulations becomes problematic.)
I think the burden of proof goes the other way? Like, the default wisdom for polling is that each polling error[1] is another sample from a distribution centered around 0. It’s not very surprising that it output a R bias twice in a row (even if we ignore the midterms and assume it was properly twice in a row). It’s only two samples! That happens all the time.
If you want a positive argument: pollsters will have attempted to correct mistakes, and if they knew that there would be an R/D bias this time, they’d adjust in the opposite way, hence the error must be unpredictable.
- ↩︎
That is, for a smart polling average; individual polls have predictable bias.
- ↩︎
Strong upvote because I literally wanted to write a quick take saying the same thing and then forgot (and since then the price has moved down even more).
I don’t think the inefficiency is as large as in 2020, but like, I still think the overall theme is the same—the theme being that the vibes are on the R side. The polling errors in 2016 and 2020 just seemed to have traumatized everyone. So basically if you don’t think the vibes are tracking something real—or in other words, if you think the polling error in 2024 remains unpredictable / the underlying distribution is unbiased—then the market is mispriced and there’s a genuine exploit.
Fwiw I was too much of a coward/too conflict averse to say anything myself but I agree with this critique. (As I’ve said in my post I think half of all people are talking about mostly the same thing when they say ‘consciousness’, which imo is fully compatible with the set of responses Andrew listed in his post given how they were collected.)