I’ve read (almost all of) this post and I believe in an objective utility function of the universe, so let me try to respond. I’m happy defending wireheading as an outcome even though no-one is seriously advocating for it.
Let’s start by questioning the very concept of “happiness” or “utility”. [...] But that doesn’t mean that there is an objective way to always precisely measure everything in such terms: the notion is fuzzy and uncertain; ascribing measures is costly and imprecise, totally subjective, and relative to a narrow choice at hand. Even under the best hypotheses (von Neumann Morgenstern utility), it is only defined up to a costly measure and an affine transformation — and what more every measure modifies the system being observed. It certainly isn’t a well-defined number easily accessible through introspection, even less so through external observation. Here’s for one individual’s “utility”.
The problem here is that [the fact that we don’t have the technology to measure utility yet] doesn’t mean that it doesn’t exist.
If The Symmetry Theory of Valence (STV) is true, then the bolded part is simply wrong. Such a number exists and it would probably be accessible through external observation. And if it can’t be assessed exactly, then this is only a problem insofar as the measurement is flawed; if it’s off by 5%, this makes wireheading at most 5% worse, which doesn’t change much about it being a desirable outcome.
Given the STV and something like panpsychism, the next two chapters of your post become meaningless because ‘individual’ is not even a relevant abstraction. Consciousness and valence is just a thing fundamental to the universe, there is nothing special about heaps of atoms that form an individual. You have this mathematical representation that, insofar as we can derive it, tells us precisely how much valence there is for any heap of particles we apply it to.
Unfortunately, your post doesn’t really include arguments why the Symmetry Theory of Valence wouldn’t be true. And note that this is just an example of a particular theory; Jeff might not even be aware of it but believe that utility or valence can be precisely measured in some other way.
A lot of your post is just pointing out how the premise yields to unintuitive conclusions, which I agree with.
Indeed, from this substrate independence of consciousness and consequently of happiness, he concludes that a bone and flesh wirehead is no better than an emulated wirehead in a physical simulation of the universe, or in a properly optimized abstraction thereof. Once you have captured the simulation blueprints of a “maximally happy moment” (by unit cost of replication by the wirehead maximizer), be it a junkie at the top of his high, a rapist ejaculating inside a helpless high-profile victim, a mass-murdering tyrant getting a kick out of slaughtering a billion victims, or the three combined at once, then you can run this maximally blissful moment in a closed loop, and replicate that loop identically on gazillions of computers, each a paperclip to our paperclip multiplier.
Now, a basic principle of software abstraction and optimization is that programs that have identical input, output and side-effects are indistiguishable and that one can replace the other. Since this simulation in a closed loop has no input, no output and no side-effect, then it is indistinguishable from the empty program that does nothing. You don’t even need to run the simulation, or to capture it for that matter; doing nothing is already a valid implementation of the paperclip multiplier to unreachable transfinite heights, far beyond what any physical implementation thereof could even conceivably dream of achieving.
This is basically an argument against Functionalism, by which in the idea that consciousness can be described in terms of inputs and outputs only. I agree that this is definitely not true.
Beyond that, it doesn’t show much because an emulation of a brain can’t just replicate the input/output behavior. At least, it can implement the same computation, which some believe would be enough to produce the same conscious states. You might also be a physicalist and think that this is not enough and what matters is what the particles do. Either way, what you said isn’t an argument against happiness out of simulations. In theory, you could run a simulation such that the atoms produce positive valence.
That’s all I have to say on object-level critiques. Note that I don’t know much about consciousness research or anything. Insofar as understanding it requires understanding physics, I’m totally out of my depth because I don’t understand much about physics and am not interested in learning it.
Now, let me go meta.
Once you have captured the simulation blueprints of a “maximally happy moment” (by unit cost of replication by the wirehead maximizer), be it a junkie at the top of his high, a rapist ejaculating inside a helpless high-profile victim, a mass-murdering tyrant getting a kick out of slaughtering a billion victims, or the three combined at once, then you can run this maximally blissful moment in a closed loop, and replicate that loop identically on gazillions of computers, each a paperclip to our paperclip multiplier.
I think this sentence is extremely bad and a good example of what’s wrong with this post.
It’s emotionally manipulative by choosing the rapist example. This is not an ok thing to do on LessWrong and will get you downvoted. Pointing out that wireheading has unintuitive consequences is fine, but once you’ve done that, using it to evoke a feeling of disgust in this way is poor sportsmanship.
It’s redundant; you could cut all three examples and the point would still stand. Certainly, you could cut two of them.
It’s an extremely long sentence. I strongly get the impression that you’re getting a lot of enjoyment out of the sounds your words make. This is dangerous because it can make you optimize for the wrong thing (i.e., sounding good rather than making sense & being easily understandable).
Indeed, there is no question that Jeff is much more intelligent than most (I would notably wager that he’s more intelligent than I, as measurable by an IQ test), and much more knowledgeable of this topic than anyone; few could oppose him a rational refutation like I’m going to offer (no false modesty from me here); yet one need not have identified the precise mistakes of a demonstration to reject its provably absurd conclusions.
yuck!
Though I don’t think my overall opinion of this post would be much different if this weren’t there.
The STV supposes that pleasantness is valuable independently from the agent’s embedding in reality—thus is a Pixie Dust Theory of Happiness, that I indeed argue against in my essay (see section “A Pixie Dust Theory of Happiness”).
While the examples and repetition used in the paragraph cited are supposed to elicit a strong emotion, the underlying point holds: If you’re trying to find the most emotional happiness intensive moment to reproduce, a violent joyful emotion from an insane criminal mastermind is more likely to be it than a peaceful zen moment by a mellow sage. The extreme negative cost to the victims, however great, is in this hypothesis only accounted once; it is thus dwarfed by the infinitely-replicated benefit to the criminal.
Emotions are a guide. You ought to feel them, and if they’re wrong, you ought to explain them away, not ignore them. But, especially in an already long essay, it’s easier and more convincing to show than to explain. If mass murder in the name of wireheading feels deeply wrong, that’s a very strongly valid argument that indeed it is. Maybe I should update the essay to add this very explanation right afterwards.
Admittedly, my essay may not optimized to the audience of LessWrong, but that’s my first couple essays, optimized for my preexisting audience. I wanted to share it here because of the topic, which is extremely relevant to LessWrong.
Finally I’ll reply to meta with meta: if you are “totally out of [your] depth… and am not interested in learning it”, that’s perfectly fine, but then you should disqualify yourself from having an opinion on an “objective utility function of the universe” that you start by claiming you believe in, when you later admit that understanding one issue depends on understanding the other. Or maybe you somehow have an independent proof using a completely different line of argument that makes you confident enough not to look at my argument—in which case you should express more sympathy towards those who’d dismiss Jeff’s argument as insane without examining his in detail.
I’ve read (almost all of) this post and I believe in an objective utility function of the universe, so let me try to respond. I’m happy defending wireheading as an outcome even though no-one is seriously advocating for it.
The problem here is that [the fact that we don’t have the technology to measure utility yet] doesn’t mean that it doesn’t exist.
If The Symmetry Theory of Valence (STV) is true, then the bolded part is simply wrong. Such a number exists and it would probably be accessible through external observation. And if it can’t be assessed exactly, then this is only a problem insofar as the measurement is flawed; if it’s off by 5%, this makes wireheading at most 5% worse, which doesn’t change much about it being a desirable outcome.
Given the STV and something like panpsychism, the next two chapters of your post become meaningless because ‘individual’ is not even a relevant abstraction. Consciousness and valence is just a thing fundamental to the universe, there is nothing special about heaps of atoms that form an individual. You have this mathematical representation that, insofar as we can derive it, tells us precisely how much valence there is for any heap of particles we apply it to.
Unfortunately, your post doesn’t really include arguments why the Symmetry Theory of Valence wouldn’t be true. And note that this is just an example of a particular theory; Jeff might not even be aware of it but believe that utility or valence can be precisely measured in some other way.
A lot of your post is just pointing out how the premise yields to unintuitive conclusions, which I agree with.
This is basically an argument against Functionalism, by which in the idea that consciousness can be described in terms of inputs and outputs only. I agree that this is definitely not true.
Beyond that, it doesn’t show much because an emulation of a brain can’t just replicate the input/output behavior. At least, it can implement the same computation, which some believe would be enough to produce the same conscious states. You might also be a physicalist and think that this is not enough and what matters is what the particles do. Either way, what you said isn’t an argument against happiness out of simulations. In theory, you could run a simulation such that the atoms produce positive valence.
That’s all I have to say on object-level critiques. Note that I don’t know much about consciousness research or anything. Insofar as understanding it requires understanding physics, I’m totally out of my depth because I don’t understand much about physics and am not interested in learning it.
Now, let me go meta.
I think this sentence is extremely bad and a good example of what’s wrong with this post.
It’s emotionally manipulative by choosing the rapist example. This is not an ok thing to do on LessWrong and will get you downvoted. Pointing out that wireheading has unintuitive consequences is fine, but once you’ve done that, using it to evoke a feeling of disgust in this way is poor sportsmanship.
It’s redundant; you could cut all three examples and the point would still stand. Certainly, you could cut two of them.
It’s an extremely long sentence. I strongly get the impression that you’re getting a lot of enjoyment out of the sounds your words make. This is dangerous because it can make you optimize for the wrong thing (i.e., sounding good rather than making sense & being easily understandable).
As someone with strong status-regulating emotions, I also bulk at sentences like this
yuck!
Though I don’t think my overall opinion of this post would be much different if this weren’t there.
The STV supposes that pleasantness is valuable independently from the agent’s embedding in reality—thus is a Pixie Dust Theory of Happiness, that I indeed argue against in my essay (see section “A Pixie Dust Theory of Happiness”).
While the examples and repetition used in the paragraph cited are supposed to elicit a strong emotion, the underlying point holds: If you’re trying to find the most emotional happiness intensive moment to reproduce, a violent joyful emotion from an insane criminal mastermind is more likely to be it than a peaceful zen moment by a mellow sage. The extreme negative cost to the victims, however great, is in this hypothesis only accounted once; it is thus dwarfed by the infinitely-replicated benefit to the criminal.
Emotions are a guide. You ought to feel them, and if they’re wrong, you ought to explain them away, not ignore them. But, especially in an already long essay, it’s easier and more convincing to show than to explain. If mass murder in the name of wireheading feels deeply wrong, that’s a very strongly valid argument that indeed it is. Maybe I should update the essay to add this very explanation right afterwards.
Admittedly, my essay may not optimized to the audience of LessWrong, but that’s my first couple essays, optimized for my preexisting audience. I wanted to share it here because of the topic, which is extremely relevant to LessWrong.
Finally I’ll reply to meta with meta: if you are “totally out of [your] depth… and am not interested in learning it”, that’s perfectly fine, but then you should disqualify yourself from having an opinion on an “objective utility function of the universe” that you start by claiming you believe in, when you later admit that understanding one issue depends on understanding the other. Or maybe you somehow have an independent proof using a completely different line of argument that makes you confident enough not to look at my argument—in which case you should express more sympathy towards those who’d dismiss Jeff’s argument as insane without examining his in detail.