A Refutation of (Global) “Happiness Maximization”
NB: This is a follow-up to my previous essay The Criminal Stupidity of Intelligent People, as cross-posted to my blog: http://fare.livejournal.com/168562.html.
As promised, here is a detailed rational refutation of the utilitarian concept of (Global) “Happiness Maximization” in general, and its specific variant that concludes in favor of wireheading, the cultivation of individuals in vats in a permanent state of artificially stimulated bliss.
Wireheading
To recall the background, I recently had a discussion with a very intelligent and extremely well-meaning colleague of mine, Jeff Kaufman, who, as an aside during an evening he organized to foster efficient philanthropy, was seriously defending the utilitarian ideal of “Happiness Maximization” to its wantonly absurd consequence, wireheading: putting humans, by force if needs be, into vats, where the pleasure centers of their brains will be constantly stimulated for lifelong bliss through properly implanted electric wires and/or chemical injections. Or perhaps instead of humans, the Utilitarian Elite would use rats, or ants, or some brain cell cultures or perhaps nano-electronic simulations of such electro-chemical stimulations; in the latter cases, biological humans, being less-efficient forms of happiness substrate, would be done away with or at least not renewed as embodiments of the Holy Happiness to be maximized. He even wrote at least two blog posts on this theme: hedonic vs preference utilitarianism in the Context of Wireheading, Value of a Computational Process.
Failing the Smell Test
Now, in my previous post, The Criminal Stupidity of Intelligent People, I gave several blanket principles by which one might reject such mass-criminal schemes. Indeed, there is no question that Jeff is much more intelligent than most (I would notably wager that he’s more intelligent than I, as measurable by an IQ test), and much more knowledgeable of this topic than anyone; few could oppose him a rational refutation like I’m going to offer (no false modesty from me here); yet one need not have identified the precise mistakes of a demonstration to reject its provably absurd conclusions. Just like you can without looking reject elaborate alleged proofs of the squaring of the circle, you can without looking reject proofs that some higher principle calls for mass murder: that’s the Libertarian principle, recognizing that peace is the first social value, and it depends on mutual respect for other individuals’ life and property. You can also reject any denial of intuition, tradition or common sense that does not first explain why intuition, tradition and common sense work in most usual cases yet not the case that is being considered. That’s the Conservative principle. Finally, you shouldn’t yield to the opinion of any self-styled or even established expert, however intelligent and authoritative they may be, before you’ve pitted the proponents of that opinion against the most intelligent and authoritative opponents of that opinion you can find (who are seldom those that the proponents would choose to debate). That’s the Eristic debate principle. And that’s where I come: to champion the debunking of “Global Happiness Maximization” in general, and “Wireheading” in particular.
Individual Utility
Let’s start by questioning the very concept of “happiness” or “utility”. Certainly, it is sometimes useful when trying to make a decision to measure opportunities and their consequences in terms of ordinal preferences, and even of some scalar utility. But that doesn’t mean that there is an objective way to always precisely measure everything in such terms: the notion is fuzzy and uncertain; ascribing measures is costly and imprecise, totally subjective, and relative to a narrow choice at hand. Even under the best hypotheses (von Neumann Morgenstern utility), it is only defined up to a costly measure and an affine transformation — and what more every measure modifies the system being observed. It certainly isn’t a well-defined number easily accessible through introspection, even less so through external observation. Here’s for one individual’s “utility”.
Given individual (cardinal) utilities or even just individual (ordinal) preference scales, the only objective notion of interpersonal optimization is Pareto-optimality: to respect each person’s property and let them interact through voluntary exchanges and consensual transactions. Any enhancement that respects everyone’s rights and interests is legit and improves each and everyone’s utility. This is the Libertarian principle of progress through peaceful commerce. Anything else that tries to go “beyond” this Pareto-optimization necessarily violates some individual’s rights, and obviously doesn’t improve each and everyone’s utility. Yet that’s what collectivist “utilitarians” claim to do.
Problems with Collective Utility at a Small Scale
Collectivist Utilitarians somehow claim to aggregate individual utility functions into some collective utility function. Now, even assuming each individual’s utilities were somehow well-defined and could be measured or estimated, there is no objective way to do interpersonal utility comparison and establish such an aggregate. Any set of weight you attribute to individual utilities to compute your common scale is purely arbitrary and subjective. Yet, the collectivist utilitarian claims to somehow establish an official common scale that can be agreed upon as “objective”, as opposed to just acknowledging the infinite plurality of ways to place each individual’s utility on a scale, related to each other by how much more or less weight each unified scale gives to each individual’s utility as compared to the other unified scale. In a given small newly formed homogeneous community, there could be a case made that somehow giving “equal” weight as less costly than endlessly trying to argue for each one’s importance; but already this “equality” supposes one possesses a common measure. What is this measure? Time? But you cannot even equate in either subjective or objective effects the same amount of time spent by the same person doing different activities; it makes no sense at all doing such equation between two different persons. Furthermore, not everyone gives the same value to the widely varying amount of whatever available time they have left to live that isn’t already bound to some existing obligations and necessities of life. Money? Assuming the community is advanced enough to possess some currency, indeed you could try to equate each member’s marginal preference for a small (but not too small) amount of currency; that’s indeed what neo-classical economists typically do when they indulge in their kind of calculations. But even then you’d find that this is a very elastic scale, and that it is as “unfair” as it is coarsely-defined: it gives disproportionate weight to lazy and greedy people living in cheap areas with plenty of time on their hand and in their future to spend money as opposed to old hard-working generous people in sophisticated areas. Indeed, whichever way you define your common scale, when you’re trying to apply this scale to very large heterogeneous populations, it will yield absurd results. And even assuming you started with an agreed upon distribution at one point in a small newly formed homogeneous community, your distribution won’t hold water as the community grows, some people leave, others join, babies are born, some people become superproductive pillars of the community while others become criminals negatively affecting it, and every situation in between.Meaninglessness of Collective Utility at a Large Scale
Now, depending on their ambition and their gall, collectivist utilitarians may claim to be able to establish a common utility scale not just over a small group of people for a day, but over an entire country for a year, over humanity for centuries, over all living organisms on the planet for millions of years, and even over all entities in the entire universe for aeons to come. But what common scale can you establish between a few humans who hardly know each other, between billions of humans who live completely different lives and have no clue about each other’s existence, between humans and rats or other mammals, between mammals and insects or microbes, between animals and plants, between biological life forms and electronically simulated life forms, between life forms as we know them and potential future emergent artificial intelligences, between terrestrial life as we may know it and life that may have evolved in completely different ways in other parts of the universe? How can you establish a common scale between entities that live at different places in different times, separated by vast distances and durations? How can you establish a common scale between actual entities that are known to exist, entities that exist but are not known to exist, and infinitely many potential entities that may or may not some day come into being or fail to come into being depending on a large number of unpredictable decisions and unknowable events and their largely unforeseeable consequences? Yet this is the kind of feat that collectivist utilitarians such as Jeff not only assume to be possible, but claim can be the basis for making effective decisions that are to bind each and every one.
Of course, if you take the building of such scales seriously, you’ll find that depending on the fine-tuning of the contributing factors to your scale, you might have to consider such dilemmas as follow: Should we forcefully impregnate women, inconveniencing them somewhat for the sake of the large number of potential unborn happy descendants years from now? Is the fraction of life value associated to ebola virus big enough that we should sacrifice each and every human on the planet for the sake of feeding their bodies to a tremendous number of copies of these valuable life-forms? Should our omniscient economic advisers assign to everyone a positive or negative tax (or subsidy) factor on everything he purchases, perhaps with a different rate on each potential purchase? Or should they directly buy what each of us needs according to their perfect knowledge of our utility? If we count “each man equally” in an open system such as “a country”, shouldn’t we encourage prolific invaders to replace the inhabitants of the country after either slaughtering them or letting them die without children or with few children? Depending on how you compute GNP (or GNH) statistics, you may want to import richer and happier invaders (so they bring their capital), or poorer and unhappier invaders (so their portion of GNx increases most). If you slaughter the original inhabitants, though, you may also want to excommunicate or exterritoriate them before you do them in, so that their suffering does not count negatively for “the country”, before you re-import the loot from their dead bodies. Or maybe GNP and GNH studies are only valid for “closed” systems, and then require some World or Universe Government to oversee everyone, and strictly regulate and control the procreation of new children to maintain a fair approximation of closeness between the initial inhabitants; the only totally fair non-approximate way to keep the system closed would be to strictly forbid children; but if there are no children at all, “in the long run, we’re all dead”. Maybe to be meaningful, the National Utility Scale can only be used by a country’s government at a given date; then to continue action it will have to morph into some kind of government of the descendants of the inhabitants of the country at said date, the authority of which is somehow prorated by their contribution to the subject’s genetic material. Plenty of questions such as these, absurd as they may sound, naturally arise as prerequisites to any further action, when one claims to put Everyone and Everything on a same Utility Scale with an explicit formula.
Assuming Knowledge of the Unknowable
The implicit premise of any discussion about Ethics is that there are choices to be made that haven’t been made, and that the future is therefore unknown, and essentially so. But the implicit premise behind Collectivist Utilitarianism is that there is a knowable objective scale that allows decisions to be made mechanically for everyone. Of course, if the claim is that “there is a scale, but it’s unknowable and we can’t use it”, then it’s not much of a claim, but is more akin to invisible pink unicorns and mystical gods retired from the world. Now if the claim is “we don’t know but we can act as if we knew”, then it’s just a justification for the arbitrary power and dreadful decisions of those entitled to act as if they knew even though they admittedly don’t. In any case, the pretense to sum over unknowable variables is bogus. Inasmuch as it might possibly mean anything, the knowledge of it is out of the reach not just of mere humans but of any finite entity within the universe. The entire project of a global utility calculation is found to be meaningless.
In the end, any concept of utility, happiness or suffering only has a clear or knowable validity but in the context of an individual who makes decisions based on such a concept. Those evaluations that are valid for one individual are both invalid and unknowable for other individuals. These evaluations do not meaningfully generalize to “collectives”, much less to a whole planet, even less across years, and not at all to the entire Universe across future aeons.
Hidden Totalitarian Premise
Furthermore, a damning feature of any recommendation issued in the name of any collective utility is that it presumes totalitarian ownership by the few individuals who may or may not decide based on such recommendations over all the resources of all individuals in the alleged collective that will be somehow shuffled by the decision. Wherever such recommendations depart from those of Pareto Optimality, they require violence toward some parties; such violence implies that the party who imposes the decision either can exercise some tyrannical supremacy, or will experience abject failure in trying to enforce its recommendation. In the first case, this violent tyranny more than offsets the alleged positive consequences of the recommendation; in the second, its failure invalidates the unachieved positive consequences that allegedly justified it. In either case, the denial of Pareto Optimality and Libertarian property rights as a limit to action by some proclaimed Utilitarian Elite ignores the costs of enforcement of such action, that offset any benefits, as showed by the Law of Bitur-Camember. Although really, it’s usually the elite being spiteful rather than ignorant of how victims are sacrificed to fulfill their own personal preferences, plans, and lust for power; and the same elites cultivating ignorance, misunderstanding, apathy and disorganization among the masses upon which they trample.
Back to Individualism
There remains the case where some individual only acts on his own property without initiating conflict with any other individuals, while respecting every one else’s individual property rights, but claims to act in the name and interest of a Collective for which he has established some Collective Utility Scale. We can then start by examining whether the actions of this individual is indeed compatible with said avowed Collective Utility Scale, or if he are just lying to himself or to others; but if he is indeed sincere and coherent, which he may be, that only means that he has some possibly strange but extended notion of himself. Which might not be so strange when the Collective is actually a tight knit family, clan, etc. Everyone living in society has some notion of one’s extended self.
But what is strange and probably mystically deluded is when one claims to act in the interest of the entirety of Mankind, when one in the end is only actually possibly helping but a small number of very specific individuals out of billions of humans; and this delusion becomes blatant evil when under the claim of acting according to some Objective utility scale, one systematically acts to give to people in proportion to the objectively awful criteria that these recipients should be the worst kind of lowlifes, the dumbest, laziest, nastiest and least shameful at demanding money “due” to them.
Even assuming the latter pitfall is avoided, the implicit recognition of the preeminence of the Universal Law of Libertarian Non-Aggression by those who respect it casts a shadow on any attempt to “Maximize Utility” that explicitly ignores or rejects this preeminence in its recommendations: however you account for Mankind as a whole, you’ll find that violations of Universal Law are the one greatest source of dis-Utility in the world, and that as long as such violations casually occur, the greatest possible charitable act may be to act towards the universal recognition and enforcement of this Universal Law.
This seals the notion of a Global Happiness to calculate and maximize for what it is: at best, a long diversion from Pareto Optimality back to Pareto Optimality; usually, a useless unknowable, unactionable fantasy that only serves as the cover for some people to grab the power to decide over others; at worst, a criminal absurdity. There is only one knowable and actionable way to foster Happiness, which is to care for those you love and to uphold the Law by defending victims from their aggressors. Gross National Product, Gross National Happiness, etc., are but pretenses for bureaucrats and politicians and through them lobbyists to steal from confused citizens. And the particular variant of Global Happiness Maximization that recommends Wireheading is but a criminal absurdity. Nevertheless, there are often useful concepts to clarify in examining the crazy fallacies of intelligent people; indeed the same or some other intelligent people may use these fallacies in other contexts and theories as well; therefore let’s examine the notion still.
Wireheads as Paperclips
The implicit hypothesis behind Wireheading is that “happiness” or “utility” consists in an independent electro-chemico-mechanical phenomenon that can be physically isolated within a human’s brain or an animal’s brain; for instance, Jeff seems to equate it with some pleasure center of the brain releasing or receiving dopamine. Once isolated, this “happiness” can be replicated, miniaturized, and mass-produced. Therefore, this whole Wireheading is just another variant of the infamous Paperclip maximizer that anyone in the field is warned about: an imagined runaway AI is tasked with producing paperclips; it ascribes a positive utility to the sheer number of paperclips in the universe, and is so successful at maximizing this utility that it turns everything into paperclips; in particular this includes taking apart the bodies of humans and posthumans to use them as raw material for its goal. Wireheading is similar, except that paperclips have been replaced by wireheads or by whichever streamlined reduction thereof (if any) our pan-philanthropic utilitarian wireheading overlord comes up with.
A Pixie Dust Theory of Happiness
Behind this wireheading proposal, we have therefore identified a pixie dust theory of happiness. A pixie dust theory is one where some phenomenon (in this case happiness), is believed to have its source in some kind of magical compound (in this case, a brain in blissful stimulation, or its isolated reduction to some unstable dopamine-receptor chemical complex), each uniform physical or ethereal amount of which (in this case, physical copies) possesses value in itself that it additively contributes to the whole: the more pixie dust, the more happiness. In a previous essay (fmftcl), I debunked the pixie dust theory of freedom. The pixie dust theory of happiness can be debunked the same way.
Freedom isn’t possibly embodied in quantum wave collapse, elementary particles to be called eleutherons, or magic souls or microsouls that angels mystically attach to the brains of individuals to endow them with free will. Similarly, happiness is not possibly physically embodied in dopamine-receptor reaction complexes, elementary particles to be called hedons, or actual dust sprinkled by invisible pixies on the neurons of some people to make them happy. Instead, both are phenomena that result from the situational interaction of an individual with the rest of universe, (of being “thrown” into the universe, would say Heidegger; a good reading suggestion at this point is “Understanding Computers and Cognition” by Winograd & Flores). More than that, they are functional sub-phenomena of life, i.e. of an organism behaving in such a way as to sustain itself, in a feedback loop with its environment. Freedom is the acknowledgement that the inside of the organism controls its behaviour much more than any other entity outside it; happiness is the judgment regarding the organism achieving its goals, whatever they be.
Neither freedom nor happiness makes any sense for a neuron in a vat, where it doesn’t actually control an organism thrown into the universe, where it doesn’t have meaningful goals to achieve. Its existence, artificially sustained by an outer organism, is a pure waste of resources and of information-processing ability. It is interesting how Jeff, who finds his satisfaction in trying hard to maximize his impact on the world (and indeed speaks of giving efficiently for maximized “impact”), doesn’t realize that happiness means nothing to brains in a vat that can have no impact whatsoever on the world. Trying to isolate these phenomena from the rest of an individual organism’s behavior, what more to “maximize” them afterwards, is absurd on its face — even more so than trying to isolate magnetic monopoles so as to thereafter maximize their intensity. It’s not even impossible: it doesn’t make sense at all.
Emulated Happiness
Jeff recognizes that the substrate of consciousness, and therefore of happiness, need not be biological, but could be electronic, be it through emulation. In this, he is totally right: the ability to process information and act upon it isn’t tied to any specific chemical process, and there could therefore be living organisms, and sentient beings, based on a substrate of electronic computations on silicon transistors rather than on organic chemistry inside and across neurons. But combining this common piece of wisdom with the above pixie dust theory of happiness, he reaches new heights in absurd statements that he cannot get out of.
Indeed, from this substrate independence of consciousness and consequently of happiness, he concludes that a bone and flesh wirehead is no better than an emulated wirehead in a physical simulation of the universe, or in a properly optimized abstraction thereof. Once you have captured the simulation blueprints of a “maximally happy moment” (by unit cost of replication by the wirehead maximizer), be it a junkie at the top of his high, a rapist ejaculating inside a helpless high-profile victim, a mass-murdering tyrant getting a kick out of slaughtering a billion victims, or the three combined at once, then you can run this maximally blissful moment in a closed loop, and replicate that loop identically on gazillions of computers, each a paperclip to our paperclip multiplier.
Now, a basic principle of software abstraction and optimization is that programs that have identical input, output and side-effects are indistiguishable and that one can replace the other. Since this simulation in a closed loop has no input, no output and no side-effect, then it is indistinguishable from the empty program that does nothing. You don’t even need to run the simulation, or to capture it for that matter; doing nothing is already a valid implementation of the paperclip multiplier to unreachable transfinite heights, far beyond what any physical implementation thereof could even conceivably dream of achieving. Jeff, faced with this prospect, tries to somehow demand an actual expensive computation of some physical process, his original pixie dust; somehow you have to disable optimizations and observe the blissful brain in its dopaminergic glory. Jeff obviously hasn’t read Greg Egan’s Permutation City, in which delicious SF novel all kinds of concepts related to the emulation of consciousness are explored, to the point of absurdity — with each reader getting to decide where it starts to be absurd and why.
Of course, even without optimization at the level observed by Jeff, what about optimizations and pessimizations at an intermediate level below? What if a hedonic simulation is run on a redundant computer, where two identical processors in lockstep simultaneously carry the same computation? Does that make the simulation count double or single? What if the redundancy happens in software rather than in hardware? On the contrary, what if a de-duplicating server optimizes zillions of identical computations by running them on one virtual machine instance? What if after the first simulation, the results are cached, and the computation implemented as a virtual lookup, itself optimized away, except when you plug a monitor to observe said results? One way or the other, “linearity” (as in “linear logic”), the resource-like nature of some phenomena, is not preserved by software abstraction in general. If happiness is embodied in some pixie dust, then software emulation is going to do strange things to it incompatible with accounting for utilons. Yet Jeff, a software engineer, doesn’t seem to register.
Terminal Insanity
Jeff falls in the most basic insanity trap, as notably identified by Korzybski’s General Semantics: confusing a thing and its representation. The map is not the territory; the representation of something is not the thing itself; a software emulation of a happy man is not a happy man; the neural switch that triggers a feeling of happiness is not the feeling of happiness; the feeling of happiness in a man itself is but a representation, an internalization, of a situation of happiness, real or not, that if real relates the man to his environment: the goals he is pursuing are reached; his frustrations are overcome; his impact is large; his choices are vindicated; his actions are meaningful; etc.
What makes the mental representation of something worthwhile is its connection to the reality, with the thinking individual choosing how to act within in his real environment based on this representation in his mind. When the representation of reality is inadequate, the mind is mistaken. When the relation to reality is broken, the mind is insane. A concentration camp prisoner injected with drugs inducing a feeling of happiness is not actually happy; his feeling of happiness is not just a mistaken representation, what more it is a representation that has been deliberately corrupted. When there isn’t even a reality to be adequately or inadequately represented, and not even an individual acting in that reality, then there isn’t a mind at all that may be sane or insane; and the person who claims that there is is himself deeply mistaken, if not insane. What makes mental representation meaningful is that it’s actually connected to sensors and actuators, at which point this connection is making it worthwhile, or not; and that’s precisely the opposite of Jeff’s view of utilons as an isolatable physical phenomenon.
Understanding Criminal Altruism
In conclusion, here was another case of very intelligent, well-meaning people making criminal recommendations based on seriously accepting absurd ideas. It is important to recognize these ideas as absurd and these recommendations as criminal; but as said Claude Bernard, it isn’t enough to say: “I was mistaken”; one must say how one was mistaken. Only by analyzing the root of the issue can we possibly prevent more people from following these same ideas, which, if widespread would be used to justify actual crimes. It is important to debunk these ideas, to confront the essential contradictions in their underlying assumptions. And as we do it, we can explore and restore a few interesting concepts that are often overlooked but unlike the debunked ideas can be used for effective ethical decision making.
I think you have misunderstood the genre of some of the conversations you’ve been having. Wireheading is a philosophical thought experiment, not a policy proposal. Getting angry and calling it a “criminal proposal” implies a significant misunderstanding of what is being talked about and what kind of conversation is being had.
Combining this with references to an in-person conversation where it isn’t clear what was said, and links to a few posts that don’t quite match the thing you’re responding to, makes this whole post very confusing. I don’t think I could discuss the topic at the object level without quite a few rounds of clarification first.
If wireheading were a serious policy proposal being actively pursued with non-negligible chances of success, I would be shooting to kill wireheaders, not arguing with them.
I am arguing precisely because Jeff and other people musing about wireheading are not actual criminals—but might inspire a future criminal AI if their argument is accepted.
Arguing about a thought experiment means taking it seriously, which I do. And if the conclusion is criminal, this is an important point that needs to be stated. When George Bernard Shaw calmly claims the political necessity of large scale extermination of people unfit for his socialist paradise, and doing it scientifically, he is not being a criminal—but it is extremely relevant to note that his ideas, if implemented, would be criminal, and that accepting them as true might indeed inspire criminals to act, and inspire good people to let criminals act.
If I am not to take wireheading seriously, there is nothing to argue. Just a good laugh to have.
And I am not angry at all about wireheading. But apparently, the first post of this series made a lot of commenters angry indeed who took it personally.
I’ve read (almost all of) this post and I believe in an objective utility function of the universe, so let me try to respond. I’m happy defending wireheading as an outcome even though no-one is seriously advocating for it.
The problem here is that [the fact that we don’t have the technology to measure utility yet] doesn’t mean that it doesn’t exist.
If The Symmetry Theory of Valence (STV) is true, then the bolded part is simply wrong. Such a number exists and it would probably be accessible through external observation. And if it can’t be assessed exactly, then this is only a problem insofar as the measurement is flawed; if it’s off by 5%, this makes wireheading at most 5% worse, which doesn’t change much about it being a desirable outcome.
Given the STV and something like panpsychism, the next two chapters of your post become meaningless because ‘individual’ is not even a relevant abstraction. Consciousness and valence is just a thing fundamental to the universe, there is nothing special about heaps of atoms that form an individual. You have this mathematical representation that, insofar as we can derive it, tells us precisely how much valence there is for any heap of particles we apply it to.
Unfortunately, your post doesn’t really include arguments why the Symmetry Theory of Valence wouldn’t be true. And note that this is just an example of a particular theory; Jeff might not even be aware of it but believe that utility or valence can be precisely measured in some other way.
A lot of your post is just pointing out how the premise yields to unintuitive conclusions, which I agree with.
This is basically an argument against Functionalism, by which in the idea that consciousness can be described in terms of inputs and outputs only. I agree that this is definitely not true.
Beyond that, it doesn’t show much because an emulation of a brain can’t just replicate the input/output behavior. At least, it can implement the same computation, which some believe would be enough to produce the same conscious states. You might also be a physicalist and think that this is not enough and what matters is what the particles do. Either way, what you said isn’t an argument against happiness out of simulations. In theory, you could run a simulation such that the atoms produce positive valence.
That’s all I have to say on object-level critiques. Note that I don’t know much about consciousness research or anything. Insofar as understanding it requires understanding physics, I’m totally out of my depth because I don’t understand much about physics and am not interested in learning it.
Now, let me go meta.
I think this sentence is extremely bad and a good example of what’s wrong with this post.
It’s emotionally manipulative by choosing the rapist example. This is not an ok thing to do on LessWrong and will get you downvoted. Pointing out that wireheading has unintuitive consequences is fine, but once you’ve done that, using it to evoke a feeling of disgust in this way is poor sportsmanship.
It’s redundant; you could cut all three examples and the point would still stand. Certainly, you could cut two of them.
It’s an extremely long sentence. I strongly get the impression that you’re getting a lot of enjoyment out of the sounds your words make. This is dangerous because it can make you optimize for the wrong thing (i.e., sounding good rather than making sense & being easily understandable).
As someone with strong status-regulating emotions, I also bulk at sentences like this
yuck!
Though I don’t think my overall opinion of this post would be much different if this weren’t there.
The STV supposes that pleasantness is valuable independently from the agent’s embedding in reality—thus is a Pixie Dust Theory of Happiness, that I indeed argue against in my essay (see section “A Pixie Dust Theory of Happiness”).
While the examples and repetition used in the paragraph cited are supposed to elicit a strong emotion, the underlying point holds: If you’re trying to find the most emotional happiness intensive moment to reproduce, a violent joyful emotion from an insane criminal mastermind is more likely to be it than a peaceful zen moment by a mellow sage. The extreme negative cost to the victims, however great, is in this hypothesis only accounted once; it is thus dwarfed by the infinitely-replicated benefit to the criminal.
Emotions are a guide. You ought to feel them, and if they’re wrong, you ought to explain them away, not ignore them. But, especially in an already long essay, it’s easier and more convincing to show than to explain. If mass murder in the name of wireheading feels deeply wrong, that’s a very strongly valid argument that indeed it is. Maybe I should update the essay to add this very explanation right afterwards.
Admittedly, my essay may not optimized to the audience of LessWrong, but that’s my first couple essays, optimized for my preexisting audience. I wanted to share it here because of the topic, which is extremely relevant to LessWrong.
Finally I’ll reply to meta with meta: if you are “totally out of [your] depth… and am not interested in learning it”, that’s perfectly fine, but then you should disqualify yourself from having an opinion on an “objective utility function of the universe” that you start by claiming you believe in, when you later admit that understanding one issue depends on understanding the other. Or maybe you somehow have an independent proof using a completely different line of argument that makes you confident enough not to look at my argument—in which case you should express more sympathy towards those who’d dismiss Jeff’s argument as insane without examining his in detail.
Regarding wireheading, it only just fully occurred to me that whoever has his happiest moment indefinitely duplicated in this Effective Altruist’s pixie dust paperclipping fantasy is literally an infinite utility monster, whose psychopathic enjoyment in abusing others is therefore a big “plus”.
Thus, replicating the triumph of ultimate evil is not just a possibility, but a logical necessity of wireheading: The vilest most selfish junkie-rapist-mass murderer is precisely who will most enjoy having an infinite hedonistic multiplier while everyone else’s is zero.
In the end, inasmuch as their theory leads to wireheading, then the ultimate goal of Effective Altruists is maximal evil, infinitely replicated in a paperclipping of the entire universe.
Wow. Talk about a death cult!
Twelve years on, I edited the upstream article on LiveJournal to reflect this realization.