Free Will as Unsolvability by Rivals
Nadia wanted to solve Alonzo. To reduce him to a canonical, analytic representation, sufficient to reconfigure him at will. If there was a potential Alonzo within potential-Alonzo-space, say, who was utterly devoted to Nadia, who would dote on her and die for her, an Alonzo-solution would make its generation trivial.
from True Names, by Cory Doctorow and Benjamin Rosenbaum
Warning: this post tends toward the character of mainstream philosophy, in that it relies on the author’s intuitions to draw inferences about the nature of reality.
If you are dealing with an intelligence vastly more or less intelligent than yourself, there is no contest. One of you can play the other like tic-tac-toe. The stupid party’s values are simply irrelevant to the final outcome.
If you are dealing with an intelligence extremely close to your own—say, two humans within about five IQ points of each other—then both parties’ values will significantly affect the outcome.
If you are dealing with an intelligence moderately more or less intelligent than yourself, such as a world-class politician or an average eight-year-old child respectively, then the weaker intelligence might be able to slightly affect the outcome.
If we formalize free will as the fact that what we want to do has a causal effect on what we actually do, then perhaps we can characterize the sensation of free will—the desire to loudly assert in political arguments that we have free will—as a belief that our values will have a causal effect on the eventual outcome of reality.
This matches the sense that facing a terrifyingly powerful intelligence, one that can solve us completely, strips away our free will, which in turn probably explains the common misconception that free will is incompatible with reductionism—knowing that an explanation exists feels like having the explanation be known by someone. We don’t want to be understood.
It matches the sense that a person’s free will can be denied by forcing them into a straitjacket and tossing them in a padded cell. It matches the assumption that not having free will would feel like sitting at the wheel of a vehicle that was running on autopilot and refusing manual commands.
In general, we can distinguish three successive stages at which free will can be cut off:
The creature can be constructed non-heuristically to begin with; that is, it lacks a utility function.
The creature can control insufficient resources to be in a winnable state; that is, it is physically helpless.
The creature can be outsmarted; that is, it has a vastly superior opponent.
Probably the last two, and possibly all three, cannot remain cleanly separated under close scrutiny. But the model has such a deep psychological appeal that I think it must be useful somehow, if only as an intermediate step in easing lay folk into compatibilism, or in predicting and manipulating the vast majority of humans that believe or alieve it.
- Free Will as Unsolvability by Rivals by (28 Mar 2011 3:28 UTC; 25 points)
- 's comment on Human errors, human values by (10 Apr 2011 17:49 UTC; 7 points)
- 's comment on Separate morality from free will by (11 Apr 2011 17:07 UTC; 3 points)
- 's comment on Epistle to the New York Less Wrongians by (30 Apr 2011 19:34 UTC; 3 points)
Possible counterexamples: the rabies virus is pretty good at getting mammals to do its bidding. Similarly, the protozoa Toxoplasma gondii, when it infects mice and rats, actually causes the infected rodents to become attracted to, instead of averse to, the scent of cat urine, which assists the parasite in transferring itself to its preferred host.
Both of those seem to fit the pattern perfectly when you consider evolution as an actor.
Maybe we should be discussing optimization power instead of intelligence; evolution seems a pretty decent manipulator considering how stupid it is.
Yes, that’s what I had in mind. Optimization power; strength of causation from desire to effect.
Consider a specific thought experiment where modeling others is important, for example Prisoner’s Dilemma. Even if your opponent is much smarter than you, smart enough to simulate your every thought, you can bargain with them using your access to the outcome-button. You can threaten them with defecting if they think too fast or too difficult for you to follow (in particular, they won’t be able to unconditionally simulate you if they comply). And so you’ll rob them of this first-mover advantage by having your finger on the trigger. Your decision is still your own.
(This idea comes from the discussion of “unintended simulation” thought experiment variant on SIAI decision theory list.)
+1. Look at the reaction (of just about anybody) to being reduced to a “type”, no matter how well it fits.
-- Vernor Vinge, A Fire Upon The Deep (published in April 1992)
I thank you for this new insight, besides general rationality training this is the kind of thing I visit LW for!
This might also explain why one needed something as strong as religion to make hundreds of millions of people to stop nominally believing in Libertarian free will. The number of people’s who ceased to believe in it because of their belief in a omnipotent omniscient “terrifyingly powerful intelligence” dwarfs the number who have done so on materialist grounds.
I only just now found this pithy, relevant quote:
...though of course it would be rude to suggest that the meaning I attribute to it was the intended one.
I agree that the feeling of free will is the mind projection bias, but I think we can make things simpler and less arbitrary by applying it to ourselves rather than rivals. It goes like “Why do I think I have free will? Because I cannot predict what I will do next. Therefore I am unpredictable.” So “feels like free will” would be a property of any chaotic self-reflecting being with the mind projection bias.
This can be extended back to define free will itself with the additional stipulation that free will is defined as the stuff that makes you feel like you have free will. Some people definitely reject this additional stipulation, but I think the other options are silly :P
Pavitra, I’d love to know what you think about my post on free will:
In other words, I think a paperclip maximizer is dangerous because it has more free will, i.e. is free to (not free from) realize what it wants as its effect on the universe is much larger than that of a human(s). An agent’s perception to be free is therefore correlated with the ability to realize its goals, the probability of success.
Your linked post seems to be more about an agent interacting with a dumb-matter environment, and about the relationship between free will and determinism. My post is specifically about what happens when two agents interact with each other. The point I was trying to make is that the sense of indignation that accompanies the intuition of free will is tied to the desire to protect one’s utility function from alteration in the presence of a hostile intelligence.
Your comment bridges the two nicely.