I agree that “think for yourself” is important. That includes updating on the words of the smart thinkers who read a lot of the relevant material. In which category I include Zvi, Eliezer, Nate Soares, Stuart Armstrong, Anders Sandberg, Stuart Russell, Rohin Shah, Paul Chistiano, and on and on.
RickJS
Karma: 74
I will say that .PDF format is end-user hostile.
Thanks, Eliezer!
This one was actually news to me. Separately is more efficient, eh? Hmmm… now I get to rethink my actions.
I had deliberately terminated my donations to charities that seemed closer to “rescuing lost puppies”. I had also given up personal volunteering (I figured out {work—earn—donate} before I heard it here.) And now I’m really struggling with akrasia / procrastination / laziness /rebellion / escapism.
“You could, of course, reply that you don’t trust selfish acts that are supposed to be other-benefiting as an “ulterior motive” ”. That’s a poisonous meme that runs in my brain. But I consciously declare that to be nonsense. I don’t ever want to discuss “pure altruism” ever again! I applaud ulterior motives, “Just so long as people get helped.” If you can figure out your ulterior motives, use them! Put them in harness. You might as well, they aren’t going away.
Thanks, Matt!
That’s a nice educational post.
I want to pick a nit, not with you, but with Gigerenzer and ” … the conjunction fallacy can be mitigated by changing the wording of the question … ” Unfortunately, in real life, the problems come at you the way they do, and you need to learn to deal with it.
I say that rational thinking looks like this: pencil applied to paper. Or a spreadsheet or other decision support program in use. We can’t do this stuff in our heads. At least I can’t. Evolution didn’t deliver arithmetic, much less rationality. We teach arithmetic to kids, slowly and painstakingly. We had better start teaching them rationality. Slowly and painstakingly, not like a 1-hour also-mentioned.
And, since I have my spreadsheet program open, I will indeed convert probabilities into frequencies and look at the world both ways, so my automatic processors can participate. But, I only trust the answers on the screen. My brain lies to me too often.
Once again, thanks Matt. Well done!
Thanks, Eliezer!
That’s good stuff. I really relate to ” … the poisonous meme saying that someone who gives mere money must not care enough to get personally involved.” That one runs on automatic in my head. It’s just one of many ways my brain lies to me.
“Every time I spend money I feel like I’m losing hit points. ” Now, I don’t know your personal situation, and I can certainly relate. My mother is a child of the Great Depression and lived her life out of a fear of poverty. She taught me to worship Bargain and Sale and to abhor “unnecessary” spending.
I suspect that most people make it worse by not saving enough. I suspect most people have only a few months salary in savings. But in many highly-skilled (specialized) professions, it can take years to find your next career job.
Anyway, I made it a life practice, starting in college, to make saving a high priority, and then don’t look at my net wealth often. That was so I didn’t get stress-related diseases from worrying myself sick over money. Once I got “rich” (I retired at age 51), I got a financial planner, set up lots of disparate investments, and I look at my net worth only once a year.
Now it’s a different world, now that I have discovered existential risks. Now I fight with my financial planner to raise my outflow rate, and that goes to the Future of Humanity Institute, Dr. Martin Hellman (see nuclearrisk.org), and mostly to The Institute Which Must Not Be Named. My goal used to be to outlive my money. Now it is Saving Humanity from Homo Sapiens™.
I say that not to brag, but to invite you each to take on an extraordinary mission for your life. This optimal philanthropy thread gives a lot of the practical steps. In the Landmark Education Curriculum for Living™ you will create yourself as an extraordinary person, living your extraordinary commitment. Now that’s cool!
Thanks, Eliezer!
As one of your supporters, I have been sometimes concerned that you are doing blog posts instead of working out the Friendly AI theory. Much more concerned than I show. I do try to hold it down to an occasional straight question, and hold myself back from telling you what to do. The hypothesis that I know better than you is at least −50dB.
This post is yet another glimpse into the Grand Strategy behind the strategy, and helps me dispel the fear from my less-than-rational mind.
I find it unsettling that ” … after years of bogging down I threw up my hands and explicitly recursed on the job of creating rationalists.”
You learned that, “The human brain can’t grasp large stakes and people are not anything remotely like expected utility maximizers, and we are generally altruistic akrasics.” Evolution didn’t deliver rationality any more than it delivered arithmetic. They have to be taught and executed as procedures. They aren’t natural. And I wonder if they can be impressed into System 1 through practice, to become semi-automatic. Right now, my rational side isn’t being successful at getting me to put in 8-hour work days to save humanity.
You learned that, “Dollars come out of different mental accounts, cost different amounts of willpower (the true limiting resource) under different circumstances … ” That makes some of MY screwy behavior start to make sense! It’s much more explanatory than, “I’m cheap!” or lazy, or insensitive, or rebellious, or contrary. That looks to me like a major, practical breakthrough. I will take that to my coach and my therapist, we will use it if we can.
I don’t think my psychologist ever said it. I doubt it is taught in undergraduate Psychology classes. Am I just out of touch ? Has this principle been put into school curricula? That you had to learn it the hard way, that it isn’t just common knowledge about people, ” … paints a disturbing picture.”
You’ve done it again. In a little over one thousand words, you have given me a conceptual tool that makes the world (and myself) more intelligible and perhaps a little more manageable. Perhaps even a LOT more manageable. We shall see, in real practice.
I would appreciate any links to information on the mental accounts and amounts of willpower.
Thank you, Eliezer.
--RickJS
“If you don’t believe that the outputs of your thought processes are entangled with reality, why do you believe the outputs of your thought processes? ”
I don’t. Well not like Believe. Some few of them I will give 40 or even 60 deciBels.
But I’m clear that my brain lies to me. Even my visual processor lies. (Have you ever been looking for your keys, looked right at them, and gone on looking?)
I hold my beliefs loosely. I’m coachable. Maybe even gullible. You can get me to believe some untruth, but I’ll let go of that easily when evidence appears.
Thanks, Eliezer!
“Are there motives for seeking truth besides curiosity and pragmatism?”
I can think of several that have showed up in my life. I’m offering these for consideration, but not claiming these are good or bad, pure or impure etc. Some will doubtless overlap somewhat with each other and the ones stated.
As a weapon. Use it to win arguments (sometimes the point of an argument is to WIN, never mind learning the truth. I’ve got automatic competitiveness I need to keep on a short leash). Use it to win bar room bets. Acquire knowledge about the “buttons” people have, and use it to manipulate them. Use it to thwart opposition to my plans, however sleazy. (“What are we going to do tonight, Brain?” … )
As evidence that I deserve an A in school. Even if I never have a pragmatic use for the knowledge, there is (briefly) value in demonstrably having the knowledge.
As culture. I don’t think I have ever found a practical use for the facts of history ( of science, of politics, or of art ), but they participated in shaping my whole world view. Out of that, I came out of retirement and dedicated myself to saving humanity. Go figure.
As a contact, as in, “I know Nick Bostrom.” (OK, that’s a bit of a stretch, but it is partly informational.) 5, As pleasure & procreation, as in, “Cain knew his wife.” ;-)
“To make rationality into a moral duty is to give it all the dreadful degrees of freedom of an arbitrary tribal custom. People arrive at the wrong answer, and then indignantly protest that they acted with propriety, rather than learning from their mistake.” Yes. I say, “Morality is for agents that can’t figure out the probable consequences of their actions.” Which includes me, of course. However, whenever I can make a good estimate, I pretty much become a consequentialist.
Seeking knowledge has, for me, an indirect but huge value. I say: Humanity needs help to survive this century, needs a LOT of help. I think Friendly AI is our best shot at getting it. And we’re missing pieces of knowledge. There may be whole fields of knowledge that we’re missing and we don’t know what they are.
I would not recommend avoiding lines of research that might enable making terribly powerful weapons. We’ve already got that problem, there’s no avoiding it. But there’s no telling what investigations will produce bits of information that will trigger some human mind into a century-class breakthrough that we had no idea we needed.
My reason for writing this is not to correct Eliezer. Rather, I want to expand on his distinction between prior information and prior probability. Pages 87-89 of Probability Theory: the Logic of Science by E. T. Jaynes (2004 reprint with corrections, ISBN 0 521 59271 2) is dense with important definitions and principles. The quotes below are from there, unless otherwise indicated.
Jaynes writes the fundamental law of inference as
P(H|DX) = P(H|X) P(D|HX) / P(D|X) (4.3)Which the reader may be more used to seeing as
P(H|D) = P(H) P(D|H) / P(D)Where
H = some hypothesis to be tested D = the data under immediate consideration X = all other information knownX is the misleadingly-named ‘prior information’, which represents all the information available other than the specific data D that we are considering at the moment. “This includes, at the very least, all it’s past experiences, from the time it left the factory to the time it received its current problem.”—Jaynes p.87, referring to a hypothetical problem-solving robot. It seems to me that in practice, X ends up being a representation of a subset of all prior experience, attempting to discard only what is irrelevant to the problem. In real human practice, that representation may be wrong and may need to be corrected.
“ … to our robot, there is no such thing as an ‘absolute’ probability; all probabilities are necessarily conditional on X at the least.” “Any probability P(A|X) which is conditional on X alone is called a prior probability. But we caution that ‘prior’ … does not necessarily mean ‘earlier in time’ … the distinction is purely a logical one; any information beyond the immediate data D of the current problem is by definition ‘prior information’.”
“Indeed, the separation of the totality of the evidence into two components called ‘data’ and ‘prior information’ is an arbitrary choice made by us, only for our convenience in organizing a chain of inferences.” Please note his use of the word ‘evidence’.
Sampling theory, which is the basis of many treatments of probability, “ … did not need to take any particular note of the prior information X, because all probabilities were conditional on H, and so we could suppose implicitly that the general verbal prior information defining the problem was included in H. This is the habit of notation that we have slipped into, which has obscured the unified nature of all inference.”
“From the start, it has seemed clear how one how one determines numerical values of of sampling probabilities¹ [e.g. P(D|H) ], but not what determines prior probabilities [AKA ‘priors’ e.g. P(H|X)]. In the present work we shall see that this s only an artifact of the unsymmetrical way of formulating problems, which left them ill-posed. One could see clearly how to assign sampling probabilities because the hypothesis H was stated very specifically; had the prior information X been specified equally well, it would have been equally clear how to assign prior probabilities.”
Jaynes never gives up on that X notation (though the letter may differ), he never drops it for convenience.
“When we look at these problems on a sufficiently fundamental level and realize how careful one must be to specify prior information before we have a well-posed problem, it becomes clear that … exactly the same principles are needed to assign either sampling probabilities or prior probabilities …” That is, P(H|X) should be calculated. Keep your copy of Kendall and Stuart handy.
I think priors should not be cheaply set from an opinion, whim, or wish. “ … it would be a big mistake to think of X as standing for some hidden major premise, or some universally valid proposition about Nature.”
The prior information has impact beyond setting prior probabilities (priors). It informs the formulation of the hypotheses, of the model, and of “alternative hypotheses” that come to mind when the data seem to be showing something really strange. For example, data that seems to strongly support psychokinesis may cause a skeptic to bring up a hypothesis of fraud, whereas a career psychic researcher may not do so. (see Jaynes pp.122-125)
I say, be alert for misinformation, biases, and wishful thinking in your X. Discard everything that is not evidence.
I’m pretty sure the free version Probability Theory: The Logic of Science is off line. You can preview the book here: http://books.google.com/books?id=tTN4HuUNXjgC&printsec=frontcover&dq=Probability+Theory:+The+Logic+of+Science&cd=1#v=onepage&q&f=false .
Also see the Unofficial Errata and Commentary for E. T. Jaynes’s Probability Theory: The Logic of Science
SEE ALSO
FOOTNOTES
There are massive compendiums of methods for sampling distributions, such as
Feller (An Introduction to Probability Theory and its Applications, Vol1, J. Wiley & Sons, New York, 3rd edn 1968 and Vol 2. J. Wiley & Sons, New York, 2nd edn 1971) and Kendall and
Stuart (The Advanced Theory of Statistics: Volume 1, Distribution Theory, McMillan, New York 1977).
** Be familiar with what is in them.
Edited 05/05/2010 to put in the actual references.
Edited 05/19/2010 to put in SEE ALSO
E.T. Jaynes, Probability Theory: The Logic of Science
and make sure you get the “unofficial errata”
personality tests
Another test set is Gallup / Clifton StrengthsFinder 2.0 (http://www.strengthsfinder.com/113647/Homepage.aspx).
For me, the results were far more useful than the various “personality profiles” I have taken , sometimes at considerable cost to my employer.
“The CSF is an online measure of personal talent that identifies areas where an individual’s greatest potential for building strengths exists. … The primary application of the CSF is as an evaluation that initiates a strengths-based development process in work and academic settings. As an omnibus assessment based on positive psychology, its main application has been in the work domain, but it has been used for understanding individuals in a variety of settings — employees, executive teams, students, families, and personal development. … Given that CSF feedback is provided to foster intrapersonal development, comparisons across profiles of individuals are discouraged.”
“When educational psychologist Donald O. Clifton first designed the interviews that subsequently became the basis for the CSF, he began by asking, “What would happen if we studied what is right with people?”Thus emerged a philosophy of using talents as the basis for consistent achievement of excellence (strength). Specifically, the strengths philosophy is the assertion that individuals are able to gain far more when they expend effort to build on their greatest talents than when they spend a comparable amount of effort to remediate their weaknesses (Clifton & Harter, 2003).”
The above two paragraphs are from Gallup’s research report, available at http://strengths.gallup.com/110389/Research-Behind-StrengthsFinder-20.aspx . (Suggestion: download the file and open it in Adobe Reader. I’ve had trouble reading it inside Firefox.)
There is a small financial cost for the test: buy the book ($13 at Amazon http://www.amazon.com/dp/159562015X/ref=nosim/?tag=thegalluporganiz) to get access to the test and support tools.
My strengths are: Intellection, Analytical, Input, Restorative, Learner
Yes, I read about ” … disappears in a puff of smoke.” I wasn’t coming back for a measly $1K, I was coming back for another million! I’ll see if they’ll let me play again. Omega already KNOWS I’m greedy, this won’t come as a shock. He’ll probably have told his team what to say when I try it.
″ … and come back for more.” was meant to be funny.
Anyway, this still doesn’t answer my questions about “Omega has been correct on each of 100 observed occasions so far—everyone who took both boxes has found box B empty and received only a thousand dollars; everyone who took only box B has found B containing a million dollars.”
Someone please answer my questions! Thanks!
Well, I mulled that over for a while, and I can’t see any way that contributes to answering my questions.
As to ” … what does your choice effect and when?”, I suppose there are common causes starting before Omega loaded the boxes, that affect both Omega’s choices and mine. For example, the machinery of my brain. No backwards-in-time is required.
In Eliezer’s article on Newcomb’s problem, he says, “Omega has been correct on each of 100 observed occasions so far—everyone who took both boxes has found box B empty and received only a thousand dollars; everyone who took only box B has found B containing a million dollars. ” Such evidence from previous players fails to appear in some problem descriptions, including Wikipedia’s.
For me this is a “no-brainer”. Take box B, deposit it, and come back for more. That’s what the physical evidence says. Any philosopher who says “Taking BOTH boxes is the rational action,” occurs to me as an absolute fool in the face of the evidence. (But I’ve never understood non-mathematical philosophy anyway, so I may a poor judge.)
Clarifying (NOT rhetorical) questions:
Have I just cheated, so that “it’s not the Newcomb Problem anymore?”
When you fellows say a certain decision theory “two-boxes”, are those theory-calculations including the previous play evidence or not?
Thanks for your time and attention.
LessWrong.com sends the user’s password in the clear (as reported by ZoneAlarm Extreme Security 8.
Please consider warning people that is so.
Oh. My mistake. When you wrote, “Plus wishing for all people to be under the rule of a god-like totalitarian sounds to me like the best way to destroy humanity.”, I read:
[Totalitarian rule… ] … [is] … the best way to destroy humanity, (as in cause and effect.)
OR maybe you meant: wishing … [is] … the best way to destroy humanity
It just never occurred to me you meant, “a god-like totalitarian pretty much comes out where extinction does in my utility function”.
Are you willing to consider that totalitarian rule by a machine might be a whole new thing, and quite unlike totalitarian rule by people?
OK.
Actually, I’m going to restrain myself to just clarifying questions while I try to learn the assumed, shared, no-need-to-mention-it body of knowledge you fellows share.
Thanks.
HOMEWORK REPORT
With some trepidation! I’m intensely aware I don’t know enough.
“Why do I believe I have free will? It’s the simplest explanation!” (Nothing in neurobiology is simple. I replace Occam’s Razor with a metaphysical growth restriction: Root causes should not be increased without dire necessity).
OK, that was flip. To be more serious:
Considering just one side of the debate, I ask: “What cognitive architecture would give me an experience of uncaused, doing-whatever-I-want, free-as-a-bird Capricious Action that is so strong that I just can’t experience (be present to) being a fairly deterministic machine?”
Cutting it down to a bare minimum: I imagine that I have a Decision Module (DM) that receives input from sensory-processing modules and suggested-action modules at its “boundary”, so those inputs are distinguishable from the neuron-firings inside the boundary: the ones that make up the DM itself. IMO, there is no way for those internal neuron firings to be presented to the input ports. I guess that there is no provision for the DM to sense anything about its own machinery.
By dubious analogy, a Turing machine looks at its own tapes, it doesn’t look at the action table that determines its next action, nor can it modify that table.
To a first approximation, no matter what notion of cause and effect I get, I just can’t see any cause for my own decisions. Even if somebody asks, “Why did you stay and fight?”, I’m just stuck with “It seemed like a good idea at the time!”
And these days, it seems to me that culture, the environment a child grows up within, is just full of the accouterments of free will: make the right choice, reward & punishment, shame, blame, accountability, “Why did you write on the wall? How could you be so STUPID!!?!!”, “God won’t tempt you beyond your ability to resist.” etc.
Being a machine, I’m not well equipped to overcome all that on the strength of mere evidence and reason.
Now I’ll start reading The Solution, and see if I was in the right ball park, or even the right continent.
Thanks for listening.
META: thread parser failed?
It sounds like these posts should have been a sub-thread instead of all being attached to the original article?:
09 March 2008 11:05:11PM
09 March 2008 11:33:14PM
10 March 2008 01:14:45AMAlso, see the mitchell porter2 - Z. M. Davis—Frank Hirsch—James Blair—Unknown discussion below.
That’s a terrible focus on punishment. Read “Don’t Shoot the Dog” by Karen Pryor and learn about behavior shaping through positive rewards.