utilitymonster

Karma: 339

utilitymonster Aug 12, 2010, 4:02 PM
10 points
on: Should I believe what the SIAI claims?
I’m not exactly an SIAI true believer, but I think they might be right. Here are some questions I’ve thought about that might help you out. I think it would help others out if you told us exactly where you’d be interested in getting off the boat.
1. How much of your energy are you willing to spend on benefiting others, if the expected benefits to others will be very great? (It needn’t be great for you to support SIAI.)
2. Are you willing to pursue a diversified altruistic strategy if it saves fewer expected lives (it almost always will for donors giving less than $1 million or so)?
3. Do you think mitigating x-risk is more important than giving to down-to-earth charities (GiveWell style)? (This will largely turn on how you feel about supporting causes with key probabilities that are tough to estimate, and how you feel about low-probability, high expected utility prospects.)
4. Do you think that trying to negotiate a positive singularity is the best way to mitigate x-risk?
5. Is any known organization likely to do better than SIAI in terms of negotiating a positive singularity (in terms of decreasing x-risk) on the margin?
6. Are you likely to find an organization that beats SIAI in the future?
Judging from your post, you seem most skeptical about putting your efforts into causes whose probability of success is very difficult to estimate, and perhaps low.

utilitymonster Aug 12, 2010, 3:44 PM
5 points
in reply to: Eliezer Yudkowsky’s comment on: Against Cryonics & For Cost-Effective Charity

There are maybe two or three people in the entire world who spend only the bare possible minimum on themselves, and contribute everything else to a rationally effective charity. They have an excuse for not signing up. No one else does.

I guess I agree that only the specified people can be said to have made consistently rational decisions when it comes to allocating money between benefiting themselves and benefiting others (at least of those who know something about the issues). I don’t think this implies that all but these people should sign up for cryonics. General point: [Your actions cannot be described as motivated by coherent utility function unless you do A] does not imply [you ought to do A].

Simple example: Tom cares about the welfare of others as much as his own, but biases lead him to consistently act as if he cared about his welfare 1,000 times as much as the welfare of others. Tom could overcome these biases, but he has not in the past. In a moment when he is unaffected by these biases, Tom sacrifices his life to save the lives of 900 other people.

[All that said, I take your point that it may be rational for you to advocate signing up for cryonics, since cryonics money and charity money may not be substitutes.]

utilitymonster Aug 11, 2010, 2:34 PM
0 points
on: Book Recommendations
Peter Unger’s Living High and Letting DIe is excellent, especially the first few chapters. It is mostly applied ethics, partly moral psychology, partly methodology for first-order moral philosophy.

The book makes a strong case for an obligation to engage in high-impact philanthropy. Rationalists will enjoy it because it highlights various forms of irrationality in our ethical judgments. Unger identifies various factors that influence our ethical judgments for the worse, many of which have not received similar discussion in psych literature on heuristics and biases (at least I haven’t seen it). [When I say he shows that these things influence our judgments for the worse, I have in mind that many people would cease to approve of the judgments if they understood what was driving them.]

utilitymonster Aug 5, 2010, 1:41 PM
0 points
in reply to: gwern’s comment on: Open Thread, August 2010
Don’t have. I can e-mail a copy if you want (got it through university’s subscription).

utilitymonster Aug 5, 2010, 12:31 PM
0 points
in reply to: gwern’s comment on: Open Thread, August 2010
Link

utilitymonster Aug 3, 2010, 7:09 PM
7 points
on: Open Thread, August 2010
If you want to eliminate hindsight bias, write down some reasons that you think justify your judgment.

Those who consider the likelihood of an event after it has occurred exaggerate their likelihood of having been able to predict that event in advance. We attempted to eliminate this hindsight bias among 194 neuropsychologists. Foresight subjects read a case history and were asked to estimate the probability of three different diagnoses. Subjects in each of the three hindsight groups were told that one of the three diagnoses was correct and were asked to state what probability they would have assigned to each diagnosis if they were making the original diagnosis. Foresight-reasons and hindsight-reasons subjects performed the same task as their foresight and hindsight counterparts, except they had to list one reason why each of the possible diagnoses might be correct. The frequency of subjects succumbing to the hindsight bias was lower in the hindsight-reasons groups than in the hindsight groups not asked to list reasons.

ARKES, H.R., et al., 1988. Eliminating the hindsight bias. Journal of applied psychology.

utilitymonster Jul 27, 2010, 12:35 PM
11 points
in reply to: CarlShulman’s comment on: Metaphilosophical Mysteries
I’m a philosophy grad student. While I agree that many epistemologists still think it is important to talk in terms of believe/not-believe and justified/non-justfied, I find relatively few epistemologists who reject the notion of credence or think that credences shouldn’t be probabilities. Of those who think credences shouldn’t be probability functions, most would not object to using a weaker system of imprecise probabilities (Reference: James M. Joyce (2005). How Probabilities Reflect Evidence. Philosophical Perspectives 19 (1):153–178). These people are still pretty much on team Bayesianism.

So, in a way, the Bayesian domination is pretty strong. In another way, it isn’t: few debates in traditional epistemology have been translated in Bayesian terms and solved (though this would probably solve very many of them). And many epistemologists doubt that Bayesianism will be genuinely helpful with respect to their concerns.

utilitymonster Jul 27, 2010, 12:24 PM
8 points
in reply to: JoshuaZ’s comment on: Metaphilosophical Mysteries

According to the The PhilPapers Survey, 25.8% of surveyed philosophers were consequentialists of some form. That makes it hard to argue for a consensus about maximizing expected utility.

There are a lot of philosophers who buy into maximizing expected utility, but aren’t consequentialists. Proof: If you look at philosophers specializing in decision theory, 58% buy into consequentialism link. Of this group, the vast majority would go for something very close to expected utility maximization.

Part of this has to do with consequentialism not having a crisp definition that fits philosophers’ intuitive usage. Some think consequentialism must be agent-neutral and get off the boat there (but could still be EU maximizers). Others have preferences that could (if made more coherent) satisfy the axioms of decision theory, but don’t think that the utility function that represents those preferences also orders outcomes in terms of goodness. I.e., these people want to be EU maximizers, but don’t want to maximize goodness (maybe they want to maximize some weighting of goodness and keeping their hands clean).

utilitymonster Jul 5, 2010, 1:26 PM
3 points
in reply to: Alexandros’s comment on: Open Thread: July 2010
I’d suggest starting with a list of common biases and producing a question (or a few?) for each. The questions could test the biases and you could have an explanation of why the biased reasoning is bad, with examples.

It would also be useful to group the biases together in natural clusters, if possible.

utilitymonster Jul 4, 2010, 2:35 PM
0 points
in reply to: utilitymonster’s comment on: Open Thread: July 2010
Link to talk.

utilitymonster Jul 4, 2010, 12:06 PM
1 point
in reply to: Vladimir_M’s comment on: Open Thread: July 2010
I was assuming Charlie would show Bob the first person to see ⁷⁵⁄₁₀₀.

Anyway, your analysis solves this as well. Being the first to see a particular result tells you essentially nothing about the composition of the pond (provided N_sci is sufficiently large that someone or other was nearly certain to see the result). Thus, each of Al and Bob should regard their previous observations as irrelevant once they learn that they were the first to get those results. Thus, they should just stick with their priors and be ⁵⁰⁄₅₀ about the composition of the pond.

utilitymonster Jul 4, 2010, 11:47 AM
1 point
in reply to: Vladimir_M’s comment on: Open Thread: July 2010
You’re right, thanks.

I was considering an example with 10^100 scientists. I thought that since there would be a lot more scientists who got 25 big in the ¹⁄₄ scenario than in the ³⁄₄ scenario (about 9.18 10^98 vs. 1.279 10^75), you’d be more likely to be first the ³⁄₄ scenario. But this forgets about the probability of getting an improbable result.

In general, if there are N scientists, and the probability of getting some result is p, then we can expect Np scientists to get that result on average. If the order is shuffled as you suggest, then the probability of being the first to get that result is p * 1/(Np) = 1/N. So the probability of being the first to get the result is the same, regardless of the likelihood of the result (assuming someone will get the result).

EDIT: It occurs to me that I might have been thinking about the probability of being selected by Al conditional on getting ²⁵⁄₁₀₀. In that case, you’re a lot more likely to be selected if the pond is ³⁄₄ big than if it is ¹⁄₄ big, since WAY more people got similar results in the latter case. JGMWeissman was probably thinking the same.

utilitymonster Jul 3, 2010, 7:45 PM
0 points
in reply to: JGWeissman’s comment on: Open Thread: July 2010
This should be important to the finite case. The probability of being the first to see ²⁵⁄₁₀₀ is WAY higher (x 10^25 or so) if the lake is ³⁄₄ full of big fish than if it is ¹⁄₄ full of big fish.

But in the infinite case the probability of being first is 0 either way...

utilitymonster Jul 3, 2010, 7:11 PM
0 points
in reply to: prase’s comment on: Open Thread: July 2010
Glad to see we’re on the same page.

utilitymonster Jul 3, 2010, 7:01 PM
0 points
in reply to: Blueberry’s comment on: Open Thread: July 2010
I’m not sure about this:

If Bob is as certain as Al that Bob was picked specifically for his result, then they do have the same information, and they should both discount Bob’s observations to the same degree for that reason.

Here’s why:

VARIANT 2: Charlie has both Al and Bob into his office before the drawings take place. He explains that the first guy (other than Al) to see ²⁵⁄₁₀₀ big will report to Al. Bob goes out and sees ²⁵⁄₁₀₀ big. To his surprise, he gets called into Charlie’s office and informed that he was the first to see that result.

Question: right now, what should Bob expect to hear from Al?

Intuitively, he should expect that Al had similar results. But if you’re right, it would seem that Bob should discount his results once he talks to Charlie and fights out that he is the messenger. If that’s right, he should have no idea what to expect Al to say. But that seems wrong. He hasn’t even heard anything from Al.

If you’re still not convinced, consider:

VARIANT 3: Charlie has both Al and Bob into his office before the drawings take place. He explains that the first guy (other than Al) to see ²⁵⁄₁₀₀ big will win a trip to Hawaii. Bob goes out and sees ²⁵⁄₁₀₀ big. To his surprise, he gets called into Charlie’s office and informed that he was the first to see that result.

I can see no grounds for treating VARIANT 3 differently from VARIANT 2. And it is clear that in VARIANT 3 Bob should not discount his results.

utilitymonster Jul 3, 2010, 6:47 PM
0 points
in reply to: RobinZ’s comment on: Open Thread: July 2010

Al is special because he has been selected to meet Bob regardless of what he observes. Therefore his data is genuinely uncorrelated with how he was selected for the meeting.

Not quite. He was selected to meet someone like Bob, in the sense that whoever the messenger was, he’d have seen ²⁵⁄₁₀₀ big. He didn’t know he’d meet Bob. But he regards the identity of the messenger as irrelevant.

You can bring out the difference by considering a variant of the case in which both Al and Bob hear about Charlie’s plan in advance. (In this variant, the first to see ²⁵⁄₁₀₀ big will visit Al.)

What is the relevance of the fact that they observed highly improbable event?

utilitymonster Jul 3, 2010, 6:42 PM
2 points
in reply to: prase’s comment on: Open Thread: July 2010
Al, not AI. (“Al” as in “Alan”)

utilitymonster Jul 3, 2010, 5:28 PM
10 points
on: Open Thread: July 2010
Here’s a puzzle I’ve been trying to figure out. It involves observation selection effects and agreeing to disagree. It is related to a paper I am writing, so help would be appreciated. The puzzle is also interesting in itself.

Charlie tosses a fair coin to determine how to stock a pond. If heads, it gets ³⁄₄ big fish and ¹⁄₄ small fish. If tails, the other way around. After Charlie does this, he calls Al into his office. He tells him, “Infinitely many scientists are curious about the proportion of fish in this pond. They are all good Bayesians with the same prior. They are going to randomly sample 100 fish (with replacement) each and record how many of them are big and how many are small. Since so many will sample the pond, we can be sure that for any n between 0 and 100, some scientist will observe that n of his 100 fish were big. I’m going to take the first one that sees 25 big and team him up with you, so you can compare notes.” (I don’t think it matters much whether infinitely many scientists do this or just 3^^^3.)

Okay. So Al goes and does his sample. He pulls out 75 big fish and becomes nearly certain that ³⁄₄ of the fish are big. Afterwards, a guy named Bob comes to him and tells him he was sent by Charlie. Bob says he randomly sampled 100 fish, 25 of which were big. They exchange ALL of their information.

Question: How confident should each of them be that ³⁄₄ of the fish are big?

Natural answer: Charlie should remain nearly certain that ¾ of the fish are big. He knew in advance that someone like Bob was certain to talk to him regardless of what proportion of fish were big. So he shouldn’t be the least bit impressed after talking to Bob.

But what about Bob? What should he think? At first glance, you might think he should be ⁵⁰⁄₅₀, since 50% of the fish he knows about have been big and his access to Al’s observations wasn’t subject to a selection effect. But that can’t be right, because then he would just be agreeing to disagree with Al! (This would be especially puzzling, since they have ALL the same information, having shared everything.) So maybe Bob should just agree with Al: he should be nearly certain that ¾ of the fish are big.

But that’s a bit odd. It isn’t terribly clear why Bob should discount all of his observations, since they don’t seem to subject to any observation selection effect; at least from his perspective, his observations were a genuine random sample.

Things get weirder if we consider a variant of the case.

VARIANT: as before, but Charlie has a similar conversation with Bob. Only this time, he tells him he’s going to introduce Bob to someone who observed exactly 75 of 100 fish to be big.

New Question: Now what should Bob and Al think?

Here, things get really weird. By the reasoning that led to the Natural Answer above, Al should be nearly certain that ¾ are big and Bob should be nearly certain that ¼ are big. But that can’t be right. They would just be agreeing to disagree! (Which would be especially puzzling, since they have ALL the same information.) The idea that they should favor one hypothesis in particular is also disconcerting, given the symmetry of the case. Should they both be 50/50?

Here’s where I’d especially appreciate enlightenment: 1.If Bob should defer to Al in the original case, why? Can someone walk me through the calculations that lead to this?

2.If Bob should not defer to Al in the original case, is that because Al should change his mind? If so, what is wrong with the reasoning in the Natural Answer? If not, how can they agree to disagree?

3.If Bob should defer to Al in the original case, why not in the symmetrical variant?

4.What credence should they have in the symmetrical variant?

5.Can anyone refer me to some info on observation selection effects that could be applied here?

utilitymonster Jul 3, 2010, 4:20 PM
2 points
in reply to: Nisan’s comment on: Open Thread: July 2010
This is always hard with infinities. But I think it can be a mistake to worry about this too much.

A rough way of making the point would be this. Pick a freaking huge number of years, like 3^^^3. Look at our universe after it has been around for that many years. You can be pretty damn sure that most of the beings with evidence like yours are Botlzmann brains on the model in question.

utilitymonster Jul 3, 2010, 4:12 PM
6 points
in reply to: Nisan’s comment on: Open Thread: July 2010
I gathered this from a talk by Sean Carroll that I attended, and it was supposed to be a consequence of the standard picture. All the Boltzmann brains come up in the way distant future, after thermal equilibrium, as random fluctuations. Carroll regarded this as a defect of the normal approach, and used this as a launching point to speculate about a different model.

I wish I had a more precise reference, but this isn’t my area and I only heard this one talk. But I think this issue is discussed in his book From Eternity to Here. Here’s a blogpost that, I believe, faithfully summarizes the relevant part of the talk. The normal solution to Boltzmann brains is to add a past hypothesis. Here is the key part where the post discusses the benefits and shortcomings of this approach:

Solution: Albert adds a Past Hypothesis (PAST), which says roughly that the universe started in very low entropy state (much lower than this one). So the objective probability that this is the lowest entropy state of the universe is 0—meaning we can’t be Boltzmann brains. As a bonus, we get an explanation of the direction of time, why ice cubes melt, why we can cause things to happen in the future and not the past, and how we have records of the past and not the future: all these things get a very high objective probability.

But (Sean Carroll argues) this moves too fast: just adding the past hypothesis allows the universe to eventually reach thermal equilibrium. Once that happens (in about 10100 years) there will be an extremely long period (~10^10120 years) during which random fluctuations bring about all sorts of things, including our old enemies, Boltzmann brains. And there will be a lot of them. And some of them will have the same experiences we do.

The years there are missing some carats. Should be 10^100 and 10^10^120.
What links here?
- red75's comment on MWI, copies and probability by Roko (Jul 4, 2010, 6:51 AM; 0 points)