Toby_Ord

Karma: 207

Toby_Ord 30 Jan 2010 14:35 UTC
13 points
in reply to: Paul Crowley’s comment on: Complexity of Value ≠ Complexity of Outcome
In metaethics, there are typically very good arguments against all known views, and only relatively weak arguments for each of them. For anything in philosophy, a good first stop is the Stanford Encyclopedia of Philosophy. Here are some articles on the topic at SEP:
I think the best book to read on metaethics is:
- An Introduction to Contemporary Metaethics
What links here?
- Zack_M_Davis's comment on Complexity of Value ≠ Complexity of Outcome by Wei Dai (31 Jan 2010 20:20 UTC; 6 points)

Toby_Ord 30 Jan 2010 11:45 UTC
21 points
on: Complexity of Value ≠ Complexity of Outcome
There are a lot of posts here that presuppose some combination of moral anti-realism and value complexity. These views go together well: if value is not fundamental, but dependent on characteristics of humans, then it can derive complexity from this and not suffer due to Occam’s Razor.

There are another pair of views that go together well: moral realism and value simplicity. Many posts here strongly dismiss these views, effectively allocating near-zero probability to them. I want to point out that this is a case of non-experts being very much at odds with expert opinion and being clearly overconfident. In the Phil Papers survey for example, 56.3% of philosophers lean towards or believe realism, while only 27.7% lean towards or accept anti-realism.

http://philpapers.org/surveys/results.pl

Given this, and given comments from people like me in the intersection of the philosophical and LW communities who can point out that it isn’t a case of stupid philosophers supporting realism and all the really smart ones supporting anti-realism, there is no way that the LW community should have anything like the confidence that it does on this point.

Moreover, I should point out that most of the realists lean towards naturalism, which allows a form of realism that is very different to the one that Eliezer critiques. I should also add that within philosophy, the trend is probably not towards anti-realism, but towards realism. The high tide of anti-realism was probably in the middle of the 20th Century, and since then it has lost its shiny newness and people have come up with good arguments against it (which are never discussed here...).

Even for experts in meta-ethics, I can’t see how their confidence can get outside the 30%-70% range given the expert disagreement. For non-experts, I really can’t see how one could even get to 50% confidence in anti-realism, much less the kind of 98% confidence that is typically expressed here.
What links here?
- Superintelligence 20: The value-loading problem by KatjaGrace (27 Jan 2015 2:00 UTC; 8 points)
- Wei Dai's comment on What bothers you about Less Wrong? by Will_Newsome (22 May 2011 23:59 UTC; 2 points)

Toby_Ord 24 Jan 2010 18:55 UTC
1 point
in reply to: Wei Dai’s comment on: Value Uncertainty and the Singleton Scenario

Why?

Whenever you deviate from maximizing expected value (in contexts where this is possible) you can normally find examples where this behaviour looks incorrect. For example, we might be value-pumped or something.

(And why do you find it odd, BTW?)

For one thing, negentropy may well be one of the most generally useful resources, but it seems somewhat unlikely to be intrinsically good (more likely it matters what you do with it). Thus, the question looks like one of descriptive uncertainty, just as if you had asked about money: uncertainty about whether you value that according to a particular function is descriptive uncertainty for all plausible theories. Also, while evaluative uncertainty does arise in self-interested cases, this example is a strange case of self-interest for reasons others have pointed out.

Toby_Ord 24 Jan 2010 13:34 UTC
5 points
on: Value Uncertainty and the Singleton Scenario
Thanks for the post Wei, I have a couple of comments.

Firstly, the dichotomy between Robin’s approach and Nick and mine is not right. Nick and I have always been tempted to treat moral and descriptive uncertainty in exactly the same way insofar as this is possible. However, there are cases where this appears to be ill-defined (eg how much happiness for utilitarians is worth breaking a promise for Kantians?), and to deal with these cases Nick and I consider methods that are more generally applicable. We don’t consider the bargaining/voting/market approach to be very plausible as a contender for a unique canonical answer, but as an approach to at least get the hard cases mostly right instead of remaining silent about them.

In the case you consider (which I find rather odd...) Nick and I would simply multiply it out. However, even if you looked at what our bargaining solution would do, it is not quite what you say. One thing we know is that simple majoritarianism doesn’t work (it is equivalent to picking the theory with the highest credence in two-theory cases). We would prefer to use a random dictator model, or allow bargaining over future situations too, or all conceivable situations, such that the proponent of the square-root view would be willing to offer to capitulate in most future votes in order to win this one.

Toby_Ord 19 Jan 2010 12:07 UTC
1 point
in reply to: CronoDAS’s comment on: The Prediction Hierarchy
Not quite. It depends on your beliefs about how the calculation could go wrong and how much this would change the result. If you are very confident in all parts except a minor correcting term, and are simply told that there is an error in the calculation, then you can still have some kind of rough confidence in the result (you can see how to spell this out in maths). If you know the exact part of the calculation that was mistaken, then the situation is slightly different, but still not identical to reverting to your prior.

Toby_Ord 19 Jan 2010 11:33 UTC
11 points
on: The Prediction Hierarchy
You may wish to check out the paper we wrote at the FHI on the problem of taking into account mistakes in one’s own argument. The mathematical result is the same as the one here, but the proof is more compelling. Also, we demonstrate that when applied to the LHC, the result is very different to the above analysis.

http://www.fhi.ox.ac.uk/__data/assets/pdf_file/0006/4020/probing-the-improbable.pdf

Toby_Ord 1 Dec 2009 13:25 UTC
15 points
on: The Moral Status of Independent Identical Copies
We talk about this a bit at FHI. Nick has written a paper which is relevant:

http://www.nickbostrom.com/papers/experience.pdf

Toby_Ord 26 Nov 2009 17:20 UTC
9 points
in reply to: taw’s comment on: Contrarianism and reference class forecasting

Everybody knows blue/green are correct categories, while grue/bleen are not.

Philosophers invented grue/bleen in order to be obviously incorrect categories, yet difficult to formally separate from the intuitively correct ones. There are of course less obvious cases, but the elucidation of the problem required them to come up with a particularly clear example.

Toby_Ord 29 Oct 2009 19:56 UTC
4 points
in reply to: alyssavance’s comment on: Expected utility without the independence axiom
You can also listen to an interview with one of Sarah Lichtenstein’s subjects who refused to make his preferences consistent even after the money-pump aspect was explained:

http://www.decisionresearch.org/publications/books/construction-preference/listen.html

Toby_Ord 12 Aug 2009 21:06 UTC
2 points
in reply to: PhilGoetz’s comment on: Deleting paradoxes with fuzzy logic

Calling fuzzy logic “truth functional” sounds like you’re changing the semantics;

‘Truth functional’ means that the truth value of a sentence is a function of the truth values of the propositional variables within that sentence. Fuzzy logic works this way. Probability theory does not. It is not just that one is talking about degrees of truth and the other is talking about probabilities. The analogue to truth values in probability theory are probabilities, and the probability of a sentence is not a function of the probabilities of the variables that make up that sentence (as I pointed out in the preceding comment, when A and B have the same probability, but A^A has a different probability to A^B). Thus propositional fuzzy logic is inherently different to probability theory.

You might be able to create a version of ‘fuzzy logic’ in which it is non truth-functional, but then it wouldn’t really be fuzzy logic anymore. This would be like saying that there are versions of ‘mammal’ where fish are mammals, but we have to understand ‘mammal’ to mean what we normally mean by ‘animal’. Sure, you could reinterpret the terms in this way, but the people who created the terms don’t use them that way, and it just seems to be a distraction.

At least that is as far as I understand. I am not an expert on non-classical logic, but I’m pretty sure that fuzzy logic is always understood so as to be truth-functional.

Toby_Ord 11 Aug 2009 23:54 UTC
2 points
in reply to: PhilGoetz’s comment on: Deleting paradoxes with fuzzy logic

You can select your “fuzzy logic” functions (the set of functions used to specify a fuzzy logic, which say what value to assign A and B, A or B, and not A, as a function of the values of A and B) to be consistent with probability theory, and then you’ll always get the same answer as probability theory.

How do you do this? As far as I understand, it is impossible since probability is not truth functional. For example, suppose A and B both have probability 0.5 and are independent. In this case, the probability of ‘A^B’ is 0.25, while the probability of ‘A^A’ is 0.5. You can’t do this in a (truth-functional) logic, as it has to produce the same value for both of these expressions if A and B have the same truth value. This is why minimum and maximum are used.

Toby_Ord 10 May 2009 14:42 UTC
4 points
in reply to: Wei Dai’s comment on: A Request for Open Problems
I agree that there are very interesting questions here. We have quite natural ways of describing uncomputable functions very far up the arithmetical hierarchy, and it seems that they can be described in some kind of recursive language even if the things they describe are not recursive (using recursive in the recursion theory sense both times). Turing tried something like this in Systems of Logic Based on Ordinals (Turing, 1939), but that was with formal logic and systems where you repeatedly add the Godel sentence of a system into the system as an axiom, repeating this into the transfinite. A similar thing could be done describing levels of computability transfinitely far up the arithmetical hierarchy using recursively represented ordinals to index them. However, then people like you and I will want to use certain well defined but non-recursive ordinals to do the indexing, and it seems to degenerate in the standard kind of way, just a lot further up the hierarchy than before.

Toby_Ord 6 Apr 2009 9:06 UTC
4 points
in reply to: cousin_it’s comment on: Formalizing Newcomb’s
I outlined a few more possibilities on Overcoming Bias last year:

There are many ways Omega could be doing the prediction/placement and it may well matter exactly how the problem is set up. For example, you might be deterministic and he is precalculating your choice (much like we might be able to do with an insect or computer program), or he might be using a quantum suicide method, (quantum) randomizing whether the million goes in and then destroying the world iff you pick the wrong option (This will lead to us observing him being correct ¹⁰⁰⁄₁₀₀ times assuming a many worlds interpretation of QM). Or he could have just got lucky with the last 100 people he tried it on.

If it is the deterministic option, then what do the counterfactuals about choosing the other box even mean? My approach is to say that ‘You could choose X’ means that if you had desired to choose X, then you would have. This is a standard way of understanding ‘could’ in a deterministic universe. Then the answer depends on how we suppose the world to be different to give you counterfactual desires. If we do it with a miracle near the moment of choice (history is the same, but then your desires change non-physically), then you ought two-box as Omega can’t have predicted this. If we do it with an earlier miracle, or with a change to the initial conditions of the universe (the Tannsjo interpretation of counterfactuals) then you ought one-box as Omega would have predicted your choice. Thus, if we are understanding Omega as extrapolating your deterministic thinking, then the answer will depend on how we understand the counterfactuals. One-boxers and Two-boxers would be people who interpret the natural counterfactual in the example in different (and equally valid) ways.

If we understand it as Omega using a quantum suicide method, then the objectively right choice depends on his initial probabilities of putting the million in the box. If he does it with a 50% chance, then take just one box. There is a 50% chance the world will end either choice, but this way, in the case where it doesn’t, you will have a million rather than a thousand. If, however, he uses a 99% chance of putting nothing in the box, then one-boxing has a 99% chance of destroying the world which dominates the value of the extra money, so instead two-box, take the thousand and live.

If he just got lucky a hundred times, then you are best off two-boxing.

If he time travels, then it depends on the nature of time-travel...

Thus the answer depends on key details not told to us at the outset. Some people accuse all philosophical examples (like the trolley problems) of not giving enough information, but in those cases it is fairly obvious how we are expected to fill in the details. This is not true here. I don’t think the Newcomb problem has a single correct answer. The value of it is to show us the different possibilities that could lead to the situation as specified and to see how they give different answers, hopefully illuminating the topic of free-will, counterfactuals and prediction.