New knowledge comes from observation. If you are referring to knowledge of what a theory says rather than of which theory is true, then this is assumed to be known. The math of how to deal with a situation where a theory is known but its consequences cannot be fully understood due to mathematical limitations is still in its infancy, but this has never posed a problem in practice.
That is a substantive and strong empiricist claim which I think is false.
For example, we have knowledge of things we never observed. Like stars. Observation is always indirect and its correctness always depends on theories such as our theories about whether the chain of proxies we are observing with will in fact observe what we want to observe.
Do you understand what I’m talking about and have a reply, or do you need me to explain further?
then this is assumed to be known
Could you understand why I might object to making a bunch of assumptions in one’s epistemology?
Could you understand why I might object to making a bunch of assumptions in one’s epistemology?
It is assumed in practice, applied epistemology being a rather important thing to have. In ‘pure’ epistemology, it is just labelled incomplete; we definitely don’t have all the answers yet.
it is just labelled incomplete; we definitely don’t have all the answers yet.
It seems to me that you’re pretty much conceding that your epistemology doesn’t work. (All flaws can be taken as “incomplete” parts where, in the future, maybe a solution will be found.)
That would leave the following important disagreement: Popper’s epistemology is not incomplete in any significant way. There is room for improvement, sure, but not really any flaws worth complaining about. No big unsolved problems marring it. So, why not drop this epistemology that doesn’t have the answers yet for one that does?
It seems to me that you’re pretty much conceding that your epistemology doesn’t work.
Would you describe quantum mechanics’ incompatibility with general relativity as “the theory doesn’t work”? For a being with unlimited computing power in a universe that is known to be computable (except for the being itself obviously), we are almost entirely done. Furthermore, many of the missing pieces to get from that to something much more complete seem related.
Popper’s epistemology is not incomplete in any significant way.
No, it is just wrong. Expected utility allows me to compute the right course of action given my preferences and a probability distribution over all theories. Any consistent consequentialist decision rule must be basically equivalent to that. The statement that there is no way to assign probabilities to theories therefore implies that there is no algorithm that a consequentialist can follow to reliably achieve their goals. Note that even if Popper’s values are not consequentialist, a consequentialist should still be able to act based on the knowledge obtained by a valid epistemology.
I suspect you are judging Popperian epistemology by standards it states are mistaken. Would you agree that doing that would be a mistake?
Expected utility allows me to compute the right course of action given my preferences and a probability distribution over all theories.
Note the givens. There’s more givens which you didn’t mention too, e.g. some assumptions about people’s utilities having certain mathematical properties (you need this for, e.g., comparing them).
I don’t believe these givens are all true. If you think otherwise could we start with you giving the details more? I don’t want to argue with parts you simply omitted b/c I’ll have to guess what you think too much.
As a separate issue, “given my preferences” is such a huge given. It means that your epistemology does not deal in moral knowledge. At all. It simply takes preferences as givens and doesn’t tell you which to have. So in practice in real life it cannot be used for a lot of important issues. That’s a big flaw. And it means a whole entire second epistemology is needed to deal in moral knowledge. And if we have one of those, and it works, why not use it for all knowledge?
The rest of the paragraph was what I meant by this. You agree that Popperian epistemology states that theories should not be assigned probabilities.
I suspect you are judging Popperian epistemology by standards it states are mistaken. Would you agree that doing that would be a mistake?
Depends. If it’s standards make it useless, then, while internally consistent, I can judge it to be pointless. I just want an epistemology that can help me actually make decisions based on what I learn about reality.
Expected utility allows me to compute the right course of action given my preferences and a probability distribution over all theories.
Note the givens. There’s more givens which you didn’t mention too, e.g. some assumptions about people’s utilities having certain mathematical properties (you need this for, e.g., comparing them).
I don’t think I was clear. A utility here just means a number I use to say how good a possible future is, so I can decide whether I want to work toward that future. In this context, it is far more general than anything composed of a bunch of term, each of which describes some properties of a person.
It simply takes preferences as givens and doesn’t tell you which to have.
I can learn more about my preferences from observation of my own brain using standard Bayesian epistemology.
I just want an epistemology that can help me actually make decisions based on what I learn about reality.
Popperian epistemology does this. What’s the problem? Do you think that assigning probabilities to theories is the only possible way to do this?
Overall you’ve said almost nothing that’s actually about Popperian epistemology. You just took one claim (which has nothing to do with what it’s about, it’s just a minor point about what it isn’t) and said it’s wrong (without detailed elaboration).
I don’t think I was clear. A utility here just means a number I use to say how good a possible future is, so I can decide whether I want to work toward that future.
I understood that. I think you are conflating “utility” the mathematical concept with “utility” the thing people in real life have. The second may not have the convenient properties the first has. You have not provided an argument that it does.
I can learn more about my preferences from observation of my own brain using standard Bayesian epistemology.
How do you learn what preferences are good to have, in that way?
Popperian epistemology does this. What’s the problem? Do you think that assigning probabilities to theories is the only possible way to do this?
It is a theorem that every consistent consequentialist decision rule is either a Bayesian decision rule or a limit of Bayesian decision rules. Even if the probabilities are not mentioned when constructing the rule, they can be inferred from its final form.
I understood that. I think you are conflating “utility” the mathematical concept with “utility” the thing people in real life have. The second may not have the convenient properties the first has. You have not provided an argument that it does.
I don’t know what you mean by ′ “utility” the thing people in real life have’.
How do you learn what preferences are good to have, in that way?
Can we please not get into this. If it helps, assume I am an expected paperclip maximizer. How would I decide then?
It is a theorem that every consistent consequentialist decision rule is either a Bayesian decision rule or a limit of Bayesian decision rules.
What was the argument for that?
And what is the argument that actions should be judged ONLY by consequences? What is the arguing for excluding all other considerations?
I don’t know what you mean by ′ “utility” the thing people in real life have’.
People have preferences and values. e.g. they might want a cat or an iPhone and be glad to get it. The mathematical properties of these real life things are not trivial or obvious. For example, suppose getting the cat would add 2 happiness and the iPhone would add 20. Would getting both add 22 happiness? Answer: we cannot tell from the information available.
Can we please not get into this.
But the complete amorality of your epistemology—it’s total inability to create entire categories of knowledge—is a severe flaw in it. There’s plenty of other examples I could use to make the same point, however in general they are a bit less clear. One example is epistemology: epistemology is also not an empirical field. But I imagine you may argue about that a bunch, while with morality I think it’s clearer.
It is a theorem that every consistent consequentialist decision rule is either a Bayesian decision rule or a limit of Bayesian decision rules.
What was the argument for that?
I’ve actually been meaning to find a paper that proves that myself. There’s apparently a proof in Mathematical Statistics, Volume 1: Basic and Selected Topics by Peter Bickel and Kjell Doksum.
And what is the argument that actions should be judged ONLY by consequences? What is the arguing for excluding all other considerations?
None. I’ve just never found any property of an action that I care about other the consequences. I’d gladly change my mind on this if one were pointed out to me.
People have preferences and values. e.g. they might want a cat or an iPhone and be glad to get it. The mathematical properties of these real life things are not trivial or obvious. For example, suppose getting the cat would add 2 happiness and the iPhone would add 20. Would getting both add 22 happiness? Answer: we cannot tell from the information available.
Agreed, and agreed that this is a common mistake. If you thought I was making this error, I was being far less clear than I thought.
But the complete amorality of your epistemology—it’s total inability to create entire categories of knowledge—is a severe flaw in it. There’s plenty of other examples I could use to make the same point, however in general they are a bit less clear. One example is epistemology: epistemology is also not an empirical field. But I imagine you may argue about that a bunch, while with morality I think it’s clearer.
Well all my opinions about the foundations of morality and epistemology are entirely deductive.
I’ve actually been meaning to find a paper that proves that myself. There’s apparently a proof in Mathematical Statistics, Volume 1: Basic and Selected Topics by Peter Bickel and Kjell Doksum.
Agreed, and agreed that this is a common mistake. If you thought I was making this error, I was being far less clear than I thought.
I thought you didn’t address the issue (and need to): you did not say what mathematical properties you think that real utilities have and how you deal with them.
Well all my opinions about the foundations of morality and epistemology are entirely deductive.
Using what premises?
None. I’ve just never found any property of an action that I care about other the consequences. I’d gladly change my mind on this if one were pointed out to me.
What about explanations about whether it was a reasonable decision for the person to make that action, given the knowledge he had before making it?
I’ve actually been meaning to find a paper that proves that myself. There’s apparently a proof in Mathematical Statistics, Volume 1: Basic and Selected Topics by Peter Bickel and Kjell Doksum.
Ordered. But I think you should be more cautious asserting things that other people told you were true, which you have not checked up on.
Why do you guys reject and largely ignore it? Is it merely because Eliezer published a few sentences of nasty anti-Popper myths in an old essay?
Every possible universe is associated with a utility.
Any two utilities can be compared.
These comparisons are transitive.
Weighted averages of utilities can be taken.
For any three possible universe, L, M, and N, with L < M, a weighted average of L and N is less than a weighted average of M and N, if N is accorded the same weight in both cases.
Well all my opinions about the foundations of morality and epistemology are entirely deductive.
Using what premises?
Basically just definitions. I’m currently trying to enumerate them, which is why I wanted to find the proof of the theorem we were discussing.
None. I’ve just never found any property of an action that I care about other the consequences. I’d gladly change my mind on this if one were pointed out to me.
What about explanations about whether it was a reasonable decision for the person to make that action, given the knowledge he had before making it?
Care about in the sense of when I’m deciding whether to make it. I don’t really care about how reasonable other people’s decisions are unless it’s relevant to my interactions with them, where I will need that knowledge to make my own decisions.
Ordered. But I think you should be more cautious asserting things that other people told you were true, which you have not checked up on.
Wait, you bought the book just for that proof? I don’t even know if its the best proof of it (in terms of making assumptions that aren’t necessary to get the result). I’m confidant in the proof because of all the other similar proofs I’ve read, though none seem as widely applicable as that one. I can almost sketch a proof in my mind. Some simple ones are explained well at http://en.wikipedia.org/wiki/Coherence_%28philosophical_gambling_strategy%29 .
For your first 5 points, how is that a reply about Popper? Maybe you meant to quote something else.
I don’t think that real people’s way of considering utility is based on entire universes at a time. So I don’t think your math here corresponds to how people think about it.
Wait, you bought the book just for that proof?
No, I used inter library loan.
I don’t really care about how reasonable other people’s decisions are
Then put yourself in as the person under consideration. Do you think it matters whether you make decisions using rational thought processes, or do only the (likely?) consequences matter?
Basically just definitions.
How do you judge whether you have the right ones? You said “entirely deductive” above, so are you saying you have a deductive way to judge this?
For your first 5 points, how is that a reply about Popper? Maybe you meant to quote something else.
Yes, I did. Oops.
I don’t think that real people’s way of considering utility is based on entire universes at a time. So I don’t think your math here corresponds to how people think about it.
But that is what a choice is between—the universe where you choose one way and the universe where you choose another. Often large parts of the universe are ignored, but only because the action’s consequences for those parts are not distinguishable from how those part would be if a different action was taken. A utility function may be a sum (or more complicated combination) of parts referring to individual aspects of the universe, but, in this context, let’s not call those ‘utilities’; we’ll reserve that word for the final thing used to make decisions. Most of this is not consciously invoked when people make decisions, but a choice that does not stand when you consider its expected effects on the whole universe is a wrong choice.
I don’t really care about how reasonable other people’s decisions are
Then put yourself in as the person under consideration. Do you think it matters whether you make decisions using rational thought processes, or do only the (likely?) consequences matter?
I could could achieve better consequences using an ‘irrational’ process, I would, but this sounds nonsensical because I am used to defining ‘rational’ as that which reliably gets the best consequences.
How do you judge whether you have the right ones? You said “entirely deductive” above, so are you saying you have a deductive way to judge this?
Definitions as in “let’s set up this situation and see which choices make sense”. It’s pretty much all like the Dutch book arguments.
Definitions as in “let’s set up this situation and see which choices make sense”. It’s pretty much all like the Dutch book arguments.
I don’t think I understand. This would rely on your conception of the real life situation (if you want it to apply to real life), of what what makes sense, being correct. That goes way beyond deductive or definitions into substantive claims.
About decisions, if a method like “choose by whim” gets you a good result in a particular case, you’re happy with it? You don’t care that it doesn’t make any sense if it works out this time?
But that is what a choice is between—the universe where you choose one way and the universe where you choose another.
So what? I think you’re basically saying that your formulation is equivalent to what people (should) do. But that doesn’t address the issue of what people actually do—it doesn’t demonstrate the equivalence. As you guys like to point out, people often think in ways that don’t make sense, including violating basic logic.
But also, for example, I think a person might evaluate getting a cat, and getting an iphone, and then they might (incorrectly) evaluate both by adding the benefits instead of by considering the universe with both based on its own properties.
Another issue is that I don’t think any two utilities people have can be compared. They are sometimes judged with different, contradictory standards. This leads to two major issues when trying to compare them 1) the person doesn’t know how 2) it might not be possible to compare even in theory because one or both contain some mistakes. the mistakes might need to be fixed before comparing, but that would change it.
a choice that does not stand when you consider its expected effects on the whole universe is a wrong choice
I’m not saying people are doing it correctly. Whether they are right or wrong has no bearing on whether “utility” the mathematical object with the 5 properties you listed corresponds to “utility” the thing people do.
If you want to discuss what people should do, rather than what they do do, that is a moral issue. So it leads to questions like: how does bayesian epistemology create moral knowledge and how does it evaluate moral statements?
If you want to discuss what kind of advice is helpful to people (which people?), then I”m sure how you can see how talking about entire universes could easily confuse people, and how some other procedure being a special case of it may not be very good advice which does not address the practical problems they are having.
Definitions as in “let’s set up this situation and see which choices make sense”. It’s pretty much all like the Dutch book arguments.
I don’t think I understand. This would rely on your conception of the real life situation (if you want it to apply to real life), of what what makes sense, being correct. That goes way beyond deductive or definitions into substantive claims.
Do you think that the Dutch book arguments go “way beyond deductive or definitions”? Well, I guess that would depend on what you conclude from them. For now, lets say “there is a need to assign probabilities to events, no probability can be less than 0 or more than 1 and probabilities of mutually exclusive events should add”.
About decisions, if a method like “choose by whim” gets you a good result in a particular case, you’re happy with it? You don’t care that it doesn’t make any sense if it works out this time?
The confusion here is that we’re not judging an action. If I make a mistake and happen to benefit from it, there were good consequences, but there was no choice involved. I don’t care about this; it already happened. What I do care about, and what I can accomplish, is avoiding similar mistakes in the future.
If you want to discuss what people should do, rather than what they do do, that is a moral issue.
Yes, that is what I was discussing. I probably don’t want to actually get into my arguments here. Can you give an example of what you mean by “moral knowledge”?
Applying dutch book arguments to real life situations always goes way behind deduction and definitions, yes.
lets say “there is a need to assign probabilities to events, no probability can be less than 0 or more than 1 and probabilities of mutually exclusive events should add”.
A need? Are you talking about morality now?
Why are we saying this? You now speak of probabilities of events. Previously we were discussing epistemology which is about ideas. I object to assigning probabilities to the truth of ideas. Assigning them to events is OK when
1) the laws of physics are indeterministic (never, as far as we know)
2) we have incomplete information and want to make a prediction that would be deterministic except that we have to put several possibilities in some places, which leads to several possible answers. and probability is a reasonable way to organize thoughts about that.
So what?
Can you give an example of what you mean by “moral knowledge”?
Murder is immoral.
Being closed minded makes ones life worse because it sabotages improvement.
Can you give an example of what you mean by “moral knowledge”?
Murder is immoral.
Are you saying Popper would evaluate “Murder is immoral.” in the same way as “Atoms are made up of electrons and a nucleus.”? How would you test this? What would you consider a proof of it?
I prefer to leave such statements undefined, since people disagree too much on what ‘morality’ means. I am a moral realist to some, a relativist to others, and an error theorist to other others. I could prove the statement for many common non-confused definitions, though not for, for example, people who say ‘morality’ is synomnymous to ‘that which is commanded by God’, which is based on confusion but at least everyone can agree on when it is or isn’t true and not for error theorists, as both groups’ definitions make the sentence false.
Being closed minded makes ones life worse because it sabotages improvement.
In theory I could prove this sentence, but in practice I could not do this clearly, especially over the internet. It would probably be much easier for you to read the sequences, which get to this toward the end, but, depending on your answers to some of my questions, there may be an easier way to explain this.
Are you saying Popper would evaluate “Murder is immoral.” in the same way as “Atoms are made up of electrons and a nucleus.”?
Yes. One epistemology. All types of knowledge. Unified!
How would you test this?
You would not.
What would you consider a proof of it?
We don’t accept proofs of anything, we are fallibilists. We consider mathematical proofs to be good arguments though. I don’t really want to argue about those (unless you’re terribly interested. btw this is covered in the math chapter of The Fabric of Reality by David Deutsch). But the point is we don’t accept anything as providing certainty or even probableness. In our terminology, nothing provides justification.
What we do instead is explain our ideas, and to criticize mistakes, and in this way to improve our ideas. This, btw, creates knowledge in the same way as evolution (replication of ideas, with variation, and selection by criticism). That’s not a metaphor or analogy by literally true.
I prefer to leave such statements undefined, since people disagree too much on what ‘morality’ means.
Wouldn’t it be nice if you had an epistemology that helped you deal with all kinds of knowledge, so you didn’t have to simply give up on applying reason to important issues like what is a good life, and what are good values?
This, btw, creates knowledge in the same way as evolution (replication of ideas, with variation, and selection by criticism). That’s not a metaphor or analogy by literally true.
Well, biological evolution is a much smaller part of conceptspace than “replication, variation, selection” and now I’m realizing that you probably haven’t read A Human’s Guide to Words which is extremely important and interesting and, while you’ll know much of it, has things that are unique and original and that you’ll learn a lot from. Please read it.
I prefer to leave such statements undefined, since people disagree too much on what ‘morality’ means.
Wouldn’t it be nice if you had an epistemology that helped you deal with all kinds of knowledge, so you didn’t have to simply give up on applying reason to important issues like what is a good life, and what are good values?
I do apply reason to those things, I just don’t use the words ‘morality’ in my reasoning process because too many people get confused. It is only a word after all.
On a side note, I am staring to like what I hear of Popper. It seems to embody an understanding of the brain and a bunch of useful advice for it. I think I disagree with some things, but on grounds that seems like the sort of thing that is accepted as motivation for the theory self-modify. Does that make sense? Anyways, it’s not Popper’s fault that there are a set of theorems that in principle remove the need for other types of thought and in practice cause big changes in the way we understand and evaluate the heuristics that are necessary because the brain is fallible and computationally limited.
Wei Dai likes thinking about how to deal with questions outside of Bayesianism’s current domain of applicability, so he might be interested in this.
lets say “there is a need to assign probabilities to events, no probability can be less than 0 or more than 1 and probabilities of mutually exclusive events should add”.
A need? Are you talking about morality now?
Interpret this as a need in order to achieve some specified goal in order to keep this part the debate out of morality. A paperclip maximizer, for example would obviously need to not pay 200 paperclips for a lottery with a maximum payout of 100 paperclips in order to achieve its goals. Furthermore, this applies to any consequentialist set of preferences.
Why are we saying this? You now speak of probabilities of events.
So you assume morality (the “specified goal”). That makes your theory amoral.
Well there’s a bit more than this, but it’s not important right now. One can work toward any goal just by assuming it as a goal.
Why is there a need to assign probabilities to theories? Popperian epistemology functions without doing that.
Because of the Dutch book arguments. The probabilities can be inferred from the choices. I’m not sure if the agent’s probability distribution can be fully determined from a finite set of wagers, but it can be definitely be inferred to an arbitrary degree of precision by adding enough wagers.
Can you give an example of how you use a Dutch book argument on a non-gambling topic? For example, if I’m considering issues like whether to go swimming today, and what nickname to call my friend, and I don’t assign probabilities like “80% sure that calling her Kate is the best option”, how do I get Dutch Booked?
First you hypothetically ask what would happen if you were asked to make bets on whether calling her Kate would result in world X (with utility U(X)). Do this for all choices and all possible worlds. This gives you probabilities and utilities. You then take a weighted average, as per the VNM theorem.
You don’t get to decide utilities so much as you have to figure out what they are. You already have a utility function, and you do your best to describe it . How do you weight the things you value relative to each other?
This takes observation, because what we think we value often turns out not to be a good description of our feelings and behavior.
By criticizing them. And conjecturing improvements which meet the challenges of the criticism. It is the same method as for improving all other knowledge.
In outline it is pretty simple. You may wonder things like what would be a good moral criticism. To that I would say: there’s many books full of examples, why dismiss all that? There is no one true way of arguing. Normal arguments are ok, I do not reject them all out of hand but try to meet their challenges. Even the ones with some kind of mistake (most of them), you can often find some substantive point which can be rescued. It’s important to engage with the best versions of theories you can think of.
BTW once upon a time I was vaguely socialist. Now I’m a (classical) liberal. People do change their fundamental moral values for the better in real life. I attended a speech by a former Muslim terrorist who is now a pro-Western Christian (walid shoebat).
I’ve changed my social values plenty of times, because I decided different policies better served my terminal values. If you wanted to convince me to support looser gun control, for instance, I would be amenable to that because my position on gun control is simply an avenue for satisfying my core values, which might better be satisfied in a different way.
If you tried to convince me to support increased human suffering as an end goal, I would not be amenable to that, unless it turns out I have some value I regard as even more important that would be served by it.
This is what Popper called the Myth of the Framework and refuted in his essay by that name. It’s just not true that everyone is totally set in their ways and extremely closed minded, as you suggest. People with different frameworks learn from each other.
One example is children learn. They are not born sharing their parents framework.
You probably think that frameworks are genetic, so they are. Dealing with that would take a lengthy discussion. Are you interested in this stuff? Would you read a book about it? Do you want to take it seriously?
I’m somewhat skeptical b/c e.g. you gave no reply to some of what I said.
I think a lot of the reason people don’t learn other frameworks, in practice, is merely that they choose not to. They think it sounds stupid (before they understand what it’s actually saying) and decide not to try.
When did I suggest that everyone is set in their ways and extremely closed minded? As I already pointed out, I’ve changed my own social values plenty of times. Our social frameworks are extremely plastic, because there are many possible ways to serve our terminal values.
I have responded to moral arguments with regards to more things than I could reasonably list here (economics, legal codes, etc.) I have done so because I was convinced that alternatives to my preexisting social framework better served my values.
Valuing strict gun control, to pick an example, is not genetically coded for. A person might have various inborn tendencies which will affect how they’re likely to feel about gun control; they might have innate predispositions towards authoritarianism or libertarianism, for instance, that will affect how they form their opinion. A person who valued freedom highly enough might support little or no gun control even if they were convinced that it would result in a greater loss of life. You would have a hard time finding anyone who valued freedom so much that they would support looser gun control if they were convinced it would destroy 90% of the world population, which gives you a bit of information about how they weight their preferences.
If you wanted to convince me to support more human suffering instead of more human happiness, you would have to appeal to something else I value even more that would be served by this. If you could argue that my preference for happiness is arbitrary, that preference for suffering is more natural, even if you could demonstrate that the moral goodness of human suffering is intrinsically inscribed on the fabric of the universe, why should I care? To make me want to make humans unhappy, you’d have to convince me there’s something else I want enough to make humans unhappy for its sake.
I also don’t feel I’m being properly understood here; I’m sorry if I’m not following up on everything, but I’m trying to focus on the things that I think meaningfully further the conversation, and I think some of your arguments are based on misapprehensions about where I’m coming from. You’ve already made it clear that you feel the same, but you can take it as assured that I’m both trying to understand you and make myself understood.
When did I suggest that everyone is set in their ways and extremely closed minded?
You suggested it about a category of ideas which you called “core values”.
If you wanted to convince me to support more human suffering instead of more human happiness, you would have to appeal to something else I value even more
You are saying that you are not open to new values which contradict your core values. Ultimately you might replace all but the one that is the most core, but never that one.
You are saying that you are not open to new values which contradict your core values. Ultimately you might replace all but the one that is the most core, but never that one.
That’s more or less correct. To quote one of Eliezer’s works of ridiculous fanfiction, “A moral system has room for only one absolute commandment; if two unbreakable rules collide, one has to give way.”
If circumstances force my various priorities into conflict, some must give way to others, and if I value one thing more than anything else, I must be willing to sacrifice anything else for it. That doesn’t necessarily make it my only terminal value; I might have major parts of my social framework which ultimately reduce to service to another value, and they’d have to bend if they ever came into conflict with a more heavily weighted value.
Well in the first half, you get Dutch booked in the usual way. It’s not necessarily actually happening, but there still must be probabilities that you would use if it were. In the second half, if you don’t follow the procedure (or an equivalent one) you violate at least one VNM axiom.
If you violate axiom 1, there are situations in which you don’t have a preferred choice—not as is “both are equally good/bad” but as in your decision process does not give an answer or gives more than one answer. I don’t think I’d call this a decision process.
If you violate axiom 2, there are outcomes L, M and N such that you’d want to switch from L to M and then from M to N, but you would not want to switch from L to N.
Axiom 3 is unimportant and is just there to simplify the math.
For axiom 4, imagine a situation where a statement with unknown truth-value, X, determines whether you get to choose between two outcomes, L and M, with L < M, or have no choice in accepting a third outcome, N. If you violate the axiom, there is a situation like this where, if you were asked for your choice before you know X (it will be ignored if X is false), you would pick L, even though L < M.
Do any of these situations describe your preferences?
And I’m still curious how the utilities are decided. By whim?
If your decision process is not equivalent to one that uses the previously described procedure, there are situations where something like one of the following will happen.
I ask you if you want chocolate or vanilla ice cream and you don’t decide. Not just you don’t care which one you get or you would prefer not to have ice cream, but you don’t output anything and see nothing wrong with that.
You prefer chocolate to vanilla ice cream, so you would willingly pay 1c to have the vanilla ice cream that you have been promised upgraded to chocolate. You also happen to prefer strawberry to chocolate, so you are willing to pay 1c to exchange a promise of a chocolate ice cream for a promise of a strawberry ice cream. Furthermore, it turn out you prefer vanilla to strawberry, so whenever you are offered a strawberry ice cream, you gladly pay a single cent to change that to an offer of vanilla, ad infinitum.
N/A
You like chocolate ice cream more than vanilla ice cream. Nobody knows if you’ll get ice cream today, but you are asked for your choice just in case, so you pick vanilla.
Let’s consider (2). Suppose someone was in the process of getting Dutch Booked like this. It would not go on ad infinitum. They would quickly learn better. Right? So even if this happened, I think it would not be a big deal.
Let’s say they did learn better. How would they do this—changing their utility function? Someone with a utility function like this really does prefer B+1c to A, C+1c to B, and A+1c to C. Even if they did change their utility function, the new one would either have a new hole or it would obey the results of the VNM-theorem.
So Bayes teaches: do not disobey the laws of logic and math.
Still wondering where the assigning probabilities to truths of theories is.
OK. So what? There’s more to life than that. That’s so terribly narrow. I mean, that part of what you’re saying is right as far as it goes, but it doesn’t go all that far. And when you start trying to apply it to harder cases—what happens? Do you have some Bayesian argument about who to vote for for president? Which convinced millions of people? Or should have convinced them, and really answers the questions much better than other arguments?
Still wondering where the assigning probabilities to truths of theories is.
Well the Dutch books make it so you have to pick some probabilities. Actually getting the right prior is incomplete, though Solomonoff induction is most of the way there.
OK. So what? There’s more to life than that. That’s so terribly narrow. I mean, that part of what you’re saying is right as far as it goes, but it doesn’t go all that far.
Where else are you hoping to go?
And when you start trying to apply it to harder cases—what happens? Do you have some Bayesian argument about who to vote for for president? Which convinced millions of people? Or should have convinced them, and really answers the questions much better than other arguments?
In principle, yes. There’s actually a computer program called AIXItl that does it. In practice I use approximations to it. It probably could be done to a very higher degree of certainty. There are a lot of issues and a lot of relevant data.
Well the Dutch books make it so you have to pick some probabilities.
Can you give an example? Use the ice cream flavors. What probabilities do you have to pick to buy ice cream without being dutch booked?
Where else are you hoping to go?
Explanatory knowledge. Understanding the world. Philosophical knowledge. Moral knowledge. Non-scientific, non-emprical knowledge. Beyond prediction and observation.
In principle, yes.
How do you know if your approximations are OK to make or ruin things? How do you work out what kinds of approximations are and aren’t safe to make?
The way I would do that is by understanding the explanation of why something is supposed to work. In that way, I can evaluate proposed changes to see whether they mess up the main point or not.
Endo, I think you are making things more confusing by combining issues of Bayesianism with issues of utility. It might help to keep them more separate or to be clear when one is talking about one, the other, or some hybrid.
I use the term Bayesianism to include utility because (a) they are connected and (b) a philosophy of probabilities as abstract mathematical constructs with no applications doesn’t seem complete; it needs an explanation of why those specific objects are studied. How do you think that any of this caused or could cause confusion?
Well, it empirically seems to be causing confusion. See curi’s remarks about the ice cream example. Also, one doesn’t need Bayesianism to include utility and that isn’t standard (although it is true that they do go very well together).
Let’s consider (2). Suppose someone was in the process of getting Dutch Booked like this. It would not go on ad infinitum. They would quickly learn better. Right? So even if this happened, I think it would not be a big deal.
So the argument is now not that that suboptimal issues don’t exist but that they aren’t a big deal? Are you aware that the primary reason that this involves small amounts of ice cream is for convenience of the example? There’s no reason these couldn’t happen with far more serious issues (such as what medicine to use).
I know. I thought it was strange that you said “ad infinitum” when it would not go on forever. And that you presented this as dire but made your example non-dire.
But OK. You say we must consider probabilities, or this will happen. Well, suppose that if I do something it will happen. I could notice that, criticize it, and thus avoid it.
How can I notice? I imagine you will say that involves probabilities. But in your ice cream example I don’t see the probabilities. It’s just preferences for different ice creams, and an explanation of how you get a loop.
And what I definitely don’t see is probabilities that various theories are true (as opposed to probabilities about events which are ok).
But OK. You say we must consider probabilities, or this will happen. Well, suppose that if I do something it will happen. I could notice that, criticize it, and thus avoid it.
Yes, but the Bayesian avoids having this step. For any step you can construct a “criticism” that will duplicate what the Bayesian will do. This is connected to a number of issues, including the fact that what constitutes valid criticism in a Popperian framework is far from clear.
But in your ice cream example I don’t see the probabilities. It’s just preferences for different ice creams, and an explanation of how you get a loop.
Ice cream is an analogy. It might not be a great one since it is connected to preferences (which sometimes gets confused with Bayesianism). The analogy isn’t a great one. It might make more sense to just go read Cox’s theorem and translate to yourself what the assumptions mean about an approach.
what constitutes valid criticism in a Popperian framework is far from clear.
Anything which is not itself criticized.
Ice cream is an analogy.
Could you pick any real world example you like, where the probabilities needed to avoid dutch book aren’t obvious, and point them out? To help concretize the idea for me.
Could you pick any real world example you like, where the probabilities needed to avoid dutch book aren’t obvious, and point them out
Well, I’m not sure, in that I’m not convinced that Dutch Booking really does occur much in real life other than in the obvious contexts. But there are a lot of contexts it does occur in. For example, a fair number of complicated stock maneuvers can be thought of essentially as attempts to dutch book other players in the stock market.
It is a theorem that every consistent consequentialist decision rule is either a Bayesian decision rule or a limit of Bayesian decision rules.
I’ve actually been meaning to find a paper that proves that myself. There’s apparently a proof in Mathematical Statistics, Volume 1: Basic and Selected Topics by Peter Bickel and Kjell Doksum.
Consequentialism is not in the index.
Decision rule is, a little bit.
I don’t think this book contains a proof mentioning consequentialism. Do you disagree? Give a page or section?
It looks like what they are doing is defining a decision rule in a special way. So, by definition, it has to be a mathematical thing to do with probability. Then after that, I’m sure it’s rather easy to prove that you should use bayes’ theorem rather than some other math.
But none of that is about decisions rules in the sense of methods human beings use for making decisions. It’s just if you define them in a particular way—so that Bayes’ is basically the only option—then you can prove it.
see e.g. page 19 where they give a definition. A Popperian approach to making decisions simply wouldn’t fit within the scope of their definition, so the conclusion of any proof like you claimed existed (which i haven’t found in this book) would not apply to Popperian ideas.
Maybe there is a lesson here about believing stuff is proven when you haven’t seen the proof, listening to hearsay about what books contain, and trying to apply proofs you aren’t familiar with (they often have limits on scope).
It says a decision rule (their term) is a function of the sample space, mapping something like complete sets of possible data to things people do. (I think it needs to be complete sets of all your data to be applied to real world human decision making. They don’t explain what they are talking about in the type of way I think is good and clear. I think that’s due to having in mind different problems they are trying to solve than I have. We have different goals without even very much overlap. They both involve “decisions” but we mean different things by the word.)
In real life, people use many different decision rules (my term, not theirs). And people deal with clashes between them.
You may claim that my multiple decision rules can be combined into one mathematical function. That is so. But the result isn’t a smooth function so when they start talking about estimation they have big problems! And this is the kind of thing I would expect to get acknowledgement and discussion if they were trying to talk about how humans make decisions, in practice, rather than just trying to define some terms (chosen to sound like they have something to do with what humans do) and then proceed with math.
e.g. they try to talk about estimating amount of error. if you know error bars on your data, and you have a smooth function, you’re maybe kind of OK with imperfect data. but if your function has a great many jumps in it, what are you to do? what if, within the margin for error on something, there’s several discontinuities? i think they are conceiving of the decision rule function as being smooth and not thinking about what happens when it’s very messy. Maybe they specified some assumptions so that it has to be which I missed, but anyway human beings have tons of contradictory and not-yet-integrated ideas in their head—mistakes and separate topics they haven’t connected yet, and more—and so it’s not smooth.
On a similar note they talk about the median and mean which also don’t mean much when it’s not smooth. Who cares what the mean is over an infinitely large sample space where you get all sorts of unrepresentative results in large unrealistic portions of it? So again I think they are looking at the issues differently than me. They expect things like mathematically friendly distributions (for which means and medians are useful); I don’t.
Moving on to a different issue, they conceive of a decision rule which takes input and then gives output. I do not conceive of people starting with the input and then deciding the output. I think decision making is more complicated. While thinking about the input, people create more input—their thoughts. The input is constantly being changed during the decision process, it’s not a fixed quantity to have a function of. Also being changed during any significant decision is the decision rule itself—it too isn’t a static function even for purposes of doing one decision (at least in the normal sense. maybe they would want to call every step in the process a decision. so when you’re deciding a flavor of ice cream that might involve 50 decisions, with updates to the decisions rules and inputs in between them. if they want to do something like that they do not explain how it works.)
They conceive of the input to decisions as “data”. But I conceive of much thinking as not using much empirical data, if any. I would pick a term that emphasizes it. The input to all decision making is really ideas, some of which are about empirical data and some of which aren’t. Data is a special case, not the right term for the general case. From this I take that they are empiricists. You can find a refutation of empiricism in The Beginning of Infinity by David Deutsch but anyway it’s a difference between us.
A Popperian approach to decision making would focus more on philosophical problems, and their solutions. It would say things like: consider what problem you’re trying to solve, and consider what actions may solve it. And criticize your ideas to eliminate errors. And … well no short summary does it justice. I’ve tried a few times here. But Popperian ways of thinking are not intuitive to people with the justificationist biases dominant in our culture. Maybe if you like everything I said I’ll try to explain more, but in that case I don’t know why you wouldn’t read some books which are more polished than what I would type in. If you have a specific, narrow question I can see that answering that would make sense.
Thank you for that detailed reply. I just have a few comments:
“data” could be any observable property of the world
in statistical decision theory, the details of the decision process that implements the mapping aren’t the focus because we’re going to try to go straight to the optimal mapping in a mathematical fashion
there’s no requirement that the decision function be smooth—it’s just useful to look at such functions first for pedagogical reasons. All of the math continues to work in the presence of discontinuities.
a weak point of statistical decision theory is that it treats the set of actions as a given; human strategic brilliance often finds expression through the realization that a particular action is possible
“data” could be any observable property of the world
Yes but using it to refer to a person’s ideas, without clarification, would be bizarre and many readers wouldn’t catch on.
in statistical decision theory, the details of the decision process that implements the mapping aren’t the focus because we’re going to try to go straight to the optimal mapping in a mathematical fashion
Straight to the final, perfect truth? lol… That’s extremely unPopperian. We don’t expect progress to just end like that. We don’t expect you get so far and then there’s nothing further. We don’t think the scope for reason is so bounded, nor do we think fallibility is so easily defeated.
In practice searches for optimal things of this kind always involve many premises with have substantial philosophical meaning. (Which is often, IMO, wrong.)
a weak point of statistical decision theory is that it treats the set of actions as a given; human strategic brilliance often finds expression through the realization that a particular action is possible
Does it use an infinite set of all possible actions? I would have thought it wouldn’t rely on knowing what each action actually is, but would just broadly specify the set of all actions and move on.
@smooth: what good is a mean or median with no smoothness? And for margins of error, with a non-smooth function, what do you do?
With a smooth region of a function, taking the midpoint of the margin of error region is reasonable enough. But when there is a discontinuity, there’s no way to average it and get a good result. Mixing different ideas is a hard process if you want anything useful to result. If you just do it in a simple way like averaging you end up with a result that none of the ideas think will work and shouldn’t be surprised when it doesn’t. It’s kind of like how if you have half an army do one general’s plan, and half do another, the result is worse than doing either one.
New knowledge comes from observation. If you are referring to knowledge of what a theory says rather than of which theory is true, then this is assumed to be known. The math of how to deal with a situation where a theory is known but its consequences cannot be fully understood due to mathematical limitations is still in its infancy, but this has never posed a problem in practice.
That is a substantive and strong empiricist claim which I think is false.
For example, we have knowledge of things we never observed. Like stars. Observation is always indirect and its correctness always depends on theories such as our theories about whether the chain of proxies we are observing with will in fact observe what we want to observe.
Do you understand what I’m talking about and have a reply, or do you need me to explain further?
Could you understand why I might object to making a bunch of assumptions in one’s epistemology?
The new knowledge that is obtained from an observation is not just the content of the observation, it is also the new probabilities resulting from the observation. This is discussed at http://lesswrong.com/lw/pb/belief_in_the_implied_invisible/ .
It is assumed in practice, applied epistemology being a rather important thing to have. In ‘pure’ epistemology, it is just labelled incomplete; we definitely don’t have all the answers yet.
It seems to me that you’re pretty much conceding that your epistemology doesn’t work. (All flaws can be taken as “incomplete” parts where, in the future, maybe a solution will be found.)
That would leave the following important disagreement: Popper’s epistemology is not incomplete in any significant way. There is room for improvement, sure, but not really any flaws worth complaining about. No big unsolved problems marring it. So, why not drop this epistemology that doesn’t have the answers yet for one that does?
Would you describe quantum mechanics’ incompatibility with general relativity as “the theory doesn’t work”? For a being with unlimited computing power in a universe that is known to be computable (except for the being itself obviously), we are almost entirely done. Furthermore, many of the missing pieces to get from that to something much more complete seem related.
No, it is just wrong. Expected utility allows me to compute the right course of action given my preferences and a probability distribution over all theories. Any consistent consequentialist decision rule must be basically equivalent to that. The statement that there is no way to assign probabilities to theories therefore implies that there is no algorithm that a consequentialist can follow to reliably achieve their goals. Note that even if Popper’s values are not consequentialist, a consequentialist should still be able to act based on the knowledge obtained by a valid epistemology.
Can you be more specific?
I suspect you are judging Popperian epistemology by standards it states are mistaken. Would you agree that doing that would be a mistake?
Note the givens. There’s more givens which you didn’t mention too, e.g. some assumptions about people’s utilities having certain mathematical properties (you need this for, e.g., comparing them).
I don’t believe these givens are all true. If you think otherwise could we start with you giving the details more? I don’t want to argue with parts you simply omitted b/c I’ll have to guess what you think too much.
As a separate issue, “given my preferences” is such a huge given. It means that your epistemology does not deal in moral knowledge. At all. It simply takes preferences as givens and doesn’t tell you which to have. So in practice in real life it cannot be used for a lot of important issues. That’s a big flaw. And it means a whole entire second epistemology is needed to deal in moral knowledge. And if we have one of those, and it works, why not use it for all knowledge?
The rest of the paragraph was what I meant by this. You agree that Popperian epistemology states that theories should not be assigned probabilities.
Depends. If it’s standards make it useless, then, while internally consistent, I can judge it to be pointless. I just want an epistemology that can help me actually make decisions based on what I learn about reality.
I don’t think I was clear. A utility here just means a number I use to say how good a possible future is, so I can decide whether I want to work toward that future. In this context, it is far more general than anything composed of a bunch of term, each of which describes some properties of a person.
I can learn more about my preferences from observation of my own brain using standard Bayesian epistemology.
Popperian epistemology does this. What’s the problem? Do you think that assigning probabilities to theories is the only possible way to do this?
Overall you’ve said almost nothing that’s actually about Popperian epistemology. You just took one claim (which has nothing to do with what it’s about, it’s just a minor point about what it isn’t) and said it’s wrong (without detailed elaboration).
I understood that. I think you are conflating “utility” the mathematical concept with “utility” the thing people in real life have. The second may not have the convenient properties the first has. You have not provided an argument that it does.
How do you learn what preferences are good to have, in that way?
It is a theorem that every consistent consequentialist decision rule is either a Bayesian decision rule or a limit of Bayesian decision rules. Even if the probabilities are not mentioned when constructing the rule, they can be inferred from its final form.
I don’t know what you mean by ′ “utility” the thing people in real life have’.
Can we please not get into this. If it helps, assume I am an expected paperclip maximizer. How would I decide then?
What was the argument for that?
And what is the argument that actions should be judged ONLY by consequences? What is the arguing for excluding all other considerations?
People have preferences and values. e.g. they might want a cat or an iPhone and be glad to get it. The mathematical properties of these real life things are not trivial or obvious. For example, suppose getting the cat would add 2 happiness and the iPhone would add 20. Would getting both add 22 happiness? Answer: we cannot tell from the information available.
But the complete amorality of your epistemology—it’s total inability to create entire categories of knowledge—is a severe flaw in it. There’s plenty of other examples I could use to make the same point, however in general they are a bit less clear. One example is epistemology: epistemology is also not an empirical field. But I imagine you may argue about that a bunch, while with morality I think it’s clearer.
I’ve actually been meaning to find a paper that proves that myself. There’s apparently a proof in Mathematical Statistics, Volume 1: Basic and Selected Topics by Peter Bickel and Kjell Doksum.
None. I’ve just never found any property of an action that I care about other the consequences. I’d gladly change my mind on this if one were pointed out to me.
Agreed, and agreed that this is a common mistake. If you thought I was making this error, I was being far less clear than I thought.
Well all my opinions about the foundations of morality and epistemology are entirely deductive.
The original is Abraham Wald’s An Essentially Complete Class of Admissible Decision Functions.
Thank you!
I thought you didn’t address the issue (and need to): you did not say what mathematical properties you think that real utilities have and how you deal with them.
Using what premises?
What about explanations about whether it was a reasonable decision for the person to make that action, given the knowledge he had before making it?
Ordered. But I think you should be more cautious asserting things that other people told you were true, which you have not checked up on.
Every possible universe is associated with a utility.
Any two utilities can be compared.
These comparisons are transitive.
Weighted averages of utilities can be taken.
For any three possible universe, L, M, and N, with L < M, a weighted average of L and N is less than a weighted average of M and N, if N is accorded the same weight in both cases.
Basically just definitions. I’m currently trying to enumerate them, which is why I wanted to find the proof of the theorem we were discussing.
Care about in the sense of when I’m deciding whether to make it. I don’t really care about how reasonable other people’s decisions are unless it’s relevant to my interactions with them, where I will need that knowledge to make my own decisions.
Wait, you bought the book just for that proof? I don’t even know if its the best proof of it (in terms of making assumptions that aren’t necessary to get the result). I’m confidant in the proof because of all the other similar proofs I’ve read, though none seem as widely applicable as that one. I can almost sketch a proof in my mind. Some simple ones are explained well at http://en.wikipedia.org/wiki/Coherence_%28philosophical_gambling_strategy%29 .
For your first 5 points, how is that a reply about Popper? Maybe you meant to quote something else.
I don’t think that real people’s way of considering utility is based on entire universes at a time. So I don’t think your math here corresponds to how people think about it.
No, I used inter library loan.
Then put yourself in as the person under consideration. Do you think it matters whether you make decisions using rational thought processes, or do only the (likely?) consequences matter?
How do you judge whether you have the right ones? You said “entirely deductive” above, so are you saying you have a deductive way to judge this?
Yes, I did. Oops.
But that is what a choice is between—the universe where you choose one way and the universe where you choose another. Often large parts of the universe are ignored, but only because the action’s consequences for those parts are not distinguishable from how those part would be if a different action was taken. A utility function may be a sum (or more complicated combination) of parts referring to individual aspects of the universe, but, in this context, let’s not call those ‘utilities’; we’ll reserve that word for the final thing used to make decisions. Most of this is not consciously invoked when people make decisions, but a choice that does not stand when you consider its expected effects on the whole universe is a wrong choice.
I could could achieve better consequences using an ‘irrational’ process, I would, but this sounds nonsensical because I am used to defining ‘rational’ as that which reliably gets the best consequences.
Definitions as in “let’s set up this situation and see which choices make sense”. It’s pretty much all like the Dutch book arguments.
I don’t think I understand. This would rely on your conception of the real life situation (if you want it to apply to real life), of what what makes sense, being correct. That goes way beyond deductive or definitions into substantive claims.
About decisions, if a method like “choose by whim” gets you a good result in a particular case, you’re happy with it? You don’t care that it doesn’t make any sense if it works out this time?
So what? I think you’re basically saying that your formulation is equivalent to what people (should) do. But that doesn’t address the issue of what people actually do—it doesn’t demonstrate the equivalence. As you guys like to point out, people often think in ways that don’t make sense, including violating basic logic.
But also, for example, I think a person might evaluate getting a cat, and getting an iphone, and then they might (incorrectly) evaluate both by adding the benefits instead of by considering the universe with both based on its own properties.
Another issue is that I don’t think any two utilities people have can be compared. They are sometimes judged with different, contradictory standards. This leads to two major issues when trying to compare them 1) the person doesn’t know how 2) it might not be possible to compare even in theory because one or both contain some mistakes. the mistakes might need to be fixed before comparing, but that would change it.
I’m not saying people are doing it correctly. Whether they are right or wrong has no bearing on whether “utility” the mathematical object with the 5 properties you listed corresponds to “utility” the thing people do.
If you want to discuss what people should do, rather than what they do do, that is a moral issue. So it leads to questions like: how does bayesian epistemology create moral knowledge and how does it evaluate moral statements?
If you want to discuss what kind of advice is helpful to people (which people?), then I”m sure how you can see how talking about entire universes could easily confuse people, and how some other procedure being a special case of it may not be very good advice which does not address the practical problems they are having.
Do you think that the Dutch book arguments go “way beyond deductive or definitions”? Well, I guess that would depend on what you conclude from them. For now, lets say “there is a need to assign probabilities to events, no probability can be less than 0 or more than 1 and probabilities of mutually exclusive events should add”.
The confusion here is that we’re not judging an action. If I make a mistake and happen to benefit from it, there were good consequences, but there was no choice involved. I don’t care about this; it already happened. What I do care about, and what I can accomplish, is avoiding similar mistakes in the future.
Yes, that is what I was discussing. I probably don’t want to actually get into my arguments here. Can you give an example of what you mean by “moral knowledge”?
Applying dutch book arguments to real life situations always goes way behind deduction and definitions, yes.
A need? Are you talking about morality now?
Why are we saying this? You now speak of probabilities of events. Previously we were discussing epistemology which is about ideas. I object to assigning probabilities to the truth of ideas. Assigning them to events is OK when
1) the laws of physics are indeterministic (never, as far as we know)
2) we have incomplete information and want to make a prediction that would be deterministic except that we have to put several possibilities in some places, which leads to several possible answers. and probability is a reasonable way to organize thoughts about that.
So what?
Murder is immoral.
Being closed minded makes ones life worse because it sabotages improvement.
Are you saying Popper would evaluate “Murder is immoral.” in the same way as “Atoms are made up of electrons and a nucleus.”? How would you test this? What would you consider a proof of it?
I prefer to leave such statements undefined, since people disagree too much on what ‘morality’ means. I am a moral realist to some, a relativist to others, and an error theorist to other others. I could prove the statement for many common non-confused definitions, though not for, for example, people who say ‘morality’ is synomnymous to ‘that which is commanded by God’, which is based on confusion but at least everyone can agree on when it is or isn’t true and not for error theorists, as both groups’ definitions make the sentence false.
In theory I could prove this sentence, but in practice I could not do this clearly, especially over the internet. It would probably be much easier for you to read the sequences, which get to this toward the end, but, depending on your answers to some of my questions, there may be an easier way to explain this.
Yes. One epistemology. All types of knowledge. Unified!
You would not.
We don’t accept proofs of anything, we are fallibilists. We consider mathematical proofs to be good arguments though. I don’t really want to argue about those (unless you’re terribly interested. btw this is covered in the math chapter of The Fabric of Reality by David Deutsch). But the point is we don’t accept anything as providing certainty or even probableness. In our terminology, nothing provides justification.
What we do instead is explain our ideas, and to criticize mistakes, and in this way to improve our ideas. This, btw, creates knowledge in the same way as evolution (replication of ideas, with variation, and selection by criticism). That’s not a metaphor or analogy by literally true.
Wouldn’t it be nice if you had an epistemology that helped you deal with all kinds of knowledge, so you didn’t have to simply give up on applying reason to important issues like what is a good life, and what are good values?
Fine, what would you consider an argument for it?
Eliezer and I probably agree with you.
Well, biological evolution is a much smaller part of conceptspace than “replication, variation, selection” and now I’m realizing that you probably haven’t read A Human’s Guide to Words which is extremely important and interesting and, while you’ll know much of it, has things that are unique and original and that you’ll learn a lot from. Please read it.
I do apply reason to those things, I just don’t use the words ‘morality’ in my reasoning process because too many people get confused. It is only a word after all.
On a side note, I am staring to like what I hear of Popper. It seems to embody an understanding of the brain and a bunch of useful advice for it. I think I disagree with some things, but on grounds that seems like the sort of thing that is accepted as motivation for the theory self-modify. Does that make sense? Anyways, it’s not Popper’s fault that there are a set of theorems that in principle remove the need for other types of thought and in practice cause big changes in the way we understand and evaluate the heuristics that are necessary because the brain is fallible and computationally limited.
Wei Dai likes thinking about how to deal with questions outside of Bayesianism’s current domain of applicability, so he might be interested in this.
Interpret this as a need in order to achieve some specified goal in order to keep this part the debate out of morality. A paperclip maximizer, for example would obviously need to not pay 200 paperclips for a lottery with a maximum payout of 100 paperclips in order to achieve its goals. Furthermore, this applies to any consequentialist set of preferences.
Not sure why I wrote that. Substitute ‘theories’.
So you assume morality (the “specified goal”). That makes your theory amoral.
Why is there a need to assign probabilities to theories? Popperian epistemology functions without doing that.
Well there’s a bit more than this, but it’s not important right now. One can work toward any goal just by assuming it as a goal.
Because of the Dutch book arguments. The probabilities can be inferred from the choices. I’m not sure if the agent’s probability distribution can be fully determined from a finite set of wagers, but it can be definitely be inferred to an arbitrary degree of precision by adding enough wagers.
Can you give an example of how you use a Dutch book argument on a non-gambling topic? For example, if I’m considering issues like whether to go swimming today, and what nickname to call my friend, and I don’t assign probabilities like “80% sure that calling her Kate is the best option”, how do I get Dutch Booked?
First you hypothetically ask what would happen if you were asked to make bets on whether calling her Kate would result in world X (with utility U(X)). Do this for all choices and all possible worlds. This gives you probabilities and utilities. You then take a weighted average, as per the VNM theorem.
How do I get Dutch Booked for not doing that?
And I’m still curious how the utilities are decided. By whim?
You don’t get to decide utilities so much as you have to figure out what they are. You already have a utility function, and you do your best to describe it . How do you weight the things you value relative to each other?
This takes observation, because what we think we value often turns out not to be a good description of our feelings and behavior.
From our genes? And the goal is just to figure out what it is, but not change it for the better?
Can you explain how you would change your fundamental moral values for the better?
By criticizing them. And conjecturing improvements which meet the challenges of the criticism. It is the same method as for improving all other knowledge.
In outline it is pretty simple. You may wonder things like what would be a good moral criticism. To that I would say: there’s many books full of examples, why dismiss all that? There is no one true way of arguing. Normal arguments are ok, I do not reject them all out of hand but try to meet their challenges. Even the ones with some kind of mistake (most of them), you can often find some substantive point which can be rescued. It’s important to engage with the best versions of theories you can think of.
BTW once upon a time I was vaguely socialist. Now I’m a (classical) liberal. People do change their fundamental moral values for the better in real life. I attended a speech by a former Muslim terrorist who is now a pro-Western Christian (walid shoebat).
I’ve changed my social values plenty of times, because I decided different policies better served my terminal values. If you wanted to convince me to support looser gun control, for instance, I would be amenable to that because my position on gun control is simply an avenue for satisfying my core values, which might better be satisfied in a different way.
If you tried to convince me to support increased human suffering as an end goal, I would not be amenable to that, unless it turns out I have some value I regard as even more important that would be served by it.
This is what Popper called the Myth of the Framework and refuted in his essay by that name. It’s just not true that everyone is totally set in their ways and extremely closed minded, as you suggest. People with different frameworks learn from each other.
One example is children learn. They are not born sharing their parents framework.
You probably think that frameworks are genetic, so they are. Dealing with that would take a lengthy discussion. Are you interested in this stuff? Would you read a book about it? Do you want to take it seriously?
I’m somewhat skeptical b/c e.g. you gave no reply to some of what I said.
I think a lot of the reason people don’t learn other frameworks, in practice, is merely that they choose not to. They think it sounds stupid (before they understand what it’s actually saying) and decide not to try.
When did I suggest that everyone is set in their ways and extremely closed minded? As I already pointed out, I’ve changed my own social values plenty of times. Our social frameworks are extremely plastic, because there are many possible ways to serve our terminal values.
I have responded to moral arguments with regards to more things than I could reasonably list here (economics, legal codes, etc.) I have done so because I was convinced that alternatives to my preexisting social framework better served my values.
Valuing strict gun control, to pick an example, is not genetically coded for. A person might have various inborn tendencies which will affect how they’re likely to feel about gun control; they might have innate predispositions towards authoritarianism or libertarianism, for instance, that will affect how they form their opinion. A person who valued freedom highly enough might support little or no gun control even if they were convinced that it would result in a greater loss of life. You would have a hard time finding anyone who valued freedom so much that they would support looser gun control if they were convinced it would destroy 90% of the world population, which gives you a bit of information about how they weight their preferences.
If you wanted to convince me to support more human suffering instead of more human happiness, you would have to appeal to something else I value even more that would be served by this. If you could argue that my preference for happiness is arbitrary, that preference for suffering is more natural, even if you could demonstrate that the moral goodness of human suffering is intrinsically inscribed on the fabric of the universe, why should I care? To make me want to make humans unhappy, you’d have to convince me there’s something else I want enough to make humans unhappy for its sake.
I also don’t feel I’m being properly understood here; I’m sorry if I’m not following up on everything, but I’m trying to focus on the things that I think meaningfully further the conversation, and I think some of your arguments are based on misapprehensions about where I’m coming from. You’ve already made it clear that you feel the same, but you can take it as assured that I’m both trying to understand you and make myself understood.
You suggested it about a category of ideas which you called “core values”.
You are saying that you are not open to new values which contradict your core values. Ultimately you might replace all but the one that is the most core, but never that one.
That’s more or less correct. To quote one of Eliezer’s works of ridiculous fanfiction, “A moral system has room for only one absolute commandment; if two unbreakable rules collide, one has to give way.”
If circumstances force my various priorities into conflict, some must give way to others, and if I value one thing more than anything else, I must be willing to sacrifice anything else for it. That doesn’t necessarily make it my only terminal value; I might have major parts of my social framework which ultimately reduce to service to another value, and they’d have to bend if they ever came into conflict with a more heavily weighted value.
Well in the first half, you get Dutch booked in the usual way. It’s not necessarily actually happening, but there still must be probabilities that you would use if it were. In the second half, if you don’t follow the procedure (or an equivalent one) you violate at least one VNM axiom.
If you violate axiom 1, there are situations in which you don’t have a preferred choice—not as is “both are equally good/bad” but as in your decision process does not give an answer or gives more than one answer. I don’t think I’d call this a decision process.
If you violate axiom 2, there are outcomes L, M and N such that you’d want to switch from L to M and then from M to N, but you would not want to switch from L to N.
Axiom 3 is unimportant and is just there to simplify the math.
For axiom 4, imagine a situation where a statement with unknown truth-value, X, determines whether you get to choose between two outcomes, L and M, with L < M, or have no choice in accepting a third outcome, N. If you violate the axiom, there is a situation like this where, if you were asked for your choice before you know X (it will be ignored if X is false), you would pick L, even though L < M.
Do any of these situations describe your preferences?
I’ll let Desrtopa handle this.
Can you give a concrete example. What happens to me? Is it that I get an outcome which is less ideal than was available?
If your decision process is not equivalent to one that uses the previously described procedure, there are situations where something like one of the following will happen.
I ask you if you want chocolate or vanilla ice cream and you don’t decide. Not just you don’t care which one you get or you would prefer not to have ice cream, but you don’t output anything and see nothing wrong with that.
You prefer chocolate to vanilla ice cream, so you would willingly pay 1c to have the vanilla ice cream that you have been promised upgraded to chocolate. You also happen to prefer strawberry to chocolate, so you are willing to pay 1c to exchange a promise of a chocolate ice cream for a promise of a strawberry ice cream. Furthermore, it turn out you prefer vanilla to strawberry, so whenever you are offered a strawberry ice cream, you gladly pay a single cent to change that to an offer of vanilla, ad infinitum.
N/A
You like chocolate ice cream more than vanilla ice cream. Nobody knows if you’ll get ice cream today, but you are asked for your choice just in case, so you pick vanilla.
Let’s consider (2). Suppose someone was in the process of getting Dutch Booked like this. It would not go on ad infinitum. They would quickly learn better. Right? So even if this happened, I think it would not be a big deal.
Let’s say they did learn better. How would they do this—changing their utility function? Someone with a utility function like this really does prefer B+1c to A, C+1c to B, and A+1c to C. Even if they did change their utility function, the new one would either have a new hole or it would obey the results of the VNM-theorem.
So Bayes teaches: do not disobey the laws of logic and math.
Still wondering where the assigning probabilities to truths of theories is.
OK. So what? There’s more to life than that. That’s so terribly narrow. I mean, that part of what you’re saying is right as far as it goes, but it doesn’t go all that far. And when you start trying to apply it to harder cases—what happens? Do you have some Bayesian argument about who to vote for for president? Which convinced millions of people? Or should have convinced them, and really answers the questions much better than other arguments?
Well the Dutch books make it so you have to pick some probabilities. Actually getting the right prior is incomplete, though Solomonoff induction is most of the way there.
Where else are you hoping to go?
In principle, yes. There’s actually a computer program called AIXItl that does it. In practice I use approximations to it. It probably could be done to a very higher degree of certainty. There are a lot of issues and a lot of relevant data.
Can you give an example? Use the ice cream flavors. What probabilities do you have to pick to buy ice cream without being dutch booked?
Explanatory knowledge. Understanding the world. Philosophical knowledge. Moral knowledge. Non-scientific, non-emprical knowledge. Beyond prediction and observation.
How do you know if your approximations are OK to make or ruin things? How do you work out what kinds of approximations are and aren’t safe to make?
The way I would do that is by understanding the explanation of why something is supposed to work. In that way, I can evaluate proposed changes to see whether they mess up the main point or not.
Endo, I think you are making things more confusing by combining issues of Bayesianism with issues of utility. It might help to keep them more separate or to be clear when one is talking about one, the other, or some hybrid.
I use the term Bayesianism to include utility because (a) they are connected and (b) a philosophy of probabilities as abstract mathematical constructs with no applications doesn’t seem complete; it needs an explanation of why those specific objects are studied. How do you think that any of this caused or could cause confusion?
Well, it empirically seems to be causing confusion. See curi’s remarks about the ice cream example. Also, one doesn’t need Bayesianism to include utility and that isn’t standard (although it is true that they do go very well together).
Yes I see what you mean.
I think it goes a bit beyond this. Utility considerations motivate the choice of definitions. I acknowledge that they are distinct things, though.
The consequences could easily be thousands of lives or more in case of sufficiently important decisions.
So the argument is now not that that suboptimal issues don’t exist but that they aren’t a big deal? Are you aware that the primary reason that this involves small amounts of ice cream is for convenience of the example? There’s no reason these couldn’t happen with far more serious issues (such as what medicine to use).
I know. I thought it was strange that you said “ad infinitum” when it would not go on forever. And that you presented this as dire but made your example non-dire.
But OK. You say we must consider probabilities, or this will happen. Well, suppose that if I do something it will happen. I could notice that, criticize it, and thus avoid it.
How can I notice? I imagine you will say that involves probabilities. But in your ice cream example I don’t see the probabilities. It’s just preferences for different ice creams, and an explanation of how you get a loop.
And what I definitely don’t see is probabilities that various theories are true (as opposed to probabilities about events which are ok).
I didn’t say that (I’m not endoself).
Yes, but the Bayesian avoids having this step. For any step you can construct a “criticism” that will duplicate what the Bayesian will do. This is connected to a number of issues, including the fact that what constitutes valid criticism in a Popperian framework is far from clear.
Ice cream is an analogy. It might not be a great one since it is connected to preferences (which sometimes gets confused with Bayesianism). The analogy isn’t a great one. It might make more sense to just go read Cox’s theorem and translate to yourself what the assumptions mean about an approach.
OK, my bad. So many people. I lose track.
Anything which is not itself criticized.
Could you pick any real world example you like, where the probabilities needed to avoid dutch book aren’t obvious, and point them out? To help concretize the idea for me.
Well, I’m not sure, in that I’m not convinced that Dutch Booking really does occur much in real life other than in the obvious contexts. But there are a lot of contexts it does occur in. For example, a fair number of complicated stock maneuvers can be thought of essentially as attempts to dutch book other players in the stock market.
Koth already had an amusing response to that.
Someone here told me it does. Maybe you can go argue with him for me ;-)
I agree.
Consequentialism is not in the index.
Decision rule is, a little bit.
I don’t think this book contains a proof mentioning consequentialism. Do you disagree? Give a page or section?
It looks like what they are doing is defining a decision rule in a special way. So, by definition, it has to be a mathematical thing to do with probability. Then after that, I’m sure it’s rather easy to prove that you should use bayes’ theorem rather than some other math.
But none of that is about decisions rules in the sense of methods human beings use for making decisions. It’s just if you define them in a particular way—so that Bayes’ is basically the only option—then you can prove it.
see e.g. page 19 where they give a definition. A Popperian approach to making decisions simply wouldn’t fit within the scope of their definition, so the conclusion of any proof like you claimed existed (which i haven’t found in this book) would not apply to Popperian ideas.
Maybe there is a lesson here about believing stuff is proven when you haven’t seen the proof, listening to hearsay about what books contain, and trying to apply proofs you aren’t familiar with (they often have limits on scope).
In what way would the Popperian approach fail to fit the decision rule approach on page 19 of Bickel and Doksum?
It says a decision rule (their term) is a function of the sample space, mapping something like complete sets of possible data to things people do. (I think it needs to be complete sets of all your data to be applied to real world human decision making. They don’t explain what they are talking about in the type of way I think is good and clear. I think that’s due to having in mind different problems they are trying to solve than I have. We have different goals without even very much overlap. They both involve “decisions” but we mean different things by the word.)
In real life, people use many different decision rules (my term, not theirs). And people deal with clashes between them.
You may claim that my multiple decision rules can be combined into one mathematical function. That is so. But the result isn’t a smooth function so when they start talking about estimation they have big problems! And this is the kind of thing I would expect to get acknowledgement and discussion if they were trying to talk about how humans make decisions, in practice, rather than just trying to define some terms (chosen to sound like they have something to do with what humans do) and then proceed with math.
e.g. they try to talk about estimating amount of error. if you know error bars on your data, and you have a smooth function, you’re maybe kind of OK with imperfect data. but if your function has a great many jumps in it, what are you to do? what if, within the margin for error on something, there’s several discontinuities? i think they are conceiving of the decision rule function as being smooth and not thinking about what happens when it’s very messy. Maybe they specified some assumptions so that it has to be which I missed, but anyway human beings have tons of contradictory and not-yet-integrated ideas in their head—mistakes and separate topics they haven’t connected yet, and more—and so it’s not smooth.
On a similar note they talk about the median and mean which also don’t mean much when it’s not smooth. Who cares what the mean is over an infinitely large sample space where you get all sorts of unrepresentative results in large unrealistic portions of it? So again I think they are looking at the issues differently than me. They expect things like mathematically friendly distributions (for which means and medians are useful); I don’t.
Moving on to a different issue, they conceive of a decision rule which takes input and then gives output. I do not conceive of people starting with the input and then deciding the output. I think decision making is more complicated. While thinking about the input, people create more input—their thoughts. The input is constantly being changed during the decision process, it’s not a fixed quantity to have a function of. Also being changed during any significant decision is the decision rule itself—it too isn’t a static function even for purposes of doing one decision (at least in the normal sense. maybe they would want to call every step in the process a decision. so when you’re deciding a flavor of ice cream that might involve 50 decisions, with updates to the decisions rules and inputs in between them. if they want to do something like that they do not explain how it works.)
They conceive of the input to decisions as “data”. But I conceive of much thinking as not using much empirical data, if any. I would pick a term that emphasizes it. The input to all decision making is really ideas, some of which are about empirical data and some of which aren’t. Data is a special case, not the right term for the general case. From this I take that they are empiricists. You can find a refutation of empiricism in The Beginning of Infinity by David Deutsch but anyway it’s a difference between us.
A Popperian approach to decision making would focus more on philosophical problems, and their solutions. It would say things like: consider what problem you’re trying to solve, and consider what actions may solve it. And criticize your ideas to eliminate errors. And … well no short summary does it justice. I’ve tried a few times here. But Popperian ways of thinking are not intuitive to people with the justificationist biases dominant in our culture. Maybe if you like everything I said I’ll try to explain more, but in that case I don’t know why you wouldn’t read some books which are more polished than what I would type in. If you have a specific, narrow question I can see that answering that would make sense.
Thank you for that detailed reply. I just have a few comments:
“data” could be any observable property of the world
in statistical decision theory, the details of the decision process that implements the mapping aren’t the focus because we’re going to try to go straight to the optimal mapping in a mathematical fashion
there’s no requirement that the decision function be smooth—it’s just useful to look at such functions first for pedagogical reasons. All of the math continues to work in the presence of discontinuities.
a weak point of statistical decision theory is that it treats the set of actions as a given; human strategic brilliance often finds expression through the realization that a particular action is possible
Yes but using it to refer to a person’s ideas, without clarification, would be bizarre and many readers wouldn’t catch on.
Straight to the final, perfect truth? lol… That’s extremely unPopperian. We don’t expect progress to just end like that. We don’t expect you get so far and then there’s nothing further. We don’t think the scope for reason is so bounded, nor do we think fallibility is so easily defeated.
In practice searches for optimal things of this kind always involve many premises with have substantial philosophical meaning. (Which is often, IMO, wrong.)
Does it use an infinite set of all possible actions? I would have thought it wouldn’t rely on knowing what each action actually is, but would just broadly specify the set of all actions and move on.
@smooth: what good is a mean or median with no smoothness? And for margins of error, with a non-smooth function, what do you do?
With a smooth region of a function, taking the midpoint of the margin of error region is reasonable enough. But when there is a discontinuity, there’s no way to average it and get a good result. Mixing different ideas is a hard process if you want anything useful to result. If you just do it in a simple way like averaging you end up with a result that none of the ideas think will work and shouldn’t be surprised when it doesn’t. It’s kind of like how if you have half an army do one general’s plan, and half do another, the result is worse than doing either one.