You do have a utility function (though it may be stochastic). You just don’t know what it is. “Utility function” means the same thing as “decision function”; it just has different connotations. Something determines how you act; that something is your utility function, even if it can be described only as a physics problem plus random numbers generated by your free will and adjustments made by God. (God must be encapsulated in an oracle function.) We call it a utility function to clue people into our purposes and the literature that we’re going to draw on for our analysis. If we wished to regard a thing as deterministic rather than as an agent with free will, we would call its decision function a probability density function instead of a utility function.
I wonder if there is any coherent sense in which I could be said to have terminal values at all.
If you truly have terminal values, they are mainly described by a large matrix of synaptic connections and weights.
When you say “I don’t have a utility function” or “I don’t have terminal values”, you are mostly complaining that approximations are only approximations. You are thinking about some approximation of your utility function or your terminal values, expressed in language or logic, using symbols that conveniently but inaccurately cluster all possible sense-experience vectors into categories, and logical operations that throw away all information but the symbols (and perhaps some statistics, such as a probability or typicality for each symbol).
When we use the words “utility function”, the level of abstraction to use to describe it, and hence its accuracy, depends on the purpose we have in mind. What’s incoherent is talking about “my utility function” absent any such purpose. It’s just like asking “What is the length of the coast of England?”
Whether you have terminal values is a more-complicated question, for uninteresting reasons such as quantum mechanical considerations. The short answer is probably, Any level of abstraction that is simple enough for you to think about, is too simple to capture values that are guaranteed not to change.
Underneath both these questions is the tricky question, “Which me is me?” Are you asking about the utility function enacted by the set of SNPs in your DNA, by your body, or by your conscious mind? These are not the same utility functions. (Whether your conscious mind has a utility function is a tricky question because we would have to separate actions controlled by your conscious mind from actions your body takes not controlled by your conscious mind. If consciousness is epiphenomenal, your mind does not have a useful utility function.)
One common use of terminal values on LW is to try to divine a set of terminal values for humans that can be used to guide an AI. So a specific, meaningful, useful question would be, “Can I discover and describe my terminal values in enough detail that I can be confident that an AI, controlled by these values, will enact the coherent extrapolated volition of these values?” (“Coherent extrapolated volition” may be meaningless, but that’s a separate issue.) I believe the answer is no, which is one reason why I don’t support MIRI’s efforts toward FAI.
Eliezer spent a lot of time years ago explaining in detail why giving an AI goals like “Make humans happy” is problematic, and began to search for the appropriate level of description of goals/values. He unfortunately didn’t pursue this to its conclusion, and chose to focus on errors caused by drift from the original utility function, or by logics that fail to achieve rationality, to the exclusion of consideration of changes caused by the inevitable inexactness of a representation of a utility function and the random component of the original utility function, or of the tricky ontological questions that crop up when you ask, “Whose utility function?”
“Utility function” means the same thing as “decision function”
This contradicts my knowledge. By “utility function”, I mean that thing which VNM proves exists; a mapping from possible worlds to real numbers.
Where are the references for “utility function” being interchangable with “decision algorithm”? I have never seen that stated in any technical discussion of decisions.
If we wished to regard a thing as deterministic rather than as an agent with free will, we would call its decision function a probability density function instead of a utility function.
I’m confused.
Do you just mean the difference between modeling a thing as an agent, vs modeling it as a causal system?
Can you elaborate on how this relates here?
Underneath both these questions is the tricky question, “Which me is me?” Are you asking about the utility function enacted by the set of SNPs in your DNA, by your body, or by your conscious mind? These are not the same utility functions.
Agree. Moral philosophy is hard. I’m working on it.
One common use of terminal values on LW is to try to divine a set of terminal values for humans that can be used to guide an AI. So a specific, meaningful, useful question would be, “Can I discover and describe my terminal values in enough detail that I can be confident that an AI, controlled by these values, will enact the coherent extrapolated volition of these values?” … I believe the answer is no
Can you elaborate on why you think it is impossible for a machine to do good things? Or why such a question is meaningless?
Whether you have terminal values is a more-complicated question, for uninteresting reasons such as quantum mechanical considerations.
What? Not having terminal values means that either you don’t care about anything at all, or that “the recursive chain of valuableness” is infinitely deep. Neither of these seems likely to me.
“Whenever you do anything, that which determines your action—whatever it may be—can be called a decision—or utility—function. You are doing something, ergo you have a utility function.” [ ]
I think Phil is confusing the economist’s (descriptive) utility function, with the VNM-ethicist’s (prescriptive) utility function. Come to think of it, a case could be made that the VNM-utilitarian is similarly confused.
By VNM-utilitarianism I mean the moral theories that one should act to maximize a utility function. Around here this is sometimes called “consequentialism” or simply “utilitarianism”. Unfortunately, both terms are ambiguous. It’s possible to have consequentialist theories that aren’t based on a utility function, and “utilitarianism” is also used to mean the theory with the specific utility function of total happiness. Thus, I’ve taken to using “VNM-utilitarianism” as a hopefully less ambiguous and self-explanatory term.
As for what I think of VNM-utilitarianism this comment gives a brief summery.
By VHM-utilitarianism I mean the moral theories that one should act to maximize a utility function. Around here this is sometimes called “consequentialism” or simply “utilitarianism”.
When it is called ‘utilitarianism’ there are other people who call it wrong. I recommend saying consequentialism to avoid confusion. Mind you, I don’t even know what you mean by those letters (VHM). My best guess is that you mean the Von Neumann Morgenstern utility theorem but got the letters wrong.
If you are referring to those axioms then you could also consider saying VNM-utility instead of VNM-utilitarianism. Because those words have meanings that are far more different than their etymology might suggest to you.
My best guess is that you mean the Von Neumann Morgenstern utility theorem but got the letters wrong.
Oops. Fixed.
If you are referring to those axioms then you could also consider saying VNM-utility instead of VNM-utilitarianism. Because those words have meanings that are far more different than their etymology might suggest to you.
That’s why I talk about “VNM-utilitarianism” rather than simply “utilitarianism”.
That’s why I talk about “VNM-utilitarianism” rather than simply “utilitarianism”.
That isn’t enough to disambiguate the meaning. In fact, your intended meaning is not even one of the options to disambiguate between. Your usage is still wrong and misleading. I suggest following nshepperd’s advice and using “VNM-rational” or “VNM-ratinality”.
(Obviously I will be downvoting all comments that persist with “VNM-utilitarianism”. Many others will not downvote but will take your muddled terminology to be strong evidence that you are confused or ill-informed about the subject matter.)
That isn’t enough to disambiguate the meaning. In fact, your intended meaning is not even one of the options to disambiguate between.
Utilitarianism in practice means some kind of aggregation of all people’s preferences. Most typically either ‘total’ or ‘average’. Even though I am a consequentialist (at least in a highly abstract combatibilist sense) I dismiss utilitarianism as stupid, arbitrary and not worth priveleging as a moral hypothesis. Adding VNM to it effectively narrows it down to ‘preference utilitarianism’ which at least gets rid of the worst of the crazy (‘hedonic utilitarianism’ Gahh!). But I don’t think that is what you are trying to refer to when you challenge VNM-X (because it wouldn’t be compatible with the points you make).
How about “VNM-consequentialism”?
Perfect! Please do. ‘Consequentialism’ means what one would naively expect ‘utilitarianism’ to mean, if not for an unfortunate history of bad philosophy having defined the term already. The VNM qualifier then narrows consequentialism down to the typical case that we tend to mean around here (because you are right, technically consequentialism is more broad than just that based on VNM axioms.)
I believe “VNM-utilitarianism” is problematic because it would suggest that it is a kind of utilitarianism. By the most usual definition of “utilitarianism” (a moral theory involving an ‘objective’ aggregative measure of value + utility-maximising decision theory) it is not.
However, I remember “VNM-rational” and “VNM-rationality” being accepted terminology.
No, I don’t think it’s just descriptive vs prescriptive, I mentioned both in my post and asserted that we had neither. Phil is saying that we do have a decision algorithm (I agree), and further, that “utility function” means “decision algorithm” (which I disagree with, but I’m not one to argue terminology.
Economists frequently assume humans have utility functions as part of their spherical cow model of human behavior. Unfortunately, they sometimes forget that this is just a spherical cow model, especially once one gets away from modeling collective economic behavior.
You do have a utility function (though it may be stochastic). You just don’t know what it is. “Utility function” means the same thing as “decision function”; it just has different connotations. Something determines how you act; that something is your utility function, even if it can be described only as a physics problem plus random numbers generated by your free will and adjustments made by God. (God must be encapsulated in an oracle function.) We call it a utility function to clue people into our purposes and the literature that we’re going to draw on for our analysis. If we wished to regard a thing as deterministic rather than as an agent with free will, we would call its decision function a probability density function instead of a utility function.
If you truly have terminal values, they are mainly described by a large matrix of synaptic connections and weights.
When you say “I don’t have a utility function” or “I don’t have terminal values”, you are mostly complaining that approximations are only approximations. You are thinking about some approximation of your utility function or your terminal values, expressed in language or logic, using symbols that conveniently but inaccurately cluster all possible sense-experience vectors into categories, and logical operations that throw away all information but the symbols (and perhaps some statistics, such as a probability or typicality for each symbol).
When we use the words “utility function”, the level of abstraction to use to describe it, and hence its accuracy, depends on the purpose we have in mind. What’s incoherent is talking about “my utility function” absent any such purpose. It’s just like asking “What is the length of the coast of England?”
Whether you have terminal values is a more-complicated question, for uninteresting reasons such as quantum mechanical considerations. The short answer is probably, Any level of abstraction that is simple enough for you to think about, is too simple to capture values that are guaranteed not to change.
Underneath both these questions is the tricky question, “Which me is me?” Are you asking about the utility function enacted by the set of SNPs in your DNA, by your body, or by your conscious mind? These are not the same utility functions. (Whether your conscious mind has a utility function is a tricky question because we would have to separate actions controlled by your conscious mind from actions your body takes not controlled by your conscious mind. If consciousness is epiphenomenal, your mind does not have a useful utility function.)
One common use of terminal values on LW is to try to divine a set of terminal values for humans that can be used to guide an AI. So a specific, meaningful, useful question would be, “Can I discover and describe my terminal values in enough detail that I can be confident that an AI, controlled by these values, will enact the coherent extrapolated volition of these values?” (“Coherent extrapolated volition” may be meaningless, but that’s a separate issue.) I believe the answer is no, which is one reason why I don’t support MIRI’s efforts toward FAI.
Eliezer spent a lot of time years ago explaining in detail why giving an AI goals like “Make humans happy” is problematic, and began to search for the appropriate level of description of goals/values. He unfortunately didn’t pursue this to its conclusion, and chose to focus on errors caused by drift from the original utility function, or by logics that fail to achieve rationality, to the exclusion of consideration of changes caused by the inevitable inexactness of a representation of a utility function and the random component of the original utility function, or of the tricky ontological questions that crop up when you ask, “Whose utility function?”
This contradicts my knowledge. By “utility function”, I mean that thing which VNM proves exists; a mapping from possible worlds to real numbers.
Where are the references for “utility function” being interchangable with “decision algorithm”? I have never seen that stated in any technical discussion of decisions.
I’m confused.
Do you just mean the difference between modeling a thing as an agent, vs modeling it as a causal system?
Can you elaborate on how this relates here?
Agree. Moral philosophy is hard. I’m working on it.
Can you elaborate on why you think it is impossible for a machine to do good things? Or why such a question is meaningless?
Tricky question indeed. Again, working on it.
I have a utility function, but it is not time-invariant, and is often not continuous on the time axis.
And I’m a universe. Just a bit stochastic around the edges...
Universes are like that. Are you deterministic, purely stochastic, or do you make decisions?
What? Not having terminal values means that either you don’t care about anything at all, or that “the recursive chain of valuableness” is infinitely deep. Neither of these seems likely to me.
I think there’s a third possibility: values have a circular, strange-loop structure.
As far as I can tell, you aren’t making any argument for your position that we have utility functions. You are merely asserting it.
“Whenever you do anything, that which determines your action—whatever it may be—can be called a decision—or utility—function. You are doing something, ergo you have a utility function.” [ ]
I have never seen “utility function” used like this in any technical discussion. Am I missing something?
I think Phil is confusing the economist’s (descriptive) utility function, with the VNM-ethicist’s (prescriptive) utility function. Come to think of it, a case could be made that the VNM-utilitarian is similarly confused.
Care to expand on what you mean by VNM-utilitarian? You refer to it a lot and I’m never quite sure what you mean.
(I’m also interested in what you think of it).
By VNM-utilitarianism I mean the moral theories that one should act to maximize a utility function. Around here this is sometimes called “consequentialism” or simply “utilitarianism”. Unfortunately, both terms are ambiguous. It’s possible to have consequentialist theories that aren’t based on a utility function, and “utilitarianism” is also used to mean the theory with the specific utility function of total happiness. Thus, I’ve taken to using “VNM-utilitarianism” as a hopefully less ambiguous and self-explanatory term.
As for what I think of VNM-utilitarianism this comment gives a brief summery.
When it is called ‘utilitarianism’ there are other people who call it wrong. I recommend saying consequentialism to avoid confusion. Mind you, I don’t even know what you mean by those letters (VHM). My best guess is that you mean the Von Neumann Morgenstern utility theorem but got the letters wrong.
If you are referring to those axioms then you could also consider saying VNM-utility instead of VNM-utilitarianism. Because those words have meanings that are far more different than their etymology might suggest to you.
Oops. Fixed.
That’s why I talk about “VNM-utilitarianism” rather than simply “utilitarianism”.
That isn’t enough to disambiguate the meaning. In fact, your intended meaning is not even one of the options to disambiguate between. Your usage is still wrong and misleading. I suggest following nshepperd’s advice and using “VNM-rational” or “VNM-ratinality”.
(Obviously I will be downvoting all comments that persist with “VNM-utilitarianism”. Many others will not downvote but will take your muddled terminology to be strong evidence that you are confused or ill-informed about the subject matter.)
I’m curious, what were the options for what you thought it meant.
How about “VNM-consequentialism”?
Utilitarianism in practice means some kind of aggregation of all people’s preferences. Most typically either ‘total’ or ‘average’. Even though I am a consequentialist (at least in a highly abstract combatibilist sense) I dismiss utilitarianism as stupid, arbitrary and not worth priveleging as a moral hypothesis. Adding VNM to it effectively narrows it down to ‘preference utilitarianism’ which at least gets rid of the worst of the crazy (‘hedonic utilitarianism’ Gahh!). But I don’t think that is what you are trying to refer to when you challenge VNM-X (because it wouldn’t be compatible with the points you make).
Perfect! Please do. ‘Consequentialism’ means what one would naively expect ‘utilitarianism’ to mean, if not for an unfortunate history of bad philosophy having defined the term already. The VNM qualifier then narrows consequentialism down to the typical case that we tend to mean around here (because you are right, technically consequentialism is more broad than just that based on VNM axioms.)
I believe “VNM-utilitarianism” is problematic because it would suggest that it is a kind of utilitarianism. By the most usual definition of “utilitarianism” (a moral theory involving an ‘objective’ aggregative measure of value + utility-maximising decision theory) it is not.
However, I remember “VNM-rational” and “VNM-rationality” being accepted terminology.
No, I don’t think it’s just descriptive vs prescriptive, I mentioned both in my post and asserted that we had neither. Phil is saying that we do have a decision algorithm (I agree), and further, that “utility function” means “decision algorithm” (which I disagree with, but I’m not one to argue terminology.
Economists frequently assume humans have utility functions as part of their spherical cow model of human behavior. Unfortunately, they sometimes forget that this is just a spherical cow model, especially once one gets away from modeling collective economic behavior.