Money wants to be linear, but wants even more to be algorithmic
I think this is mixing up two things. First, a diminishing marginal utility in consumption measured in money. This can lead to risk averse behaviour, but it could be any sublinear function, not just logarithmic, and I have seen no reason to think it’s logarithmic in actually existing humans.
if you have risk-averse behavior, other agents can exploit you by selling you insurance.
I wouldn’t call it “exploit”. It’s not a money pump that can be repeated arbitrarily often, its simply a price you pay for stability.
This “money” acts very much like utility, suggesting that utility is supposed to be linear in money.
Only the utility of the agent in question is supposed to be linear in this “money”, and that can always be achieved by a monotone transformation. This is quite different from suggesting there’s a resource everyone should be linear in under the same scaling.
The second thing is the Kelly criterion. The Kelly criterion exist because money can compound. This is also why it produces specifically a logarithmic stucture. Kelly theory recommends you to use the criterion regardsless of the shape of your utility in consumption, if you expect many more games after this one—it is much more like a convergent instrumental goal. So this:
Kelly betting is fully compatible with expected utility maximization, since we can maximize the expectation of the logarithm of money.
is just wrong AFAICT. This is compatible from the side of utility maximization, but not from the side of Kelly as theory. Of course you can always construct a utility function that will behave in a specific way—this isn’t saying much.
This means the previous counterpoint was wrong: expected-money bettors profit in expectation from selling insurance to Kelly bettors, but the Kelly bettors eventually dominate the market
Depends on how you define “dominate the market”. In most worlds, most (by headcount) of the bettors still around will be Kelly bettors. I even think that weighing by money, in most worlds Kelly bettors would outweigh expectation maximizers. But weighing by money across all worlds, the expectation maximizers win—by definition. The Kelly criterion “almost surely” beats any other strategy when played sufficiently long—but it only wins by some amount in the cases where it wins, and its infinitely behind in the infinitely unlikely case that it doesn’t win.
Kelly betting really is incompatible with expectation maximization. It deliberately takes a lower average. The conflict is essentially over two conflicting infinities: Kelly notes that for any sample size, if theres a long enough duration Kelly wins. And maximization notes that for any duration, if theres a big enough sample size maximisation wins.
Money wants to go negative, but can’t.
A lot of what you say here goes into monetary economics, and you should ask someone in the field or at least read up on it before relying on any of this. Propably you shouldn’t rely on it even then, if at all avoidable.
is just wrong AFAICT. This is compatible from the side of utility maximization, but not from the side of Kelly as theory. Of course you can always construct a utility function that will behave in a specific way—this isn’t saying much.
I agree that (1) I’m just constructing a utility function that results in the Kelly behavior, and (2) there’s still a conceptual incompatibility between the classic argument for Kelly and EV theory. But I still think it’s important to point out that the behavioral recommendations of Kelly do not violate the VNM axioms in any way, so the incompatibility is not as great as it may seem. This is important because it would be nice to reconcile the two philosophies, forging a new philosophy which is more robust than either.
Depends on how you define “dominate the market”. In most worlds, most (by headcount) of the bettors still around will be Kelly bettors. I even think that weighing by money, in most worlds Kelly bettors would outweigh expectation maximizers. But weighing by money across all worlds, the expectation maximizers win—by definition.
Right.
Kelly betting really is incompatible with expectation maximization. It deliberately takes a lower average.
And yet it doesn’t violate VNM, which means the classic argument for maximizing expected utility goes through. How can this paradox be resolved? By noting that utility is just whatever quantity expectation maximization does go through for, “by definition” (as I said in the post).
The conflict is essentially over two conflicting infinities: Kelly notes that for any sample size, if theres a long enough duration Kelly wins. And maximization notes that for any duration, if theres a big enough sample size maximisation wins.
Right, agreed.
I’m curious if you’re taking a side, here, wrt which limit one should take.
To the extent that it’s Kelly vs VNM, Kelly seems more practical (applying to real betting), while VNM provides a much more general theory of decision making (since money (or another compounding good) does not need to be present in order for VNM to be relevant).
But I still think it’s important to point out that the behavioral recommendations of Kelly do not violate the VNM axioms in any way, so the incompatibility is not as great as it may seem.
I think the interesting question is what to do when you expect many more, but only finitely many rounds. It seems like Kelly should somehow gradually transition, until it recommends normal utility maximization in the case of only a single round happening ever. Log utility doesn’t do this. I’m not sure I have anything that does though, so maybe it’s unfair to ask it from you, but still it seems like a core part of the idea, that the Kelly strategy comes from the compounding, is lost.
And yet it doesn’t violate VNM, which means the classic argument for maximizing expected utility goes through. How can this paradox be resolved? By noting that utility is just whatever quantity expectation maximization does go through for, “by definition”.
This is the sort of argument you want to be very suspicious of if youre confused, as I suspect we are. For example, you can now just apply all the arguments that made Kelly seem compelling again, but this time with respect to the new, logarithmic utility function. Do they actually seem less compelling now? A little bit, yes, because I think we really are sublinear in money, and the intuitions related to that went away. But no matter what the utility function, we can always construct bets that are compounding in utility, and then bettors which are Kelly with respect to that utility function will come to dominate the market. So if you do this reverse-inference of utility, the utility function of Kelly bettors will seems to change based on the bets offered.
I’m curious if you’re taking a side, here, wrt which limit one should take.
Not really, I think we’re to confused to say yet. I do think I understand decisions with bounded utility (all the classical foundations imply bounded utilities, including VNM. This doesn’t seem to be well known here). Bounded utility makes maximization a lot more Kelly: it means that the maximizers can no longer have the arbitrarily high pay-offs that are needed to balance the near-certainty of elimination. I also think it should make it not matter which limit you take first, but I don’t think that leads to Kelly, either, because the betting structure that leads to Kelly assumes unbounded utility. Perhaps it would end up as a local approximation somewhere.
Now I also think that bounded decision theory is inadequate. I think a decision theory should be able to implement a paperclip maximizer, and it should work in worlds that last infinitely long. But I don’t have something that fulfills that. I think theres a good chance the solution doesn’t look like utility at all: A theorem that needs its problem to be finite propably won’t do well in embedded problems.
I think the interesting question is what to do when you expect many more, but only finitely many rounds. It seems like Kelly should somehow gradually transition, until it recommends normal utility maximization in the case of only a single round happening ever. Log utility doesn’t do this. I’m not sure I have anything that does though, so maybe it’s unfair to ask it from you, but still it seems like a core part of the idea, that the Kelly strategy comes from the compounding, is lost.
Ah, I see, interesting.
This is the sort of argument you want to be very suspicious of if youre confused, as I suspect we are. For example, you can now just apply all the arguments that made Kelly seem compelling again, but this time with respect to the new, logarithmic utility function. Do they actually seem less compelling now? A little bit, yes, because I think we really are sublinear in money, and the intuitions related to that went away. But no matter what the utility function, we can always construct bets that are compounding in utility, and then bettors which are Kelly with respect to that utility function will come to dominate the market. So if you do this reverse-inference of utility, the utility function of Kelly bettors will seems to change based on the bets offered.
Yeah, I agree with this.
(all the classical foundations imply bounded utilities, including VNM. This doesn’t seem to be well known here)
Yeah. I’m generally OK with dropping continuity-type axioms, though, in which case you can have hyperreal/surreal utility to deal with expectations which would otherwise be problematic (the divergent sums which unbounded utility allows). So while I agree that boundedness should be thought of as part of the classical notion of real-valued utility, this doesn’t seem like a huge deal to me.
OTOH, logical uncertainty / radical probabilism introduce new reasons to require boundedness for expectations. What is the expectation of the self-referential quantity “one greater than your expectation for this value”? This seems problematic even with hyperreals/surreals. And we could embed such a quantity into a decision problem.
I’m generally OK with dropping continuity-type axioms, though, in which case you can have hyperreal/surreal utility to deal with expectations which would otherwise be problematic (the divergent sums which unbounded utility allows).
Have you worked this out somewhere? I’d be interested to see it but I think there are some divergences it can’t adress. There is for one the Pasadena paradox, which is also a divergent sum but one which doesn’t stably lead anywhere, not even to infinity. The second is an apparently circular dominance relation: Imagine you are linear in monetary consumption. You start with 1$ which you can either spend or leave in the bank, which doubles it every year even after accounting for your time preference/uncertainty/other finite discounting. Now for every n, leaving it in the bank for n+1 years dominates leaving it for n years, but leaving it in the bank forever gets 0 utility. Note that if we replace money with energy here, this could actually happen in universes not too different from ours.
What is the expectation of the self-referential quantity “one greater than your expectation for this value”?
What is the expectation of the self-referential quantity “one greater than your expectation for this value, except when that would go over the maximum, in which case it’s one lower than expectation instead”? Insofar as there is an answer it would have to be “one less than maximum”, but that would seem to require uncertainty about what your expectations are.
Have you worked this out somewhere? I’d be interested to see it but I think there are some divergences it can’t adress.
It’s a bit of a mess due to some formatting changes porting to LW 2.0, but here it is.
I’ve gotten the impression over the years that there are a lot of different ways to arrive at the same conclusion, although I unfortunately don’t have all my sources lined up in one place.
I think if you just drop continuity from VNM you get this kind of picture, because the VNM continuity assumption corresponds to the Archimedian assumption for the reals.
I think there’s a variant of Cox’s theorem which similarly yields hyperreal/surreal probabilities (infinitesimals, not infinities, in that case).
If you want to condition on probability zero events, you might do so by rejecting the ratio formula for conditional probabilities, and instead giving a basic axiomatization of conditional probability in its own right. It turns out that, at least under one such axiom system, this is equivalent to allowing infinitesimal probability and keeping the ratio definition of conditional probability.
(Sorry for not having the sources at the ready.)
There is for one the Pasadena paradox, which is also a divergent sum but one which doesn’t stably lead anywhere, not even to infinity.
Here’s how it works. I have to assign expectations to gambles. I have some consistency requirements in how I do this; for example, if you modify a gamble g by making a probability p outcome have v less value, then I must think the new gamble g′ is worth p⋅v less. However, how I assign value to divergent sums is subjective—it cannot be determined precisely from how I assign value to each of the elements of the sum, because I’m not trying to assume anything like countable additivity.
In a case like the St Petersburg Lottery, I believe I’m required to have some infinite expectation. But it’s up to me what it is, since there’s no one way to assign expectations in infinite hyperreal/surreal sums.
In a case like the Pasadena paradox, though, I’m thinking I’ll be subjectively allowed to assign any expectation whatsoever—so long as all my other infinite-sum expectations are consistent with the assignment.
You start with 1$ which you can either spend or leave in the bank, which doubles it every year even after accounting for your time preference/uncertainty/other finite discounting. Now for every n, leaving it in the bank for n+1 years dominates leaving it for n years, but leaving it in the bank forever gets 0 utility. Note that if we replace money with energy here, this could actually happen in universes not too different from ours.
Perhaps you can try to problematize this example for me given what I’ve written above—not sure if I’ve already addressed your essential worry here or not.
What is the expectation of the self-referential quantity “one greater than your expectation for this value, except when that would go over the maximum, in which case it’s one lower than expectation instead”? Insofar as there is an answer it would have to be “one less than maximum”, but that would seem to require uncertainty about what your expectations are.
Yes, uncertainty about your own expectations is where this takes us. But that seems quite reasonable, especially because we only need a very small amount of uncertainty, as is illustrated in this example.
However, how I assign value to divergent sums is subjective—it cannot be determined precisely from how I assign value to each of the elements of the sum, because I’m not trying to assume anything like countable additivity.
This implies that you believe in the existence of countably infinite bets but not countably infinite dutch booking processes. Thats seems like a strange/unphysical position to be in—if that were the best treatment of infinity possible, I think infinity is better abandoned. Im not even sure the framework in your linked post can really be said to contain infinte bets: the only way a bet ever gets evaluated is in a bookie strategy, and no single bookie strategy can be guaranteed to fully evaluate an infinte bet. Is there a single bookie strategy that differentiates the St. Petersburg bet from any finite bet? Because if no, then the agent at least cant distinguish them, which is very close to not existing at all here.
In a case like the St Petersburg Lottery, I believe I’m required to have some infinite expectation.
Why? I haven’t found any finite dutch books against not doing so.
Perhaps you can try to problematize this example for me given what I’ve written above—not sure if I’ve already addressed your essential worry here or not.
I dont think you have. That example doesn’t involve any uncertainty or infinite sums. The problem is that for any finite n, waiting n+1 is better than waiting n, but waiting indefinitely is worse than any. Formally, the problem is that I have a complete and transitive preference between actions, but no unique best action, just a series that keeps getting better.
Note that you talk about something related in your linked post:
I’m representing preferences on sets only so that I can argue that this reduces to binary preference.
But the proof for that reduction only goes one way: for any preference relation on sets, theres a binary one. My problem is that the inverse does not hold.
I think this is mixing up two things. First, a diminishing marginal utility in consumption measured in money. This can lead to risk averse behaviour, but it could be any sublinear function, not just logarithmic, and I have seen no reason to think it’s logarithmic in actually existing humans.
I wouldn’t call it “exploit”. It’s not a money pump that can be repeated arbitrarily often, its simply a price you pay for stability.
Only the utility of the agent in question is supposed to be linear in this “money”, and that can always be achieved by a monotone transformation. This is quite different from suggesting there’s a resource everyone should be linear in under the same scaling.
The second thing is the Kelly criterion. The Kelly criterion exist because money can compound. This is also why it produces specifically a logarithmic stucture. Kelly theory recommends you to use the criterion regardsless of the shape of your utility in consumption, if you expect many more games after this one—it is much more like a convergent instrumental goal. So this:
is just wrong AFAICT. This is compatible from the side of utility maximization, but not from the side of Kelly as theory. Of course you can always construct a utility function that will behave in a specific way—this isn’t saying much.
Depends on how you define “dominate the market”. In most worlds, most (by headcount) of the bettors still around will be Kelly bettors. I even think that weighing by money, in most worlds Kelly bettors would outweigh expectation maximizers. But weighing by money across all worlds, the expectation maximizers win—by definition. The Kelly criterion “almost surely” beats any other strategy when played sufficiently long—but it only wins by some amount in the cases where it wins, and its infinitely behind in the infinitely unlikely case that it doesn’t win.
Kelly betting really is incompatible with expectation maximization. It deliberately takes a lower average. The conflict is essentially over two conflicting infinities: Kelly notes that for any sample size, if theres a long enough duration Kelly wins. And maximization notes that for any duration, if theres a big enough sample size maximisation wins.
A lot of what you say here goes into monetary economics, and you should ask someone in the field or at least read up on it before relying on any of this. Propably you shouldn’t rely on it even then, if at all avoidable.
I agree that (1) I’m just constructing a utility function that results in the Kelly behavior, and (2) there’s still a conceptual incompatibility between the classic argument for Kelly and EV theory. But I still think it’s important to point out that the behavioral recommendations of Kelly do not violate the VNM axioms in any way, so the incompatibility is not as great as it may seem. This is important because it would be nice to reconcile the two philosophies, forging a new philosophy which is more robust than either.
Right.
And yet it doesn’t violate VNM, which means the classic argument for maximizing expected utility goes through. How can this paradox be resolved? By noting that utility is just whatever quantity expectation maximization does go through for, “by definition” (as I said in the post).
Right, agreed.
I’m curious if you’re taking a side, here, wrt which limit one should take.
To the extent that it’s Kelly vs VNM, Kelly seems more practical (applying to real betting), while VNM provides a much more general theory of decision making (since money (or another compounding good) does not need to be present in order for VNM to be relevant).
I think the interesting question is what to do when you expect many more, but only finitely many rounds. It seems like Kelly should somehow gradually transition, until it recommends normal utility maximization in the case of only a single round happening ever. Log utility doesn’t do this. I’m not sure I have anything that does though, so maybe it’s unfair to ask it from you, but still it seems like a core part of the idea, that the Kelly strategy comes from the compounding, is lost.
This is the sort of argument you want to be very suspicious of if youre confused, as I suspect we are. For example, you can now just apply all the arguments that made Kelly seem compelling again, but this time with respect to the new, logarithmic utility function. Do they actually seem less compelling now? A little bit, yes, because I think we really are sublinear in money, and the intuitions related to that went away. But no matter what the utility function, we can always construct bets that are compounding in utility, and then bettors which are Kelly with respect to that utility function will come to dominate the market. So if you do this reverse-inference of utility, the utility function of Kelly bettors will seems to change based on the bets offered.
Not really, I think we’re to confused to say yet. I do think I understand decisions with bounded utility (all the classical foundations imply bounded utilities, including VNM. This doesn’t seem to be well known here). Bounded utility makes maximization a lot more Kelly: it means that the maximizers can no longer have the arbitrarily high pay-offs that are needed to balance the near-certainty of elimination. I also think it should make it not matter which limit you take first, but I don’t think that leads to Kelly, either, because the betting structure that leads to Kelly assumes unbounded utility. Perhaps it would end up as a local approximation somewhere.
Now I also think that bounded decision theory is inadequate. I think a decision theory should be able to implement a paperclip maximizer, and it should work in worlds that last infinitely long. But I don’t have something that fulfills that. I think theres a good chance the solution doesn’t look like utility at all: A theorem that needs its problem to be finite propably won’t do well in embedded problems.
Ah, I see, interesting.
Yeah, I agree with this.
Yeah. I’m generally OK with dropping continuity-type axioms, though, in which case you can have hyperreal/surreal utility to deal with expectations which would otherwise be problematic (the divergent sums which unbounded utility allows). So while I agree that boundedness should be thought of as part of the classical notion of real-valued utility, this doesn’t seem like a huge deal to me.
OTOH, logical uncertainty / radical probabilism introduce new reasons to require boundedness for expectations. What is the expectation of the self-referential quantity “one greater than your expectation for this value”? This seems problematic even with hyperreals/surreals. And we could embed such a quantity into a decision problem.
Have you worked this out somewhere? I’d be interested to see it but I think there are some divergences it can’t adress. There is for one the Pasadena paradox, which is also a divergent sum but one which doesn’t stably lead anywhere, not even to infinity. The second is an apparently circular dominance relation: Imagine you are linear in monetary consumption. You start with 1$ which you can either spend or leave in the bank, which doubles it every year even after accounting for your time preference/uncertainty/other finite discounting. Now for every n, leaving it in the bank for n+1 years dominates leaving it for n years, but leaving it in the bank forever gets 0 utility. Note that if we replace money with energy here, this could actually happen in universes not too different from ours.
What is the expectation of the self-referential quantity “one greater than your expectation for this value, except when that would go over the maximum, in which case it’s one lower than expectation instead”? Insofar as there is an answer it would have to be “one less than maximum”, but that would seem to require uncertainty about what your expectations are.
It’s a bit of a mess due to some formatting changes porting to LW 2.0, but here it is.
I’ve gotten the impression over the years that there are a lot of different ways to arrive at the same conclusion, although I unfortunately don’t have all my sources lined up in one place.
I think if you just drop continuity from VNM you get this kind of picture, because the VNM continuity assumption corresponds to the Archimedian assumption for the reals.
I think there’s a variant of Cox’s theorem which similarly yields hyperreal/surreal probabilities (infinitesimals, not infinities, in that case).
If you want to condition on probability zero events, you might do so by rejecting the ratio formula for conditional probabilities, and instead giving a basic axiomatization of conditional probability in its own right. It turns out that, at least under one such axiom system, this is equivalent to allowing infinitesimal probability and keeping the ratio definition of conditional probability.
(Sorry for not having the sources at the ready.)
Here’s how it works. I have to assign expectations to gambles. I have some consistency requirements in how I do this; for example, if you modify a gamble g by making a probability p outcome have v less value, then I must think the new gamble g′ is worth p⋅v less. However, how I assign value to divergent sums is subjective—it cannot be determined precisely from how I assign value to each of the elements of the sum, because I’m not trying to assume anything like countable additivity.
In a case like the St Petersburg Lottery, I believe I’m required to have some infinite expectation. But it’s up to me what it is, since there’s no one way to assign expectations in infinite hyperreal/surreal sums.
In a case like the Pasadena paradox, though, I’m thinking I’ll be subjectively allowed to assign any expectation whatsoever—so long as all my other infinite-sum expectations are consistent with the assignment.
Perhaps you can try to problematize this example for me given what I’ve written above—not sure if I’ve already addressed your essential worry here or not.
Yes, uncertainty about your own expectations is where this takes us. But that seems quite reasonable, especially because we only need a very small amount of uncertainty, as is illustrated in this example.
This implies that you believe in the existence of countably infinite bets but not countably infinite dutch booking processes. Thats seems like a strange/unphysical position to be in—if that were the best treatment of infinity possible, I think infinity is better abandoned. Im not even sure the framework in your linked post can really be said to contain infinte bets: the only way a bet ever gets evaluated is in a bookie strategy, and no single bookie strategy can be guaranteed to fully evaluate an infinte bet. Is there a single bookie strategy that differentiates the St. Petersburg bet from any finite bet? Because if no, then the agent at least cant distinguish them, which is very close to not existing at all here.
Why? I haven’t found any finite dutch books against not doing so.
I dont think you have. That example doesn’t involve any uncertainty or infinite sums. The problem is that for any finite n, waiting n+1 is better than waiting n, but waiting indefinitely is worse than any. Formally, the problem is that I have a complete and transitive preference between actions, but no unique best action, just a series that keeps getting better.
Note that you talk about something related in your linked post:
But the proof for that reduction only goes one way: for any preference relation on sets, theres a binary one. My problem is that the inverse does not hold.