First of all, I want to state that Ergodicity Economics is great and I’m glad that it exists—like I said in my other comment, it’s a true frequentist alternative to Bayesian decision theory, and it does some cool things, and I would like to see it developed further.
That said, I want to defend two claims:
From a strict Bayesian perspective, it’s all nonsense, so those arguments in favor of Kelly are at best heuristic. A Bayesian should be much more interested in the question of whether their utility is log in money. (Note: I am not a strict bayesian, although I think strict Bayes is a pretty good starting point.)
From a broader perspective, I think Peters’ direction is interesting, but his current methodology is not very satisfying, and doesn’t support his strong claims that EG Kelly naturally pops out when you take a time-averaging rather than ensemble-averaging perspective.
“It’s All Nonsense”
Let’s deal with the strict Bayesian perspective first.
You start your argument by discussing the “variance drag” phenomenon. You point out that the expected monetary value of an investment can be constant or even increasing, but it may be that due to variance drag, it typically (with high probability) goes down over time.
You go on to assert that “people need to account for this” and “adjust their strategy” because “variance is bad”.
To a strict Bayesian, this is a non-sequitur. If it be the case that a strict Bayesian is a money-maximizer (IE, has utility linear in money), they’ll see no problem with variance drag. You still put your money where it gets the highest expected value.
Furthermore, repeated bets don’t make any material difference to the money-maximizer. Suppose the optimal one-step investment strategy has a tare of return R. Then the best two-step strategy is to use that strategy twice to get an expected rate of return R^2. This is because the best way they can set themselves up to succeed at stage 2 (in expectation!) is to get as much money as they can at stage 1 (in expectation!). This reasoning applies no matter how many stages we string together. So to the money-maximizer, your argument about repeated bets is totally wrong.
Where a strict Bayesian would start to agree with you is if they don’t have utility linear in money, but rather, have diminishing returns.
In that case, the best strategy for single-step investing can differ from the best strategy for multi-step investing.
Say the utility function is u(x). Treat an investment strategy as a stochastic function s(x). We assume the random variable c⋅s(x) behaves the same as the random variable s(c⋅x); IE, returns are proportional to investment. So, actually, we can treat a strategy as a random variable S which gets multiplied by our money; s(x)=S⋅x.
In a one-step scenario, the Bayesian wants to maximize E[u(s(x))] where x is your starting money. In a two-step scenario, the Bayesian wants to maximize E[u(s2(s1(x)))]. And so on. The reason the strategy was so simple when u(x)=x was that this allowed us to push the expectation inwards; E[u(s2(s1(x)))]=E[s2(s1(x))]=E[S2⋅S1⋅x]=E[S1]⋅E[S2]⋅x (the last step holds because we assume the random variables are independent). So in that case, we could just choose the best individual strategy and apply it at each time-step.
In general, however, nonlinear u stops us from pushing the expectations inward. So multi-step matters.
A many-step problem looks like E[u(S1⋅S2⋅S3⋅...⋅Sn⋅x)]. The product of many independent random variables will closely approximate a log-normal distribution, by the multiplicative central limit theorem. I think this somehow implies that if the Bayesian had to select the same strategy to be used on all days, the Bayesian would be pretty happy for it to be a log-wealth maximizing strategy, but I’m not connecting the dots on the argument right now.
So anyway, under some assumptions, I think a Bayesian will behave increasingly like a log-wealth maximizer as the number of steps goes up.
But this is because the Bayesian had diminishing returns in the first place. So it’s still ultimately about the utility function.
Most attempts to justify Kelly by repeated-bet phenomena overtly or implicitly rely on an assumption that we care about typical outcomes. A strict Bayesian will always stop you there. “Hold on now. I care about average outcomes!”
(And Ergodicity Economics says: so much the worse for Bayes!)
Bayesians may also raise other concerns with Peters-style reasoning:
Ensemble averaging is the natural response to decision-making under uncertainty; you’re averaging over different possibilities. When you try to time-average to get rid of your uncertainty, you have to ask “time average what?”—you don’t know what specific situation you’re in.
In general, the question of how to turn your current situation into a repeated sequence for the purpose of time-averaging analysis seems under-determined (even if you are certain about your present situation). Surely Peters doesn’t want us to use actual time in the analysis; in actual time, you end up dead and lose all your money, so the time-average analysis is trivial. So Peters has to invent an imagined time dimension, in which the game goes on forever. In contrast, Bayesians can deal with time as it actually is, accounting for the actual number of iterations of the game, utility of money at death, etc.
Even if you settle on a way to turn the situation into an iterated sequence, the necessary limit does not necessarily exist. This is also true of the possibility-average, of course (the St Petersburg Paradox being a classic example); but it seems easier to get failure in the time-avarage case, because you just need non-convergence; ie, you don’t need any unbounded stuff to happen.
It’s Not Nonsense, But It’s Not Great, Either
Each of the above three objections are common fare for the Frequentist:
Frequentist approaches start from the objective/external perspective rather than the agent’s internal uncertainty. (For example, an “unbiased estimator” is one such that given a true value of the parameter, the estimator achieves that true value on average.) So they’re not starting from averaging over subjective possibilities. Instead, Frequentism averages over a sequence of experiments (to get a frequency!). This is like a time-average, since we have to put it in sequence. Hence, time-averages are natural to the way Frequentists think about probability.
Even given direct access to objective truth, frequentist probabilities are still under-defined because of the reference class problem—what infinite sequence of experiments do you conceive of your experiment as part of? So the ambiguity about how to turn your situation into a sequence is natural.
And, again, once you select a sequence, there’s no guarantee that a limit exists. Frequentism has to solve this by postulating that limits exist for the kinds of reference classes we want to talk about.
So, actually, this way of thinking about decision theory is quite appealing from a Frequentist perspective.
Remember all that stuff about how a Bayesian money-maximizer would behave? That was crazy. The Bayesian money-maximizer would, in fact, lose all its money rather quickly (with very high probability). Its in-expectation returns come from increasingly improbable universes. Would natural selection design agents like that, if it could help it?
So, the fact that Bayesian decision theory cares only about in-expectation, rather than caring about typical outcomes, does seem like a problem—for some utility functions, at least, it results in behavior which humans intuitively feel is insane and ineffective. (And philosophy is all about trying to take human intuitions seriously, even if sometimes we do decide human intuition is wrong and better discarded).
So I applaud Ole Peters’ heroic attempt to fix this problem.
However, if you dig into Peters, there’s one critical place where his method is sadly arbitrary. You say:
Compounding is multiplicative, so it becomes “natural” (in some sense) to transform everything by taking logs.
Peters makes much of this idea of what’s “natural”. He talks about additive problems vs multiplicative problems, as well as the more general case (when neither additive/multiplicative work).
However, as far as I can tell, this boils down to creatively choosing a function which makes the math work out.
A resolution of the St Petersburg paradox is presented. In contrast to the standard resolution, utility is not required. Instead, the time-average performance of the lottery is computed. The final result can be phrased mathematically identically to Daniel Bernoulli’s resolution, which uses logarithmic utility, but is derived using a conceptually different argument. The advantage of the time resolution is the elimination of arbitrary utility functions.
Daniel Bernoulli “solved” the St Petersburg paradox by suggesting that utility is log in money, which makes the expectations work out. (This is not very satisfying, because we can always transform the monetary payouts by an exponential function, making things problematic again! D Bernoulli could respond “take the logarithm twice” in such a case, but by this point, it’s clear he’s just making stuff up, so we ignore him.)
Ole Peters offers the same solution, but, he says, with an improved justification: he promises to eliminate the arbitrary choice of utility functions.
He does this by arbitrarily choosing to examine the time-average behavior in terms of growth rate, which is defined multiplicatively, rather than additively.
“Growth rate” sounds like an innocuous, objective choice of what to maximize. But really, it’s sneaking in exactly the same arbitrary decision which D Bernoulli made. Taking a ratio, rather than a difference, is just a sneaky way to take the logarithm.
So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You’re leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.
Again, I think Ole Peters is pretty cool, and there’s interesting stuff in this general direction, even if I find that the particular methodology he’s developed doesn’t justify anything and instead engages in obfuscated question-begging of existing ideas (like Kelly, and like D Bernoulli’s proposed solution to St Petersburg).
Also, I could be misunderstanding something about Peters!
First and foremost: you and I have disagreed in the past on wanting descriptive vs prescriptive roles for probability/decision theory. In this case, I’d paraphrase the two perspectives as:
Prescriptive-pure-Bayes: as long as we’re maximizing an expected utility, we’re “good”, and it doesn’t really matter which utility. But many utilities will throw away all their money with probability close to 1, so Kelly isn’t prescriptively correct.
Descriptive-pure-Bayes: as long as we’re not throwing away money for nothing, we’re implicitly maximizing an expected utility. Maximizing typical (i.e. modal/median/etc) long-run wealth is presumably incompatible with throwing away money for nothing, so presumably a typical-long-run-wealth-maximizer is also an expected utility maximizer. (Note that this is nontrivial, since “typical long-run wealth” is not itself an expectation.) Sure enough, the Kelly rule has the form of expected utility maximization, and the implicit utility is logarithmic.
In particular, this is relevant to:
Remember all that stuff about how a Bayesian money-maximizer would behave? That was crazy. The Bayesian money-maximizer would, in fact, lose all its money rather quickly (with very high probability). Its in-expectation returns come from increasingly improbable universes. Would natural selection design agents like that, if it could help it?
“Does Bayesian utility maximization imply good performance?” is mainly relevant to the prescriptive view. “Does good performance imply Bayesian utility maximization?” is the key descriptive question. In this case, the latter would say that natural selection would indeed design Bayesian agents, but that does not mean that every Bayesian agent is positively selected—just that those designs which are positively selected are (approximately) Bayesian agents.
“Natural” → Symmetry
Peters makes much of this idea of what’s “natural”. He talks about additive problems vs multiplicative problems, as well as the more general case (when neither additive/multiplicative work).
However, as far as I can tell, this boils down to creatively choosing a function which makes the math work out.
I haven’t read Peters, but the argument I see in this space is about symmetry/exchangeability (similar to some of de Finetti’s stuff). Choosing a function which makes reward/utility additive across timesteps is not arbitrary; it’s making utility have the same symmetry as our beliefs (in situations where each timestep’s variables are independent, or at least exchangeable).
In general, there’s a whole cluster of theorems which say, roughly, if a function f(x1,...xn) is invariant under re-ordering its inputs, then it can be written as f(x1,...xn)=g(∑ih(xi)) for some g, h. This includes, for instance, characterizing all finite abelian groups as modular addition, or de Finetti’s Theorem, or expressing symmetric polynomials in terms of power-sum polynomials. Addition is, in some sense, a “standard form” for symmetric functions.
Suppose we have a sequence of n bets. Our knowledge is symmetric under swapping the bets around, and our terminal goals don’t involve the bets themselves. So, our preferences should be symmetric under swapping the bets around. That implies we can write it in the “standard form”—i.e. we can express our preferences as a function of a sum of some summary data about each bet.
I’m not seeing the full argument yet, but it feels like there’s something in roughly that space. Presumably it would derive a de Finetti-style exchangeability-based version of Bayesian reasoning.
I agree with your prescriptive vs descriptive thing, and agree that I was basically making that mistake.
I think the correct position here is something like: expected utility maximization; and also, utility in “these cases” is going to be close to logarithmic. (IE, if you evolve trading strategies in something resembling Kelly’s conditions, you’ll end up with something resembling Kelly agents. And there’s probably some generalization of this which plausibly abstracts aspects of the human condition.)
Ole Peters is trying to re-found decision theory on the basis of this second layer alone. I think this is basically a good instinct:
It’s good to try to firm up this second layer, since just Kelly alone is way too special-case, and we’d like to understand the phenomenon in as much generality as possible.
It’s good to try and make a 1-layer system rather than a 2-layer one, to try and make our principles as unified as possible. The Kelly idea is consistent with our foundation of expectation maximization, sure, but if “realistic” agents systematically avoid some utility functions, that makes expectation maximization a worse descriptive theory. Perhaps there is a better one.
This is similar to the way Solomonoff is a two-layer system: there’s a lower layer of probability theory, and then on top of that, there’s the layer of algorithmic information theory, which tells us to prefer particular priors. In hindsight this should have been “suspicious”; logical induction merges those two layers together, giving a unified framework which gives us (approximately) probability theory and also (approximately) algorithmic information theory, tying them together with a unified bounded-loss notion. (And also implies many new principles which neither probability theory nor algorithmic information theory gave us.)
So although I agree that your descriptive lens is the better one, I think that lens has similar implications.
As for your comments about symmetry—I must admit that I tend to find symmetry arguments to be weak. Maybe you can come up with something cool, but I would tend to predict it’ll be superseded by less symmetry-based alternatives. For one thing, it tends to be a two-layered thing, with symmetry constraints added on top of more basic ideas.
This was fascinating. Thanks for taking the time to write it. I agree with the vast majority of what you wrote, although I don’t think it actually applies to what I was trying to do in this post. I don’t disagree that a full-Bayesian finds this whole thing a bit trivial, but I don’t believe people are fully Bayesian (to the extent they know their utility function) and therefore I think coming up with heuristics is valuable to help them think about things.
So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You’re leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.
I don’t really think of it as much as an “argument”. I’m not trying to “prove” Kelly criterion. I’m trying to help people get some intuition for where it might come from and some other reasons to consider it if they aren’t utility maximising.
It’s interesting to me that you brought up the exponential St Petersburg paradox, since MacLean, Thorpe, Ziemba claim that Kelly criterion can also handle it although I personally haven’t gone through the math.
Yeah, in retrospect it was a bit straw of me to argue the pure bayesian perspective like I did (I think I was just interested in the pure-bayes response, not thinking hard about what I was trying to argue there).
So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You’re leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.
I don’t really think of it as much as an “argument”. I’m not trying to “prove” Kelly criterion.
I’m surprised at that. Your post read to me as endorsing Peters’ argument. (Although you did emphasize that you were not trying to say that Kelly was the one true rule.)
Hm. I guess I should work harder on articulating what position I would argue for wrt Kelly. I basically think there exist good heuristic arguments for Kelly, but people often confuse them for more objective than they are (either in an unsophisticated way, like reading standard arguments for Kelly and thinking they’re stronger than they are, or in a sophisticated way, like Peters’ explicit attempt to rewrite the whole foundation of decision theory). Which leads me to reflexively snipe at people who appear to be arguing for Kelly, unless they clearly distinguish themselves from the wrong arguments.
I’m very interested in your potential post on corrections to Kelly.
Comment/question about St. Petersburg and utilities: given any utility function u which goes to infinity, it should be possible to construct a custom St. Petersburg lottery for that utility function, right? I.e. a lottery with infinite expectation but arbitrarily low probability of being in the green. If we want to model an agent as universally rejecting such lotteries, it follows that utility cannot diverge, and thus must asymptotically approach some supremum (also requiring the natural condition that u is strictly monotone). Has this shape of utility function been seriously proposed in economics? Does it have a name?
I wondered if someone would bring this up! Yes, some people take this as a strong argument that utilities simply have to be bounded in order to be well-defined at all. AFAIK this is just called a “bounded utility function”. Many of the standard representation theorems also imply that utility is bounded; this simply isn’t mentioned as often as other properties of utility.
However, I am not one of the people who takes this view. It’s perfectly consistent to define preferences which must be treated as unbounded utility. In doing so, we also have to specify our preferences about infinite lotteries. The divergent sum doesn’t make this impossible; instead, what it does is allow us to take many different possible values (within some consistency constraints). So for example, a lottery with 50% probability of +1 util, 25% of −3, 1/8th chance of +9, 1⁄16 −27, etc can be assigned any expected value whatsoever. Its evaluation is subjective! So in this framework, preferences encode more information than just a utility for each possible world; we can’t calculate all the expected values just from that. We also have to know how the agent subjectively values infinite lotteries. But this is fine!
How that works is a bit technical and I don’t want to get into it atm. From a mathematical perspective, it’s pretty “standard/obvious” stuff (for a graduate-level mathematician, anyway). But I don’t think many professional philosophers have picked up on this? The literature on Infinite Ethics seems mostly ignorant of it?
Sorry to comment on a two year old post but: I’m actually quite intrigued by the arbitrary valuation of infinite lotteries thing here. Can you explain this a bit further or point me to somewhere I can learn about it? (P-adic numbers come to mind, but probably have nothing to do with it.)
OK, here is my fuller response.
First of all, I want to state that Ergodicity Economics is great and I’m glad that it exists—like I said in my other comment, it’s a true frequentist alternative to Bayesian decision theory, and it does some cool things, and I would like to see it developed further.
That said, I want to defend two claims:
From a strict Bayesian perspective, it’s all nonsense, so those arguments in favor of Kelly are at best heuristic. A Bayesian should be much more interested in the question of whether their utility is log in money. (Note: I am not a strict bayesian, although I think strict Bayes is a pretty good starting point.)
From a broader perspective, I think Peters’ direction is interesting, but his current methodology is not very satisfying, and doesn’t support his strong claims that EG Kelly naturally pops out when you take a time-averaging rather than ensemble-averaging perspective.
“It’s All Nonsense”
Let’s deal with the strict Bayesian perspective first.
You start your argument by discussing the “variance drag” phenomenon. You point out that the expected monetary value of an investment can be constant or even increasing, but it may be that due to variance drag, it typically (with high probability) goes down over time.
You go on to assert that “people need to account for this” and “adjust their strategy” because “variance is bad”.
To a strict Bayesian, this is a non-sequitur. If it be the case that a strict Bayesian is a money-maximizer (IE, has utility linear in money), they’ll see no problem with variance drag. You still put your money where it gets the highest expected value.
Furthermore, repeated bets don’t make any material difference to the money-maximizer. Suppose the optimal one-step investment strategy has a tare of return R. Then the best two-step strategy is to use that strategy twice to get an expected rate of return R^2. This is because the best way they can set themselves up to succeed at stage 2 (in expectation!) is to get as much money as they can at stage 1 (in expectation!). This reasoning applies no matter how many stages we string together. So to the money-maximizer, your argument about repeated bets is totally wrong.
Where a strict Bayesian would start to agree with you is if they don’t have utility linear in money, but rather, have diminishing returns.
In that case, the best strategy for single-step investing can differ from the best strategy for multi-step investing.
Say the utility function is u(x). Treat an investment strategy as a stochastic function s(x). We assume the random variable c⋅s(x) behaves the same as the random variable s(c⋅x); IE, returns are proportional to investment. So, actually, we can treat a strategy as a random variable S which gets multiplied by our money; s(x)=S⋅x.
In a one-step scenario, the Bayesian wants to maximize E[u(s(x))] where x is your starting money. In a two-step scenario, the Bayesian wants to maximize E[u(s2(s1(x)))]. And so on. The reason the strategy was so simple when u(x)=x was that this allowed us to push the expectation inwards; E[u(s2(s1(x)))]=E[s2(s1(x))] =E[S2⋅S1⋅x] =E[S1]⋅E[S2]⋅x (the last step holds because we assume the random variables are independent). So in that case, we could just choose the best individual strategy and apply it at each time-step.
In general, however, nonlinear u stops us from pushing the expectations inward. So multi-step matters.
A many-step problem looks like E[u(S1⋅S2⋅S3⋅...⋅Sn⋅x)]. The product of many independent random variables will closely approximate a log-normal distribution, by the multiplicative central limit theorem. I think this somehow implies that if the Bayesian had to select the same strategy to be used on all days, the Bayesian would be pretty happy for it to be a log-wealth maximizing strategy, but I’m not connecting the dots on the argument right now.
So anyway, under some assumptions, I think a Bayesian will behave increasingly like a log-wealth maximizer as the number of steps goes up.
But this is because the Bayesian had diminishing returns in the first place. So it’s still ultimately about the utility function.
Most attempts to justify Kelly by repeated-bet phenomena overtly or implicitly rely on an assumption that we care about typical outcomes. A strict Bayesian will always stop you there. “Hold on now. I care about average outcomes!”
(And Ergodicity Economics says: so much the worse for Bayes!)
Bayesians may also raise other concerns with Peters-style reasoning:
Ensemble averaging is the natural response to decision-making under uncertainty; you’re averaging over different possibilities. When you try to time-average to get rid of your uncertainty, you have to ask “time average what?”—you don’t know what specific situation you’re in.
In general, the question of how to turn your current situation into a repeated sequence for the purpose of time-averaging analysis seems under-determined (even if you are certain about your present situation). Surely Peters doesn’t want us to use actual time in the analysis; in actual time, you end up dead and lose all your money, so the time-average analysis is trivial. So Peters has to invent an imagined time dimension, in which the game goes on forever. In contrast, Bayesians can deal with time as it actually is, accounting for the actual number of iterations of the game, utility of money at death, etc.
Even if you settle on a way to turn the situation into an iterated sequence, the necessary limit does not necessarily exist. This is also true of the possibility-average, of course (the St Petersburg Paradox being a classic example); but it seems easier to get failure in the time-avarage case, because you just need non-convergence; ie, you don’t need any unbounded stuff to happen.
It’s Not Nonsense, But It’s Not Great, Either
Each of the above three objections are common fare for the Frequentist:
Frequentist approaches start from the objective/external perspective rather than the agent’s internal uncertainty. (For example, an “unbiased estimator” is one such that given a true value of the parameter, the estimator achieves that true value on average.) So they’re not starting from averaging over subjective possibilities. Instead, Frequentism averages over a sequence of experiments (to get a frequency!). This is like a time-average, since we have to put it in sequence. Hence, time-averages are natural to the way Frequentists think about probability.
Even given direct access to objective truth, frequentist probabilities are still under-defined because of the reference class problem—what infinite sequence of experiments do you conceive of your experiment as part of? So the ambiguity about how to turn your situation into a sequence is natural.
And, again, once you select a sequence, there’s no guarantee that a limit exists. Frequentism has to solve this by postulating that limits exist for the kinds of reference classes we want to talk about.
So, actually, this way of thinking about decision theory is quite appealing from a Frequentist perspective.
Remember all that stuff about how a Bayesian money-maximizer would behave? That was crazy. The Bayesian money-maximizer would, in fact, lose all its money rather quickly (with very high probability). Its in-expectation returns come from increasingly improbable universes. Would natural selection design agents like that, if it could help it?
So, the fact that Bayesian decision theory cares only about in-expectation, rather than caring about typical outcomes, does seem like a problem—for some utility functions, at least, it results in behavior which humans intuitively feel is insane and ineffective. (And philosophy is all about trying to take human intuitions seriously, even if sometimes we do decide human intuition is wrong and better discarded).
So I applaud Ole Peters’ heroic attempt to fix this problem.
However, if you dig into Peters, there’s one critical place where his method is sadly arbitrary. You say:
Peters makes much of this idea of what’s “natural”. He talks about additive problems vs multiplicative problems, as well as the more general case (when neither additive/multiplicative work).
However, as far as I can tell, this boils down to creatively choosing a function which makes the math work out.
I find this inexcusable, as-is.
For example, Peters makes much of the St Petersburg lottery:
Daniel Bernoulli “solved” the St Petersburg paradox by suggesting that utility is log in money, which makes the expectations work out. (This is not very satisfying, because we can always transform the monetary payouts by an exponential function, making things problematic again! D Bernoulli could respond “take the logarithm twice” in such a case, but by this point, it’s clear he’s just making stuff up, so we ignore him.)
Ole Peters offers the same solution, but, he says, with an improved justification: he promises to eliminate the arbitrary choice of utility functions.
He does this by arbitrarily choosing to examine the time-average behavior in terms of growth rate, which is defined multiplicatively, rather than additively.
“Growth rate” sounds like an innocuous, objective choice of what to maximize. But really, it’s sneaking in exactly the same arbitrary decision which D Bernoulli made. Taking a ratio, rather than a difference, is just a sneaky way to take the logarithm.
So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You’re leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.
Again, I think Ole Peters is pretty cool, and there’s interesting stuff in this general direction, even if I find that the particular methodology he’s developed doesn’t justify anything and instead engages in obfuscated question-begging of existing ideas (like Kelly, and like D Bernoulli’s proposed solution to St Petersburg).
Also, I could be misunderstanding something about Peters!
I have some interesting disagreements with this.
Prescriptive vs Descriptive
First and foremost: you and I have disagreed in the past on wanting descriptive vs prescriptive roles for probability/decision theory. In this case, I’d paraphrase the two perspectives as:
Prescriptive-pure-Bayes: as long as we’re maximizing an expected utility, we’re “good”, and it doesn’t really matter which utility. But many utilities will throw away all their money with probability close to 1, so Kelly isn’t prescriptively correct.
Descriptive-pure-Bayes: as long as we’re not throwing away money for nothing, we’re implicitly maximizing an expected utility. Maximizing typical (i.e. modal/median/etc) long-run wealth is presumably incompatible with throwing away money for nothing, so presumably a typical-long-run-wealth-maximizer is also an expected utility maximizer. (Note that this is nontrivial, since “typical long-run wealth” is not itself an expectation.) Sure enough, the Kelly rule has the form of expected utility maximization, and the implicit utility is logarithmic.
In particular, this is relevant to:
“Does Bayesian utility maximization imply good performance?” is mainly relevant to the prescriptive view. “Does good performance imply Bayesian utility maximization?” is the key descriptive question. In this case, the latter would say that natural selection would indeed design Bayesian agents, but that does not mean that every Bayesian agent is positively selected—just that those designs which are positively selected are (approximately) Bayesian agents.
“Natural” → Symmetry
I haven’t read Peters, but the argument I see in this space is about symmetry/exchangeability (similar to some of de Finetti’s stuff). Choosing a function which makes reward/utility additive across timesteps is not arbitrary; it’s making utility have the same symmetry as our beliefs (in situations where each timestep’s variables are independent, or at least exchangeable).
In general, there’s a whole cluster of theorems which say, roughly, if a function f(x1,...xn) is invariant under re-ordering its inputs, then it can be written as f(x1,...xn)=g(∑ih(xi)) for some g, h. This includes, for instance, characterizing all finite abelian groups as modular addition, or de Finetti’s Theorem, or expressing symmetric polynomials in terms of power-sum polynomials. Addition is, in some sense, a “standard form” for symmetric functions.
Suppose we have a sequence of n bets. Our knowledge is symmetric under swapping the bets around, and our terminal goals don’t involve the bets themselves. So, our preferences should be symmetric under swapping the bets around. That implies we can write it in the “standard form”—i.e. we can express our preferences as a function of a sum of some summary data about each bet.
I’m not seeing the full argument yet, but it feels like there’s something in roughly that space. Presumably it would derive a de Finetti-style exchangeability-based version of Bayesian reasoning.
I agree with your prescriptive vs descriptive thing, and agree that I was basically making that mistake.
I think the correct position here is something like: expected utility maximization; and also, utility in “these cases” is going to be close to logarithmic. (IE, if you evolve trading strategies in something resembling Kelly’s conditions, you’ll end up with something resembling Kelly agents. And there’s probably some generalization of this which plausibly abstracts aspects of the human condition.)
But note how piecemeal and fragile this sounds. One layer is the relatively firm expectation-maximization layer. On top of this we add another layer (based on maximization of mode/median/quantile, so that we can ignore things not true with probability 1) which argues for some utility functions in particular.
Ole Peters is trying to re-found decision theory on the basis of this second layer alone. I think this is basically a good instinct:
It’s good to try to firm up this second layer, since just Kelly alone is way too special-case, and we’d like to understand the phenomenon in as much generality as possible.
It’s good to try and make a 1-layer system rather than a 2-layer one, to try and make our principles as unified as possible. The Kelly idea is consistent with our foundation of expectation maximization, sure, but if “realistic” agents systematically avoid some utility functions, that makes expectation maximization a worse descriptive theory. Perhaps there is a better one.
This is similar to the way Solomonoff is a two-layer system: there’s a lower layer of probability theory, and then on top of that, there’s the layer of algorithmic information theory, which tells us to prefer particular priors. In hindsight this should have been “suspicious”; logical induction merges those two layers together, giving a unified framework which gives us (approximately) probability theory and also (approximately) algorithmic information theory, tying them together with a unified bounded-loss notion. (And also implies many new principles which neither probability theory nor algorithmic information theory gave us.)
So although I agree that your descriptive lens is the better one, I think that lens has similar implications.
As for your comments about symmetry—I must admit that I tend to find symmetry arguments to be weak. Maybe you can come up with something cool, but I would tend to predict it’ll be superseded by less symmetry-based alternatives. For one thing, it tends to be a two-layered thing, with symmetry constraints added on top of more basic ideas.
This was fascinating. Thanks for taking the time to write it. I agree with the vast majority of what you wrote, although I don’t think it actually applies to what I was trying to do in this post. I don’t disagree that a full-Bayesian finds this whole thing a bit trivial, but I don’t believe people are fully Bayesian (to the extent they know their utility function) and therefore I think coming up with heuristics is valuable to help them think about things.
I don’t really think of it as much as an “argument”. I’m not trying to “prove” Kelly criterion. I’m trying to help people get some intuition for where it might come from and some other reasons to consider it if they aren’t utility maximising.
It’s interesting to me that you brought up the exponential St Petersburg paradox, since MacLean, Thorpe, Ziemba claim that Kelly criterion can also handle it although I personally haven’t gone through the math.
Yeah, in retrospect it was a bit straw of me to argue the pure bayesian perspective like I did (I think I was just interested in the pure-bayes response, not thinking hard about what I was trying to argue there).
I’m surprised at that. Your post read to me as endorsing Peters’ argument. (Although you did emphasize that you were not trying to say that Kelly was the one true rule.)
Hm. I guess I should work harder on articulating what position I would argue for wrt Kelly. I basically think there exist good heuristic arguments for Kelly, but people often confuse them for more objective than they are (either in an unsophisticated way, like reading standard arguments for Kelly and thinking they’re stronger than they are, or in a sophisticated way, like Peters’ explicit attempt to rewrite the whole foundation of decision theory). Which leads me to reflexively snipe at people who appear to be arguing for Kelly, unless they clearly distinguish themselves from the wrong arguments.
I’m very interested in your potential post on corrections to Kelly.
Comment/question about St. Petersburg and utilities: given any utility function u which goes to infinity, it should be possible to construct a custom St. Petersburg lottery for that utility function, right? I.e. a lottery with infinite expectation but arbitrarily low probability of being in the green. If we want to model an agent as universally rejecting such lotteries, it follows that utility cannot diverge, and thus must asymptotically approach some supremum (also requiring the natural condition that u is strictly monotone). Has this shape of utility function been seriously proposed in economics? Does it have a name?
I wondered if someone would bring this up! Yes, some people take this as a strong argument that utilities simply have to be bounded in order to be well-defined at all. AFAIK this is just called a “bounded utility function”. Many of the standard representation theorems also imply that utility is bounded; this simply isn’t mentioned as often as other properties of utility.
However, I am not one of the people who takes this view. It’s perfectly consistent to define preferences which must be treated as unbounded utility. In doing so, we also have to specify our preferences about infinite lotteries. The divergent sum doesn’t make this impossible; instead, what it does is allow us to take many different possible values (within some consistency constraints). So for example, a lottery with 50% probability of +1 util, 25% of −3, 1/8th chance of +9, 1⁄16 −27, etc can be assigned any expected value whatsoever. Its evaluation is subjective! So in this framework, preferences encode more information than just a utility for each possible world; we can’t calculate all the expected values just from that. We also have to know how the agent subjectively values infinite lotteries. But this is fine!
How that works is a bit technical and I don’t want to get into it atm. From a mathematical perspective, it’s pretty “standard/obvious” stuff (for a graduate-level mathematician, anyway). But I don’t think many professional philosophers have picked up on this? The literature on Infinite Ethics seems mostly ignorant of it?
Sorry to comment on a two year old post but: I’m actually quite intrigued by the arbitrary valuation of infinite lotteries thing here. Can you explain this a bit further or point me to somewhere I can learn about it? (P-adic numbers come to mind, but probably have nothing to do with it.)
Commenting on old posts is encouraged :)