This post is still endorsed, it still feels like a continually fruitful line of research. A notable aspect of it is that, as time goes on, I keep finding more connections and crisper ways of viewing things which means that for many of the further linked posts about inframeasure theory, I think I could explain them from scratch better than the existing work does. One striking example is that the “Nirvana trick” stated in this intro (to encode nonstandard decision-theory problems), has transitioned from “weird hack that happens to work” to “pops straight out when you make all the math as elegant as possible”. Accordingly, I’m working on a “living textbook” (like a textbook, but continually being updated with whatever cool new things we find) where I try to explain everything from scratch in the crispest way possible, to quickly catch up on the frontier of what we’re working on. That’s my current project.
I still do think that this is a large and tractable vein of research to work on, and the conclusion hasn’t changed much.
I’m feeling very excited about this agenda. Is there currently a publicly-viewable version of the living textbook? Or any more formal writeup which I can include in my curriculum? (If not I’ll include this post, but I expect many people would appreciate a more polished writeup.)
If you’re looking for curriculum materials, I believe that the most useful reference would probably be my “Infra-exercises”, a sequence of posts containing all the math exercises you need to reinvent a good chunk of the theory yourself. Basically, it’s the textbook’s exercise section, and working through interesting math problems and proofs on one’s own has a much better learning feedback loop and retention of material than slogging through the old posts. The exercises are short on motivation and philosophy compared to the posts, though, much like how a functional analysis textbook takes for granted that you want to learn functional analysis and doesn’t bother motivating it.
The primary problem is that the exercises aren’t particularly calibrated in terms of difficulty, and in order for me to get useful feedback, someone has to actually work through all of them, so feedback has been a bit sparse. So I’m stuck in a situation where I keep having to link everyone to the infra-exercises over and over and it’d be really good to just get them out and publicly available, but if they’re as important as I think, then the best move is something like “release them one at a time and have a bunch of people work through them as a group” like the fixpoint exercises, instead of “just dump them all as public documents”.
I’ll ask around about speeding up the public—ation of the exercises and see what can be done there.
I’d strongly endorse linking this introduction even if the exercises are linked as well, because this introduction serves as the table of contents to all the other applicable posts.
I’m confused about the Nirvana trick then. (Maybe here’s not the best place, but oh well...) Shouldn’t it break the instant you do anything with your Knightian uncertainty other than taking the worst-case?
Well, taking worst-case uncertainty is what infradistributions do. Did you have anything in mind that can be done with Knightian uncertainty besides taking the worst-case (or best-case)?
And if you were dealing with best-case uncertainty instead, then the corresponding analogue would be assuming that you go to hell if you’re mispredicted (and then, since best-case things happen to you, the predictor must accurately predict you).
What if you assumed the stuff you had the hypothesis about was independent of the stuff you have Knightian uncertainty about (until proven otherwise)?
E.g. if you’re making hypotheses about a multi-armed bandit and the world also contains a meteor that might smash through your ceiling and kill you at any time, you might want to just say “okay, ignore the meteor, pretend my utility has a term for gambling wins that doesn’t depend on the meteor at all.”
The reason I want to consider stuff more like this is because I don’t like having to evaluate my utility function over all possibilities to do either an argmax or an argmin—I want to be lazy.
The weird thing about this is now whether this counts as argmax or argmin (or something else) depends on what my utility function looks like when I do include the meteor. If getting hit by the meteor only makes things worse (though potentially the meteor can still depend on which arm of of the bandit I pull!) then ignoring it is like being optimistic. If it only makes things better (like maybe the world I’m ignoring isn’t a meteor, it’s a big space full of other games I could be playing) then ignoring it is like being pessimistic.
Something analogous to what you are suggesting occurs. Specifically, let’s say you assign 95% probability to the bandit game behaving as normal, and 5% to “oh no, anything could happen, including the meteor”. As it turns out, this behaves similarly to the ordinary bandit game being guaranteed, as the “maybe meteor” hypothesis assigns all your possible actions a score of “you’re dead” so it drops out of consideration.
The important aspect which a hypothesis needs, in order for you to ignore it, is that no matter what you do you get the same outcome, whether it be good or bad. A “meteor of bliss hits the earth and everything is awesome forever” hypothesis would also drop out of consideration because it doesn’t really matter what you do in that scenario.
To be a wee bit more mathy, probabilistic mix of inframeasures works like this. We’ve got a probability distribution ζ∈ΔN, and a bunch of hypotheses ψi∈□X, things that take functions as input, and return expectation values. So, your prior, your probabilistic mixture of hypotheses according to your probability distribution, would be the function
f↦∑i∈Nζ(i)⋅ψi(f)
It gets very slightly more complicated when you’re dealing with environments, instead of static probability distributions, but it’s basically the same thing. And so, if you vary your actions/vary your choice of function f, and one of the hypotheses ψi is assigning all these functions/choices of actions the same expectation value, then it can be ignored completely when you’re trying to figure out the best function/choice of actions to plug in.
So, hypotheses that are like “you’re doomed no matter what you do” drop out of consideration, an infra-Bayes agent will always focus on the remaining hypotheses that say that what it does matters.
The meteor doesn’t have to really flatten things out, there might be some actions that we think remain valuable (e.g. hedonism, saying tearful goodbyes).
And so if we have Knightian uncertainty about the meteor, maximin (as in Vanessa’s link) means we’ll spend a lot of time on tearful goodbyes.
Said actions or lack thereof cause a fairly low utility differential compared to the actions in other, non-doomy hypotheses. Also I want to draw a critical distinction between “full knightian uncertainty over meteor presence or absence”, where your analysis is correct, and “ordinary probabilistic uncertainty between a high-knightian-uncertainty hypotheses, and a low-knightian uncertainty one that says the meteor almost certainly won’t happen” (where the meteor hypothesis will be ignored unless there’s a meteor-inspired modification to what you do that’s also very cheap in the “ordinary uncertainty” world, like calling your parents, because the meteor hypothesis is suppressed in decision-making by the low expected utility differentials, and we’re maximin-ing expected utility)
This post is still endorsed, it still feels like a continually fruitful line of research. A notable aspect of it is that, as time goes on, I keep finding more connections and crisper ways of viewing things which means that for many of the further linked posts about inframeasure theory, I think I could explain them from scratch better than the existing work does. One striking example is that the “Nirvana trick” stated in this intro (to encode nonstandard decision-theory problems), has transitioned from “weird hack that happens to work” to “pops straight out when you make all the math as elegant as possible”. Accordingly, I’m working on a “living textbook” (like a textbook, but continually being updated with whatever cool new things we find) where I try to explain everything from scratch in the crispest way possible, to quickly catch up on the frontier of what we’re working on. That’s my current project.
I still do think that this is a large and tractable vein of research to work on, and the conclusion hasn’t changed much.
I’m feeling very excited about this agenda. Is there currently a publicly-viewable version of the living textbook? Or any more formal writeup which I can include in my curriculum? (If not I’ll include this post, but I expect many people would appreciate a more polished writeup.)
If you’re looking for curriculum materials, I believe that the most useful reference would probably be my “Infra-exercises”, a sequence of posts containing all the math exercises you need to reinvent a good chunk of the theory yourself. Basically, it’s the textbook’s exercise section, and working through interesting math problems and proofs on one’s own has a much better learning feedback loop and retention of material than slogging through the old posts. The exercises are short on motivation and philosophy compared to the posts, though, much like how a functional analysis textbook takes for granted that you want to learn functional analysis and doesn’t bother motivating it.
The primary problem is that the exercises aren’t particularly calibrated in terms of difficulty, and in order for me to get useful feedback, someone has to actually work through all of them, so feedback has been a bit sparse. So I’m stuck in a situation where I keep having to link everyone to the infra-exercises over and over and it’d be really good to just get them out and publicly available, but if they’re as important as I think, then the best move is something like “release them one at a time and have a bunch of people work through them as a group” like the fixpoint exercises, instead of “just dump them all as public documents”.
I’ll ask around about speeding up the public—ation of the exercises and see what can be done there.
I’d strongly endorse linking this introduction even if the exercises are linked as well, because this introduction serves as the table of contents to all the other applicable posts.
I’m confused about the Nirvana trick then. (Maybe here’s not the best place, but oh well...) Shouldn’t it break the instant you do anything with your Knightian uncertainty other than taking the worst-case?
Notice that some non-worst-case decision rules are reducible to the worst-case decision rule.
Well, taking worst-case uncertainty is what infradistributions do. Did you have anything in mind that can be done with Knightian uncertainty besides taking the worst-case (or best-case)?
And if you were dealing with best-case uncertainty instead, then the corresponding analogue would be assuming that you go to hell if you’re mispredicted (and then, since best-case things happen to you, the predictor must accurately predict you).
What if you assumed the stuff you had the hypothesis about was independent of the stuff you have Knightian uncertainty about (until proven otherwise)?
E.g. if you’re making hypotheses about a multi-armed bandit and the world also contains a meteor that might smash through your ceiling and kill you at any time, you might want to just say “okay, ignore the meteor, pretend my utility has a term for gambling wins that doesn’t depend on the meteor at all.”
The reason I want to consider stuff more like this is because I don’t like having to evaluate my utility function over all possibilities to do either an argmax or an argmin—I want to be lazy.
The weird thing about this is now whether this counts as argmax or argmin (or something else) depends on what my utility function looks like when I do include the meteor. If getting hit by the meteor only makes things worse (though potentially the meteor can still depend on which arm of of the bandit I pull!) then ignoring it is like being optimistic. If it only makes things better (like maybe the world I’m ignoring isn’t a meteor, it’s a big space full of other games I could be playing) then ignoring it is like being pessimistic.
Something analogous to what you are suggesting occurs. Specifically, let’s say you assign 95% probability to the bandit game behaving as normal, and 5% to “oh no, anything could happen, including the meteor”. As it turns out, this behaves similarly to the ordinary bandit game being guaranteed, as the “maybe meteor” hypothesis assigns all your possible actions a score of “you’re dead” so it drops out of consideration.
The important aspect which a hypothesis needs, in order for you to ignore it, is that no matter what you do you get the same outcome, whether it be good or bad. A “meteor of bliss hits the earth and everything is awesome forever” hypothesis would also drop out of consideration because it doesn’t really matter what you do in that scenario.
To be a wee bit more mathy, probabilistic mix of inframeasures works like this. We’ve got a probability distribution ζ∈ΔN, and a bunch of hypotheses ψi∈□X, things that take functions as input, and return expectation values. So, your prior, your probabilistic mixture of hypotheses according to your probability distribution, would be the function
f↦∑i∈Nζ(i)⋅ψi(f)
It gets very slightly more complicated when you’re dealing with environments, instead of static probability distributions, but it’s basically the same thing. And so, if you vary your actions/vary your choice of function f, and one of the hypotheses ψi is assigning all these functions/choices of actions the same expectation value, then it can be ignored completely when you’re trying to figure out the best function/choice of actions to plug in.
So, hypotheses that are like “you’re doomed no matter what you do” drop out of consideration, an infra-Bayes agent will always focus on the remaining hypotheses that say that what it does matters.
The meteor doesn’t have to really flatten things out, there might be some actions that we think remain valuable (e.g. hedonism, saying tearful goodbyes).
And so if we have Knightian uncertainty about the meteor, maximin (as in Vanessa’s link) means we’ll spend a lot of time on tearful goodbyes.
Said actions or lack thereof cause a fairly low utility differential compared to the actions in other, non-doomy hypotheses. Also I want to draw a critical distinction between “full knightian uncertainty over meteor presence or absence”, where your analysis is correct, and “ordinary probabilistic uncertainty between a high-knightian-uncertainty hypotheses, and a low-knightian uncertainty one that says the meteor almost certainly won’t happen” (where the meteor hypothesis will be ignored unless there’s a meteor-inspired modification to what you do that’s also very cheap in the “ordinary uncertainty” world, like calling your parents, because the meteor hypothesis is suppressed in decision-making by the low expected utility differentials, and we’re maximin-ing expected utility)