$500 Bounty/​Contest: Explain Infra-Bayes In The Language Of Game Theory

[johnswentworth]

Here’s my current best guess at how Infra-Bayes works:

• We want to get worst-case guarantees for an agent using a Bayesian-like framework.

• So, let our agent be a Bayesian which models the environment as containing an adversary which chooses worst-case values for any of the things over which we want worst-case guarantees.

• That’s just a standard two-player zero-sum game between the agent and the adversary, so we can import all the nice intuitive stuff from game theory.

• … but instead of that, we’re going to express everything in the unnecessarily-abstract language of measure theory and convex sets, and rederive a bunch of game theory without mentioning that that’s what we’re doing.

This bounty is for someone to write an intuitively-accessible infrabayes explainer in game theoretic language, and explain how the game-theoretic concepts relate to the concepts in existing presentations of infra-bayes. In short: provide a translation.

Here’s a sample of the sort of thing I have in mind:

Conceptually, an infrabayesian agent is just an ordinary Bayesian game-theoretic agent, which models itself/​its environment as a standard two-player zero-sum game.

In the existing presentations of infra-bayes, the two-player game is only given implicitly. The agent’s strategy $\pi$ solves the problem:

${\text{max}}_{\pi }{\text{min}}_{e\in B}{E}_{\pi \cdot e}\left[U\right]$

In game-theoretic terms, the “max” represents the agent’s decision, while the “min” represents the adversary’s.

Much of the mathematical tractability stems from the fact that $B$ is a convex set of environments (i.e. functions from policy $\pi$ to probability distributions). In game-theoretic terms, the adversary’s choice of strategy determines which “environment” the agent faces, and the adversary can choose from any option in $B$. Convexity of $B$ follows from the adversary’s ability to use mixed strategies: because the adversary can take a randomized mix of any two strategies available to it, the adversary can make the agent face any convex combination of (policy → distribution) functions in $B$. Thus, $B$ is closed under convex combinations; it’s a convex set.

I’d like a writeup along roughly these conceptual lines which covers as much as possible of the major high-level definitions and results in infra-bayes to date. On the other hand, I give approximately-zero shits about all the measure theory; just state the relevant high-level results in game-theoretic language, say what they mean intuitively, maybe mention whether there’s some pre-existing standard game-theory theorem which can do the job or whether the infra-bayes version of the theorem is in fact the first proof of the game-theoretic equivalent, and move on.

Alternatively, insofar as core parts of infrabayes differ from a two-player zero-sum game, or the general path I’m pointing to doesn’t work, an explanation of how they differ and what the consequences are could also qualify for prize money.

Bounty/​Contest Operationalization

Most of the headache in administering this sort of bounty is the risk that some well-intended person will write something which is not at all what I want, expecting to get paid, and then I will either have to explain how/​why it’s not what I want (which takes a lot of work), or I have to just accept it. To mitigate that failure mode, I’ll run this as a contest: to submit, write up your explanation as a lesswrong post, then send me a message on lesswrong to make sure I’m aware of it. Deadline is end of April. I will distribute money among submissions based on my own highly-subjective judgement. If people write stuff up early, I might leave feedback on their posts, but no promises.

I will count the “sample” above as a submission in its own right—i.e. I will imagine that three-paragraph blurb were instead a three-paragraph post in its own right, and someone submitted it. That will provide a baseline for prizes to be paid out at all: if no submission adds value not already included in the three-paragraph blurb, then the three-paragraph blurb gets the prize money, i.e. I don’t pay anyone.

Note that the $500 prize is probably not enough to fully pay for the amount of effort which I expect will be involved in doing this well. Others are welcome to add to the prize pool; please leave a comment if you’d like to do so.

[leogao]

Adding $200 to the pool. Also, I endorse the existence of more bounties/​contests like this.

[Ramana Kumar]

I’ll add $500 to the pot.

[the gears to ascension]

adding $200 to the pool.

I wonder what other math distillations are worth funding.

[the gears to ascension]

Hmm, update: I’ve heard from someone who looked closely at infrabayes that it might not be that insightful anyway? I do not retract this bounty, but I am now expecting to be disappointed and find that infra-bayes basically simply boils down to game theory in general, rather than identifying new general shapes in the implications.

[Garrett Baker]

Adding $80 to the pool.

Edited from $50 to $80 after realizing market price I’m usually willing to pay for excellent math distillations.

[johnswentworth]

Aside: Vanessa mentioned in person at one point that the game-theoretic perspective on infra-bayes indeed basically works, and she has a result somewhere about the equivalence. So that might prove useful, if you’re looking to claim this prize.

[Shmi]

Are there any specific results of infra-Bayesiansm that you would like to see derived using the worst case game-theoretical approach?