Ukraine #4: Prediction Market Movement Modeling

Link post

I previously looked at various prediction markets related to Russia’s invasion of Ukraine. Scott Alexander provides an update on some prices, notices he is confused in various ways by the motions of various prices.

It seems easy enough to mostly explain any given single market and its movements. The real challenge is to combine all the markets and understand what story they are telling you together. This includes the history of their price movements as part of that story.

As usual, the Metaculus predictions cover more ground and go where others fear to tread, while any real-money markets are better where you can get them.

I am going to focus on the markets related to who is likely to win, as they tell the most interesting story. That story includes the lack of a quick Russian victory.

Will Kyiv fall to Russian forces by April 2022?

Note that the real money version of this is trading lower at 9%.

I have consistently thought this market was too high, and I still think that. The only way Kyiv falls in the next two weeks is a general surrender, that seems vanishingly unlikely. There is no way to take the city by force within two weeks given they have it very well defended – they claim to have 80,000 men, which makes me wonder why they even need that many. Naively seems to me like the second half of that only makes supplies a bigger issue and you’d rather use them elsewhere.

This is a steady climb downwards, and Scott wonders about whether it is a predictable and thus false pattern.

To some extent it is the automatic updating that Kyiv did not fall and does not appear about to fall. Each day of this makes it more likely Kyiv can hold out longer, reducing chances of a fall each day going forward, so things should reduce faster than the proportion of time expired. There’s also what looks like a clear shift away from trying that hard to take Kyiv quickly, and to focus on encircling Ukraine’s forces in the Donbass, which seems more realistic as a goal, and there isn’t time here to refocus.

What about longer-term versions of the same question?

Will Russia control Kyiv on June 1, 2022?

Notice that this started out not much higher than the first prediction. On March 3, Metaculus thought there was a 60% chance Kyiv would fall in March, and this question first settles in around 72%.

One could argue that the Russians might take the city and then leave without occupying it before June 1, which muddies the waters a bit, but that does not seem all that likely. If you ignore that possibility entirely you get a 60% chance to take the city in March but only a 30% chance to take it after that. Now we have 14% in March and 25% after that, reflecting only a moderately worse conditional chance than existed before – it’s saying that Russia didn’t underperform that much once we know they did not take Kyiv. If anything that seems like too small an adjustment.

If you think there’s a substantial chance Russia takes the city but then leaves it before June, then things get even weirder.

We can also look at whether Kyiv falls before 2023?

A lot less predictions here, but this rules out there being much of a chance of ‘it falls but is not occupied on June 1.’ Even if you discount that to zero, the conditional chance of Russia taking Kyiv in the later seven months of 2022, if it hadn’t taken it already, is put at only 15%. If Russia hasn’t taken Kyiv by then, it means they have severe problems, and the most likely scenarios are a stalemate, a negotiated settlement or a Ukrainian victory. A Russian win this late is weird, requiring something like ‘Russia mobilizes for real and therefore wins.’

Another way of looking at this is, we started off with expectation that Ukraine would collapse, but things would look better if they didn’t.

There was a 50% chance (according to this market) that Kyiv falls quickly.
Given that this did not happen, there was about a 40% chance they’d either take it or be in position to take it by mid-March.
Given that did not happen, there is a 10% chance they’ll take it by April.
Given that does not happen, there is a 25% chance they’ll take it by June.
Given that does not happen, there’s a 15% chance they’ll take it in 2022.
Presumably, most of the time they don’t take Kyiv in 2022, they never do.

Given that step three is only two weeks and not even a ‘full’ two weeks, that seems broadly consistent with a steady falling off of Russian chances over time.

So this story makes sense if you think that Russia is performing about as expected given it hasn’t taken Kyiv.

The question then becomes, are our impressions of Russia’s ability to win about what we would have expected at this point given Kyiv stands, or are they better or worse?

My sense is that they are clearly worse. There are scenarios where Kyiv holds but Russia is clearly winning elsewhere. Those didn’t happen. There are scenarios where Russia is keeping a lid on losses and/or bringing in a lot more troops. Those did not happen. Meanwhile they’re being hit very hard by sanctions and aid.

The counterargument is that in many similar scenarios, you might have expected Russia to fold, or to have suffered outright collapse, or perhaps even a full intervention. Or for there to be a negotiated settlement. Those also didn’t happen, Russian resolve to continue seems strong.

Thus, the story is that Russia is doing unexpectedly badly even for worlds where they don’t win quickly, but that is cancelled out by there being a much higher chance the war does not soon end. Do I buy that? No, not fully, but it’s a story.

Let’s see how other markets fit into this.

Will at least three of the big six cities (Kyiv, Lviv, Odesa, Mariupol, Kherson, and Kharkiv) fall by June?

This has not much budged. Lviv is not going to fall without the others falling. Kyiv’s chances of falling have gone down quite a lot and is now a relatively tough nut. So this is saying that three of the other four will fall.

And look, we also have a market on Kharkiv.

Remarkably similar, so much so that it’s plausible these are mostly the same market. Yes, they move slightly differently, but that could easily be mostly random given the dynamics.

One way for this to be true is if Russia gets Kharkiv it always gets Kherson and Mariupol as well, whereas if Russia doesn’t get Kharkiv it also doesn’t get Lviv, Kyiv or Odesa. But the correlation between Odesa and Kharkiv does not seem so high, so this implies Odesa is very hard to take, alas there is no direct market there, or even that Kharkiv could fall without two other cities falling, which is indeed necessary for Kharkiv to trade higher than the aggregate.

So it seems like the obvious arbitrage play is sell Kharkiv and buy the bundle, as the Kharkiv market was supposed to fall lower but hasn’t been updated enough yet. And presumably the value is in selling Kharkiv.

Which seems very right to me. Back at the start of this market, Kharkiv was 80% to fall, now it’s 72%? I can’t square that with what’s happened with Kyiv. If Kyiv falls Kharkiv almost certainly falls, and this therefore has not adjusted enough, especially with the Kyiv market having so many more eyes on it.

We do have this from Insight Markets.

Oddly, we do have a market on Russia controlling Vinnystia, which has gone from about 40% on March 3 down to 24% now, and also Russia’s success has been concentrated in the South, which makes the Kharkiv market look even stranger.

Next up, the market on Zelenskyy being president of Ukraine on April 22.

This is a clear example of a time progression. As of this graph there are only seven days left and very little chance Kyiv will fall in quickly. Nor is he going to give up the job on his own. This is an almost assassination market now. The only side you can really bet without insider information is on him living.

Here’s an interesting one, it’s new with almost no volume which seems to all be on no.

This is very close to a proxy on whether or not there is a meaningful peace agreement. There can be a cease fire without it, but peace? Putin agreeing to formally return Crimea to Ukraine would, in his mind and likely in reality, soon be the end for him. He’s not doing it.

So this will be one to watch, and I’d like to see higher liquidity. It currently has ~$15k.

Then of course there’s the fate of Putin (Metaculus / Polymarket / Insight).

Note that these do not quite line up, the expiration dates are end of year for Polymarket and February 1 for Metaculus.

The line at Polymarket and at Metaculus looks like a slight upward drift but largely a random walk. Does this make sense?

Consider Putin’s fate as a function of the fate of Kyiv or the war in general.

If Putin wins, in the sense that he gets to impose peace terms or outright occupy Ukraine, the chance of Putin remaining President of Russia seems very high. He could still die of natural causes as he does not look well, but the chances of a coup or other loss of power mostly dry up.

If Putin is removed from power, the war presumably ends without further Russian gains. So any taking of Kyiv or other progress almost always happens first, and means he survived that long.

At the start of this market, chances of taking Kyiv were at least 70%, and his chances of holding onto power were about where they are now, 80%. I interpret this as something like:

Very high chance of holding onto power if he takes Kyiv. Let’s say 95%.
Therefore, chance of holding onto power if not taking Kyiv is 45%.

That seems reasonable. But notice what happens when he’s only 45% to take Kyiv by the end of the year – his conditional probability of survival is now only 68%.

On top of that, we’ve had unexpectedly harsh economic fallout in Russia, and unexpectedly poor military performance. Neither of those is going to help. So what’s the counterweight that is keeping his chances static? Or is this a foolish market?

Another hint is that Putin’s approval rating, conditional on his being in power, hasn’t much changed. Which means if anything it has gone down, given higher chance the poll will be distorted by conditions in Russia.

Then again, one could say that if things are going unexpectedly badly, and yet your expected future approval rating didn’t change, that’s great news. Being in a quagmire or having to leave Ukraine would presumably be bad for his approval rating, yet he’s maintaining expectations (unless that prediction is simply stale, or there’s compensation). So it shows his crackdown and propaganda are working, or simply that Russians like the idea of the war more than we expected.

Which way to adjust on the crackdowns is another question. The need to do them does not bode well, but doing them does in terms of holding onto power. Perhaps the ability to execute this much crackdown smoothly is a large positive update.

Another possibility is if a lot of the chance Putin was removed involved him having been removed quickly. The case would be that having survived the Ides of March and the imposition of various crackdowns, it’s now too late to prevent the meltdown, and shows no one can pull this off. This improves his chances. There’s certainly some time value involved here, the question is how much, and I have a hard time believing (especially based on historical timing of such events) that this is a big factor yet.

I don’t know the right answer. I do think that the market is trading somewhat too high now relative to where it was, and I’d be tempted to sell at current market prices.

In this post, I haven’t attempted to actually lay out my model of how the war is going and who is likely to win under the Fog of War. I am inclined towards models that make this unwinnable for Russia going forward, but of course am still cautious due to the hostile information environment. And laying out the thinking there is beyond scope.

There’s two markets that together tell us how likely Zelenskyy and Putin are to meet.

I updated this to 45% because things are certainly moving in this direction and certainly should be going up now that he’s more likely to survive, but so far Putin has sent very low-level people to talk and I don’t really expect him to do it himself because it would show weakness and hurt his negotiating position to recognize Zelenskyy in this way.

If it does happen, when will it happen?

(This is my attempt to mimic the existing distribution.)

This says that Putin likely will hold out on this for a bit, but that the probability peaks around late April. My guess is it peaks earlier than that. The theory is that at some point Putin realizes some combination of (1) he can’t get submission via threats of killing civilians and destroying property, (2) he can’t win a military victory and (3) he therefore must reach an agreement, and (4) Zelenskyy is willing to give terms that are worth discussing.

An agreement could be reached without those first three, but in those cases it’s from a position of strength so I expect Putin would then refuse to meet. So if he is meeting, it’s from weakness, I’d put a high probability an agreement is then reached, and I’d expect it to be good for Ukraine. So the question is, if that happens, when does it happen? My guess is that the situation will become clear reasonably soon if it isn’t already – it shouldn’t take too long to play what cards remain that might finish it short term.

The probability here in November and December is obviously too low, but I’m pretty sure that’s because of how Metaculus has people draw curves by default – people mostly didn’t bother to add components to prevent the drop-off from being too big.

Other Interesting Markets

Lots of other markets are potentially interesting for various reasons. Scott picked out a few that I didn’t talk about so far, changes are since February 28.

Will World War III happen before 2050?: 20% —→21%
Will Russia invade any other country in 2022?: 12% —→10%
Will 50,000 civilians die in any single Ukrainian city?: 8% —→ 12%

The World War III odds barely budged. That in and of itself is interesting. There are a few different ways to interpret this, in some combination.

The chance of this escalating to World War III was never all that high, this is mostly background risk. Nope, can rule this out, market was 11% last year.
The chance was largely secondary conflicts after Ukraine, where Russia attempts to keep going, and that scenario is now even more unlikely.
The chance of this escalating involved a lot of That Escalated Quickly, and involved NATO sticking its nose into Ukraine and doing things like declaring a no-fly zone, whereas it’s now clear they’re not going to do that.
This market is kind of not real given the 30-year time frame and shouldn’t be taken seriously?
Maybe various things kind of cancel each other out?

I am mostly inclined to take door number four here, with a side of number five. Ukraine doing well should decrease odds of WW3, Russia not being willing to compromise as of yet and escalating increases it. In particular, the new path could be something like ‘Russia uses chemical and/or biological weapons, claims Ukraine did it, this forces us to escalate’ and then things spiral out of hand. But mostly I think this simply isn’t a situation prediction markets handle well.

Russia invading another country seems less likely and if anything this seems like too small an adjustment. With how badly Ukraine has gone, it seems unlikely Russia will have the resources to open a second front, or that it would want to risk that kind of escalation. Presumably they would take time to consolidate and recover. Before I was a buyer, now I’m at least much less of one.

The question on civilians dying in a city definitely seems directionally correct and if anything I’d call it too small as well. The chances of heavy civilian casualties are clearly going up quite a bit. Consider that in the worlds where Kyiv falls by now, the chances of 50,000 civilian casualties in a single city is not that high. Whereas now it’s possible there could be extended shelling of many cities. I still don’t think this is that likely, 50,000 really is a lot unless it’s fully on purpose, but I have to be at least somewhat of a buyer due to the history.

The only remaining Polymarket offering I find interesting is on the no-fly zone. It seems uncomfortably high, but also markets like this trade too high.

Other Metaculus Markets

You can always see the full state of the Metaculus Ukraine markets here.

I don’t think it’s worth going over the full list in detail again. Part of this is that they keep adding additional markets, many of which involve distributions, and not organizing the markets very well. It’s very difficult to keep up, and trying to properly compete in the contest seems super time intensive.

It also seems like it’s going to make predictions worse. The problem is that Meatculus rewards participation. Thus, I’d need to cover all the markets with predictions. But for most of them, I’m not going to have an opinion nor is it worth doing research, so I’ll throw some low-information prediction up that is mostly noise (e.g. market +/- 5% based on simple heuristics). That in turn makes it harder for prices to adjust.

I don’t know what the right solution for this is. One possibility is to have a ‘no opinion’ selection, on by default, that gives you the score for always exactly copying the market price. All I know is, I’m pretty exhausted by the whole experiment.

Any sign of the markets I asked for?

Will a NATO country invoke article 5 by [date]?
Will Russia invade any country other than Ukraine, Moldavia, Georgia or Belarus by [date]?
Will a nuclear weapon detonate in Europe by [date]?
Will a nuclear weapon detonate in a NATO country by [date]?
Will at least 10 nuclear weapons detonate anywhere in the world by [date]?
Will Russia use biological or chemical weapons?

That first one we have for March 31 but not for later in the year. The others we don’t have, although we have the question on 50k civilian deaths in a city as a kind of proxy for WMDs. I’d very much like to see a market on #6 given how things are going, whereas we have various proxies for the others of varying usefulness.

Will Russia control [Kyiv, other major city, some number of such cities] by [date]?
Will Russia control at least X% of Ukrainian [population/territory] by [date] according to the Wikipedia map? Or a particular place/objective?
Will there be video proof that Zelenskyy is physically in [Kyiv/Ukraine], alive and not in Russian custody during the week/month of [date]?
Will Zelenskyy be President of Ukraine on [date]?
Will Putin be President of Russia on [date]?
Will there be an officially agreed-upon cease fire in place on [date]?
Will Ukraine join the Union State by [date]?
Will Ukraine announce its demilitarization by [date]?
Will Ukraine officially recognize Russia’s annexation of Crimea by [date]?
Will Ukraine officially recognize the independence of Donbas by [date]?
Will Russia withdraw its forces from Ukraine by [date] [with/without] excluding the Donbas and Crimea regions?

We have good coverage of #7 as discussed above. #8 got predicted by Samo Burja as 70% of Ukrainian territory plus Kyiv by day 50, a bet was taken on Twitter at 50%. I would have happily been on the No side of that. I do think #8 would be a good market, but so far I don’t see a similar one.

#10 we only have the short-dated stuff, at this point we should be more interested in longer-dated stuff. #11 again we only have the one date, I’d like to see shorter and perhaps also longer dated ones if possible.

We have a cease-fire market (#12) for the whole year that’s currently at 78%, but some shorter-dated ones seem like they’d be valuable.

#13 is well-covered. #14 seems worthwhile. #15 and #16 we have the Polymarket synthesis that seems good enough. #17 I still want to see, likely moderate-to-long-dated (e.g. June 1 or January 1).

Will the Russian Central Bank’s assets remain frozen on [date]?
Will [major Russian bank / banks]’s assets remain frozen on [date]?
Will the Russian stock market be open on [date]?
Will foreigners be allowed to sell Russian stocks on [date]?
Will [Poland/country/EU] deny Russia access to its airspace on [date]?
Will Russia be allowed to play FIFA-sponsored matches on [date]?
Will there be a new owner of Chelsea football club on [date]?
Will at least $X of [NYC/London] real estate be confiscated or sold by Russian nationals by [date] (could be hard to verify, but could be interesting)
Will Russia cut off gas sales to Europe on [date]?
Will Russia sell at least [amount] of natural gas to Europe the [week/month] of [date]?

Mostly this stuff is moot at this point, or now represents only whether sanctions get lifted by then, but I’d like to see a moderate-or-long-dated version of #18, and I’d very much like to see #26. Others could be interesting too as ‘clean’ versions of whether things have been settled.

Will Germany have an operational nuclear power plant on [date]?
Will the United States [approve/announce] the construction of a new nuclear power plant by [date]?
Will the United States substantially change its regulatory approval process for nuclear power plants by [date]?
Will the United States announce $X or more additional funding for energy production [of any kind/nuclear/green that isn’t nuclear] by [date]?
Will [any European nation/X nations] announce the construction of [a/X] new currently unannounced nuclear power plant(s) by [date]?
Will any [min X size] Western country institute a carbon tax of at least X% by [date]?
Will [country/EU/USA] offer asylum to all deserting Russian soldiers by [date]?
Will [country/EU/USA] offer entry to all Russian citizens [with a college degree/with a sufficient job offer/at all]?
How many Ukrainian refugees will enter the EU by [date]?
How much foreign aid will USA send Ukraine by [date]?
How much foreign aid will all countries send Ukraine by [date]?
[Various questions surrounding cyber attacks but they seem not well-defined for now.]
Will Sweden join NATO by [date]?
Will Finland join NATO by [date]?
~~Will Ukraine be a candidate member of the EU on [date]?~~
~~And hell: Will Ukraine join NATO by [date]?~~

Given the way things are going I’m not seeing much remaining interest in most of this. I’d still like to see various markets on energy investment but it seems like no one is taking the issue seriously. They’re definitely not taking the brain drain options seriously, quite the opposite, alas. The cyber attack questions still aren’t defined but Metaculus is at least asking them.

What additional markets would be interesting that we don’t have yet?

I don’t want to overwhelm with options, so I’d say the questions I most want to see added right now, aside from questions about the way the fighting is going (which need to pick sources to trust, but I do wish were being explored more beyond the cities) are questions around what China will do.

Things like:

Will China provide weapons or other military equipment to Russia?
Will China provide MREs to Russia?
Will China buy at least [X] amount of Russian oil or gas?
Will Chinese company [X] honor the sanctions against Russia?
Will Chinese state media continue to essentially take Russia’s side?
Will Chinese support for the Russian invasion be at least X% in polls on [date]?
Will China take delivery of at least [X] bars of Russian gold by [date]?

And so on. That’s not a great list, but it’s a start. The question is what we can safely resolve, so let’s get creative.

All right, in the interests of speed I’ll call it there. I hope this was helpful. What markets (at Metaculus or real money) look mispriced to you? What would you buy or sell and why?