Prediction Markets Are Mediocre
The fantasy of prediction markets is that we can use them for policy proposals to inform people about the consequences of their actions. Conditional prediction market “Will X lead to terrible consequence Y” theoretically can give helpful insights and prevent X from happening, if is high enough.
The reality of prediction markets is this:
We may say, that this is not bad at all. The market is clearly reacting to the news and updating its estimate. It aggregates uncertainty of multiple people into a single probability estimate. It’s doing its job. Now it’s very confident in tariffs, and it has never been too low.
The problem is that knowing that Trump will almost certainly impose large tariffs five months after the election is useless for the initial purpose of choosing the right candidate to vote for.
Here is how the same market looked when it’s predictions were actually relevant:
It fluctuated wildly rising to 75% and falling to 36%. On the election day it was merely 56%. Again, this isn’t necessary bad, these estimates might have been reasonable, considering available information back then. It’s not that prediction markets do not work at all. It’s that they do not work good enough to make a difference.
From the Perspective of a Voter
Imagine yourself in the shoes of a voter leaning towards voting for Trump in this election. Not a MAGA fanatic. But a person whose overall cultural and aesthetic sensibilities are more aligned with Republicans. You are pro-market, pro-guns, tough on crime, dislike DEI, etc.
You are in a bit of a conundrum here. On one hand Trump just overall vibes with you. But he is talking about tariffs. And according to Economy 101, which you pride yourself on being familiar with, tariffs are really bad. So, in theory, this issue could be a deal-breaker for you.
You check a prediction market and see 56% estimate for tariffs being imposed under Trump’s first year. Will it change your vote? Will you think that even this is high enough chance of terrible consequences? Or will you come up with some rationalization, instead?
Prediction markets do not just predict the stated outcome. They predict their own estimate in the future. Maybe people voting “yes” just expect the market to go up later based on the media hysteria, which will follow Trump’s election, while not expecting at all that actual tariffs will be imposed?
Or maybe they are trying to exploit the conditional structure of the market? After all, if Trump is not elected, the market is resolved to N/A. So some Democrats can scaremonger by betting on this market in an attempt to create a self-defying prophesy and prevent Trumps election, without much risk to themselves, even if they do not believe that Trump would actually impose large tariffs. Frankly, in this case you have epistemological obligation to vote for Trump and punish these insidious plotters!
So you still vote for Trump. Just as other voters in the same situation. Then Trump is elected[1] and starts implementing tariffs, beyond your wildest fears.
Doubling Down
Can we solve this issue by even more prediction markets?
Say, we added more liquidity? That can marginally help. The fluctuations would probably be less dramatic. But there is no reason to expect more liquid market to arrive to a significantly higher estimate that would’ve persuaded our imaginary Trump voters to change their minds.
What if we created a second prediction market about the future estimate of this one? Suppose there was a market: “Trump imposes large tariffs in first year market is beyond 80% on the 4th of April 2025”. If this market was high enough on the election day, that could realistically change the minds of voters, right?
But why would such market be high if the initial market wasn’t? In the end, both of them are trying to capture the same core fact about reality. Mostly the same information is relevant for both markets. The second one simply adds an extra levels of indirection.
Moreover, suppose that second market indeed was significantly higher. What is a case where someone would want to bet “yes” on the second market but not on the first? It’s when Trump does not impose large tariffs this year, while people mistakenly believe that it will happen. So even if second market is higher, than the first, it doesn’t serve as a good reason not to vote Trump.
And so once again, Trump is elected and large tariffs are imposed.
Conclusion
Therefore, the sad conclusion—prediction markets are mediocre. They work, they give some valuable information, but their ability to predict things several months in advance is not good enough to actually be useful. Limitations of prediction markets are severe and we should be very aware of them.
- ^
I’m not claiming that prediction markets significantly contributed to Trump’s victory. Most voters probably didn’t even think to check them. The point is that even if voters did everything right and checked prediction markets as part of their decision making algorithm, it wouldn’t help.
I unfortunately don’t think this proves anything relevant. The example just shows that there was one question where the market was very uncertain. This neither tells us how certain the market is in general (that depends on its confidence on other policy questions), nor how good this particular estimate was (that, I would argue, depends on how far along the information chart it was, which is not measurable—but even putting my pet framework aside, it seems intuitively clear “it was 56% and then it happened” doesn’t tell you how much information the market utilized).
This depends on the first point, which again requires looking at a range of policy markets, not just one. And actually, I personally didn’t expect Trump to do any tarrifs at all (was 100% wrong there), so for me, the market would have updated me significantly into the right direction.
A relevant plot from the wonderful calibration.city:
Early in a prediction questions lifetime, the average Brier score is something like 0.17 to 0.24, which is like a 1-8% edge.
This market seems valuable, but it depends on what you’re using it for. “Experts can’t predict what this politician is going to do” is useful information. Also it seems like a lot of voters were assuming the chance of Trump implementing major tariffs was tiny, so updating toward 50% would have helped them.
Indeed: the background probability of a sane leader launching an unprovoked trade war on the rest of the world is near zero. 10% would be unsettling, 30% would be (and was!) alarming.
I downvoted because the conclusion “prediction markets are mediocre” does not follow from the premise “here is one example of one problem that I imagine abundant legal well-capitalized prediction markets would not have completely solved (even though I acknowledge that they would have helped move things in the right direction on the margin)”.
I’m confused. It feels like you’re basically saying that reality is uncertain, prediction markets reflect this, but in order to be useful for affecting the minds of ordinary voters (a specific use case that is not the main use case I think prediction markets matter for) they must be certain or near certain
Don’t prediction markets also serve as a tool for actually valuing a prediction? Without a clear metric to judge the likelihood of a prediction at the time it was made (as is the case with these one-off real world predictions like elections or what a politician will do), I’m liable to consider the guy who predicted the sun will come up tomorrow, and the guy who predicted the market will drop 8% last week as having an equal success rate.
We need something to judge a prediction against, otherwise people would just go for easy predictions that might sound complicated in hindsight, and when they get them right most of the time, tout their “impressive” prediction record. If we can instead say “The market was already predicting this was going to happen with a 95% certainty when you predicted it”, we can know their prediction wasn’t at all unordinary, or valuable. Likewise, if the market predicted something with a lower certainty, say 50%, and our predictor predicted with a 95% certainty, and was right, and this happened consistently, we could consider their predictions as more valuable than chance, with high alpha.
Maybe there’s some political commentator out there who analyzes Trump’s claims and expectations clearly, assigns accurate probabilities better than the market (and bets on them), then we can choose to rely on that commenter in the future as far as predicting Trump’s actions, “beating” the market. We couldn’t find that person without the market in the first place, so even if the market itself doesn’t communicate much information about the underlying probability, it can communicate information about who is better than the market at estimating it.
30% was already a ridiculously high risk, and should have already been decisive if this was the deciding issue for a voter. You don’t have to know that the bullet is in your chamber to decide not to play russian roulette.
There’s no guarantee that prediction markets do well,
I came to similar conclusions after another geopolitical event few years ago.