Here’s what would make me question Nate’s credibility: If we look at the 538′s former projections and compare them to the betting markets prior to election, make Kelly bets, and lose money.[1] Note that 538 can be on the wrong side of the market some of the time and still be profitable.
Saying Nate lacks enough credibility when he is well calibrated and on the right side of the market doesn’t make any sense and seems like an extreme case of hindsight bias. For example, you use the 2016 election to undermine 538′s credibility, but (if I recall correctly) 538′s forecast gave Trump a greater chance than the market’s odds implied, so if you bet based on 538′s forecast you would have bet on Trump and made money. I could be remembering wrong, so if someone links this data, they’d get an upvote from me.
- ↩︎
Looking at all orderings of elections
And this year too. I agree about the small sample size if we’re just asking “Who will be president?” We should use every one of 538′s predictions that could have been used to make a bet in a liquid betting market.
If a profitable set of Kelly/Markowitz bets (under all election orderings) has a worse Brier score than the market’s Brier score, I’d be more interested in the profit of the bets, since we’re talking about betting money.