I got heavily downvoted for suggesting that Nate Silver’s credibility was not strong enough to make this a good bet.
The most you could say at this point (late in the night on election day) is that it looks like the election is very close. This suggests to me that those people saying that betting on it at 65% odds for Biden was a huge steal, were overconfident. People were quoted who thought Biden’s P(win) was 96%.
I am interested in any suggestions about what went wrong. I can’t think of a lot of edifying reasons.
1. Rationalists don’t win
2. Dunning-Kruger effect
3. General overconfidence
4. Saying Trump had a chance runs the risk of signaling that you are a “Trumptard”.
5. Stating high confidence is one way to signal high intelligence. “Uh I’m not sure” doesn’t sound too smart.
We are in a scenario Nate Silver explicitly described:
But what’s tricky about this race is that — because of Trump’s Electoral College advantage, which he largely carries over from 2016 — it wouldn’t take that big of a polling error in Trump’s favor to make the election interesting. Importantly, interesting isn’t the same thing as a likely Trump win; instead, the probable result of a 2016-style polling error would be a Biden victory but one that took some time to resolve and which could imperil Democrats’ chances of taking over the Senate. On the flip side, it wouldn’t take much of a polling error in Biden’s favor to turn 2020 into a historic landslide against Trump.
As of writing this comment Biden has several paths to victory and is heavily favored to hit at least one. I think he is about 75% to win at this point, despite all the bad news. We are in a world where Biden might win despite losing all of PA FL OH NC. Very little had to go right for Biden to win. Neither myself nor Nate Silver made any statements saying the election would definitely be a landslide. I told people to bet on Biden to win not to bet on Biden winning in a landslide. I will say there were some people pushing more aggressive Bets (Rainbow Jeremy comes to mind) and we should definitely update against their point of view. But Nate remains credible.
FWIW I’m 99% sure RJ made money from this election, and I’m 50% sure he made over 90k. Why would you update against someone who has consistently made enormous amounts of money betting on his beliefs?
Edit: looks like I was a bit overoptimistic about his profits but he supposedly did make a decent amount
But he’s been a professional politics gambler for years now, which seems like much stronger evidence for evaluating his calibration than the results of one election cycle?
That does not seem like very strong evidence his bets on the 2020 election were good. He did better than people who refused to bet at all. But he did worse than people who just bet on Biden. Many people figured out the Biden bet in the poker and gambling communities. You should certainly downgrade your opinion of RJ relative to people who did a lot better.
People were quoted who thought Biden’s P(win) was 96%.
Please provide actual quotes. I don’t know anyone who held that belief. Not even Nate Silver held that belief if you take into account election shenanigans. Nobody in the thread mentioned anything close to that probability. Most expressed probabilities I’ve found by rationalists are something in the 70-80% range for Biden, which at this point still doesn’t strike me as unreasonable.
Aren’t most of Biden votes on mail votes, which are counted later?
From what I’m seeing this still doesn’t tell us that we aren’t in a “Biden wins without a huge margin”, which was the largest chunk of those 80%/90% estimates. I expected waking up at this time and seeing that the votes were momentarily favouring Trump (in fact I had to order my brain to expect and prepare for that).
“Biden wins” hypothesis has narrowed without those “Biden wins by a large chunk” scenarios being likely anymore, but in the “Biden wins” hypothesis I wouldn’t have called seeing that unlikely.
So far there’s not so much “Trump wins” evidence I’d assume something was wrong with the prior odds, I’d think that if the remaining votes would be expected to be evenly distributed, but they are not.
My estimate 2 days ago was Biden at 85% to win, because a lot of things had to go in a very specific way for it to not happen or there had to be an error without any precedent in the polls. I felt like I was supposed to go at least 90%, but couldn’t bring myself to do that because I felt overconfident in doing so, even if I was pretty sure my estimate was erring in the opposite side.
I think that a lot of rationalists assumed that the most common odds they were hearing (80% or so) had been too cautious because pollsters were afraid to make the same wrong call they made last time, so they augmented the odds a bit. What convinced me to go from 80% to 85% was seeing that Biden lead was three time the lead Hillary had, so that kind of fluke was really unlikely.
If rationalists had been overconfident I’d expect this correction from pollsters excessive under confidence played a large part on anyone who wasn’t just calculating odds on sheer polls data.
If Trump somehow wins… I think we’d have either witnessed something previously unknown that made the polls a lot less reliable (this seems to be a very particular election for lots of factors), which usually doesn’t happens so betting on Biden then at those odds was still the intelligent thing to do, since you’d make money most times you did that, or that we simply got the unlikely result with things going in a specific way.
But I can’t think of anything rationalists should have done differently.
Here’s what would make me question Nate’s credibility: If we look at the 538′s former projections and compare them to the betting markets prior to election, make Kelly bets, and lose money.[1] Note that 538 can be on the wrong side of the market some of the time and still be profitable.
Saying Nate lacks enough credibility when he is well calibrated and on the right side of the market doesn’t make any sense and seems like an extreme case of hindsight bias. For example, you use the 2016 election to undermine 538′s credibility, but (if I recall correctly) 538′s forecast gave Trump a greater chance than the market’s odds implied, so if you bet based on 538′s forecast you would have bet on Trump and made money. I could be remembering wrong, so if someone links this data, they’d get an upvote from me.
I’m too lazy to look it up, but I did research this a couple of weeks ago and found that 538 had indeed outperformed the markets both in 2008 and 2016 (I wasn’t able to find data for 2012). This is not very informative, though, since it’s just a couple of cases. Much better is to look at the state-level predictions and use brier scores as a measure of forecasting performance.
And this year too. I agree about the small sample size if we’re just asking “Who will be president?” We should use every one of 538′s predictions that could have been used to make a bet in a liquid betting market.
If a profitable set of Kelly/Markowitz bets (under all election orderings) has a worse Brier score than the market’s Brier score, I’d be more interested in the profit of the bets, since we’re talking about betting money.
Here are the results we currently have put into 538′s interactive map. I’ve been strictly kind to Trump, meaning I’ve put him as winning some races that aren’t yet called, while only giving Biden races that have been called.
538 thinks this comes out at ~86/7/7 Biden/Trump/Tie (these are simulations and the numbers seem to fluctuate). And I believe Nebraska’s second district is also won by Biden (I’m confused about that one, can’t find an official source), in which case all the ties to go Biden.
I’m not convinced this looks bad for Nate Silver. We seem to be in the pessimistic half of the distribution (from Biden’s pov), but not that far. It’s too early to be sure, but there seems to have been a massive polling error in Florida, and some smaller errors in other swing states in Trump’s favor. Again, that’s worse for Biden than the median outcome, but not crazy worse.
I got heavily downvoted for suggesting that Nate Silver’s credibility was not strong enough to make this a good bet.
The most you could say at this point (late in the night on election day) is that it looks like the election is very close. This suggests to me that those people saying that betting on it at 65% odds for Biden was a huge steal, were overconfident. People were quoted who thought Biden’s P(win) was 96%.
I am interested in any suggestions about what went wrong. I can’t think of a lot of edifying reasons.
1. Rationalists don’t win
2. Dunning-Kruger effect
3. General overconfidence
4. Saying Trump had a chance runs the risk of signaling that you are a “Trumptard”.
5. Stating high confidence is one way to signal high intelligence. “Uh I’m not sure” doesn’t sound too smart.
We are in a scenario Nate Silver explicitly described:
As of writing this comment Biden has several paths to victory and is heavily favored to hit at least one. I think he is about 75% to win at this point, despite all the bad news. We are in a world where Biden might win despite losing all of PA FL OH NC. Very little had to go right for Biden to win. Neither myself nor Nate Silver made any statements saying the election would definitely be a landslide. I told people to bet on Biden to win not to bet on Biden winning in a landslide. I will say there were some people pushing more aggressive Bets (Rainbow Jeremy comes to mind) and we should definitely update against their point of view. But Nate remains credible.
FWIW I’m 99% sure RJ made money from this election, and I’m 50% sure he made over 90k. Why would you update against someone who has consistently made enormous amounts of money betting on his beliefs?
Edit: looks like I was a bit overoptimistic about his profits but he supposedly did make a decent amount
https://twitter.com/rainbow_jeremy_/status/1336815220061327363
(keep in mind he lies all the time so this is only noisy evidence)
Many people including myself got way better percentage and absolute returns on the election. He was way too optimistic.
But he’s been a professional politics gambler for years now, which seems like much stronger evidence for evaluating his calibration than the results of one election cycle?
That does not seem like very strong evidence his bets on the 2020 election were good. He did better than people who refused to bet at all. But he did worse than people who just bet on Biden. Many people figured out the Biden bet in the poker and gambling communities. You should certainly downgrade your opinion of RJ relative to people who did a lot better.
Please provide actual quotes. I don’t know anyone who held that belief. Not even Nate Silver held that belief if you take into account election shenanigans. Nobody in the thread mentioned anything close to that probability. Most expressed probabilities I’ve found by rationalists are something in the 70-80% range for Biden, which at this point still doesn’t strike me as unreasonable.
Aren’t most of Biden votes on mail votes, which are counted later?
From what I’m seeing this still doesn’t tell us that we aren’t in a “Biden wins without a huge margin”, which was the largest chunk of those 80%/90% estimates. I expected waking up at this time and seeing that the votes were momentarily favouring Trump (in fact I had to order my brain to expect and prepare for that).
“Biden wins” hypothesis has narrowed without those “Biden wins by a large chunk” scenarios being likely anymore, but in the “Biden wins” hypothesis I wouldn’t have called seeing that unlikely.
So far there’s not so much “Trump wins” evidence I’d assume something was wrong with the prior odds, I’d think that if the remaining votes would be expected to be evenly distributed, but they are not.
My estimate 2 days ago was Biden at 85% to win, because a lot of things had to go in a very specific way for it to not happen or there had to be an error without any precedent in the polls. I felt like I was supposed to go at least 90%, but couldn’t bring myself to do that because I felt overconfident in doing so, even if I was pretty sure my estimate was erring in the opposite side.
I think that a lot of rationalists assumed that the most common odds they were hearing (80% or so) had been too cautious because pollsters were afraid to make the same wrong call they made last time, so they augmented the odds a bit. What convinced me to go from 80% to 85% was seeing that Biden lead was three time the lead Hillary had, so that kind of fluke was really unlikely.
If rationalists had been overconfident I’d expect this correction from pollsters excessive under confidence played a large part on anyone who wasn’t just calculating odds on sheer polls data.
If Trump somehow wins… I think we’d have either witnessed something previously unknown that made the polls a lot less reliable (this seems to be a very particular election for lots of factors), which usually doesn’t happens so betting on Biden then at those odds was still the intelligent thing to do, since you’d make money most times you did that, or that we simply got the unlikely result with things going in a specific way.
But I can’t think of anything rationalists should have done differently.
Here’s what would make me question Nate’s credibility: If we look at the 538′s former projections and compare them to the betting markets prior to election, make Kelly bets, and lose money.[1] Note that 538 can be on the wrong side of the market some of the time and still be profitable.
Saying Nate lacks enough credibility when he is well calibrated and on the right side of the market doesn’t make any sense and seems like an extreme case of hindsight bias. For example, you use the 2016 election to undermine 538′s credibility, but (if I recall correctly) 538′s forecast gave Trump a greater chance than the market’s odds implied, so if you bet based on 538′s forecast you would have bet on Trump and made money. I could be remembering wrong, so if someone links this data, they’d get an upvote from me.
Looking at all orderings of elections
I’m too lazy to look it up, but I did research this a couple of weeks ago and found that 538 had indeed outperformed the markets both in 2008 and 2016 (I wasn’t able to find data for 2012). This is not very informative, though, since it’s just a couple of cases. Much better is to look at the state-level predictions and use brier scores as a measure of forecasting performance.
538 totally outperformed in 2012 on intrade—it seems like there were whales pushing up the romney price on intrade.
And this year too. I agree about the small sample size if we’re just asking “Who will be president?” We should use every one of 538′s predictions that could have been used to make a bet in a liquid betting market.
If a profitable set of Kelly/Markowitz bets (under all election orderings) has a worse Brier score than the market’s Brier score, I’d be more interested in the profit of the bets, since we’re talking about betting money.
Here are the results we currently have put into 538′s interactive map. I’ve been strictly kind to Trump, meaning I’ve put him as winning some races that aren’t yet called, while only giving Biden races that have been called.
538 thinks this comes out at ~86/7/7 Biden/Trump/Tie (these are simulations and the numbers seem to fluctuate). And I believe Nebraska’s second district is also won by Biden (I’m confused about that one, can’t find an official source), in which case all the ties to go Biden.
I’m not convinced this looks bad for Nate Silver. We seem to be in the pessimistic half of the distribution (from Biden’s pov), but not that far. It’s too early to be sure, but there seems to have been a massive polling error in Florida, and some smaller errors in other swing states in Trump’s favor. Again, that’s worse for Biden than the median outcome, but not crazy worse.