Here’s a script I wrote to analyze how good Manifold Markets is at predicting Ukraine stuff. Basically: it’s about as good as you would be if you were calibrated at 80% accuracy if you average market prices over the life of the market, and if you take the probabilities at the mid-point of the market, it’s about as good as you would be if you were calibrated at 72% accuracy.
Here’s a script I wrote to analyze how good Manifold Markets is at predicting Ukraine stuff. Basically: it’s about as good as you would be if you were calibrated at 80% accuracy if you average market prices over the life of the market, and if you take the probabilities at the mid-point of the market, it’s about as good as you would be if you were calibrated at 72% accuracy.
In order to figure out how good this is, you’d also want to check how hard the questions were.