The problem is that I have no dataset on which to test an algorithm. Even if I could get access to the trade history of a large prediction market, sorted by trader, it would still be problematic to convert trades directly to predictions. if I buy Palin-Withdraw at 5% and sell it at 7%, is it because I think her chance of withdrawing is 6% or because I’m playing the market? PredictionBook.com might work—my guess is there’s not enough users and not enough calibration data yet to beat Intrade, but I could be wrong.
That’s why I’ve worded the article more like advocacy than like an academic paper. People need to try it before we can test it. If you don’t think people should try it, you need to object on hypothetical grounds, not demand unobtainable evidence.
(People like money more than they like making good predictions. They play the market, they’re risk averse, and they discount value over time. The latter two demonstrably lead to systemic calibration errors in real prediction markets, according to a paper I just stumbled across while looking for data.)
If you don’t think people should try it, you need to object on hypothetical grounds, not demand unobtainable evidence.
I’m demanding a free money generator. The point here isn’t that the evidence is unobtainable but rather that the very nature of markets dictates that if such an algorithm can be obtained then any sufficiently large market will quickly be exploited by someone with the algorithm and so will learn from it. This just means that you need to focus your attention on applications that for some reason cannot be large enough or open enough to operate remotely efficiently.
I understand the principle (and your advice is obviously sound and well-taken), although I’m still feeling adversarial enough to note that the inverse also holds. If you’re using a market to be the best voter in a histocracy, the histocracy learns from the market.
And there’s no free money to be had when the market’s being systemically irrational due to individual traders being rational. I think this contract is way too high, but I’m not going to risk (and render illiquid) a few thousand dollars now to win a few hundred dollars in November, and that’s why the contract is overvalued. There may be no investor anywhere who’s willing to pay $1000 in January in exchange for a 99% chance of $1030 in November, which would mean that no matter how big Intrade gets, longshot contracts will remain overvalued. This isn’t a counterexample to general efficient market theorems; it’s just that the economically correct price for the contract is not equal to its expected value.
I think it’s worse than that. Suppose we’re in a futarchy and there’s a proposal to build an innovative new kind of nuclear reactor in the heart of Cardiff. According to our state-determined utility function, this has positive utility if and only if the risk of meltdown within 10 years is less than .00001. But regardless of what people actually think, the price of CardiffReactor.Meltdown.Before2022 has a hard floor: the price where it’s not worth it to anyone on the planet to short-sell, because there are better investment opportunities elsewhere. If this floor is greater than .00001 (and how could it not be?), the market provides no usable information.
The difficulty here is one of constructing markets and, critically, derivatives that make trading based on that sort of information feasible. This is the sort of thing that would require infrastructure in place to allow large amounts of complicated trading. If we were actually in a serious futarchy this kind of thing could and probably would be traded on, albeit indirectly in a huge sea of conditional probability payoff systems.
For us, given that we are not in a mature futarchy (and do not otherwise have access to an advanced and heavily traded prediction market), we are of course unable to use a market to directly answer that kind of question.
The problem is that I have no dataset on which to test an algorithm. Even if I could get access to the trade history of a large prediction market, sorted by trader, it would still be problematic to convert trades directly to predictions. if I buy Palin-Withdraw at 5% and sell it at 7%, is it because I think her chance of withdrawing is 6% or because I’m playing the market? PredictionBook.com might work—my guess is there’s not enough users and not enough calibration data yet to beat Intrade, but I could be wrong.
That’s why I’ve worded the article more like advocacy than like an academic paper. People need to try it before we can test it. If you don’t think people should try it, you need to object on hypothetical grounds, not demand unobtainable evidence.
(People like money more than they like making good predictions. They play the market, they’re risk averse, and they discount value over time. The latter two demonstrably lead to systemic calibration errors in real prediction markets, according to a paper I just stumbled across while looking for data.)
I’m demanding a free money generator. The point here isn’t that the evidence is unobtainable but rather that the very nature of markets dictates that if such an algorithm can be obtained then any sufficiently large market will quickly be exploited by someone with the algorithm and so will learn from it. This just means that you need to focus your attention on applications that for some reason cannot be large enough or open enough to operate remotely efficiently.
I understand the principle (and your advice is obviously sound and well-taken), although I’m still feeling adversarial enough to note that the inverse also holds. If you’re using a market to be the best voter in a histocracy, the histocracy learns from the market.
And there’s no free money to be had when the market’s being systemically irrational due to individual traders being rational. I think this contract is way too high, but I’m not going to risk (and render illiquid) a few thousand dollars now to win a few hundred dollars in November, and that’s why the contract is overvalued. There may be no investor anywhere who’s willing to pay $1000 in January in exchange for a 99% chance of $1030 in November, which would mean that no matter how big Intrade gets, longshot contracts will remain overvalued. This isn’t a counterexample to general efficient market theorems; it’s just that the economically correct price for the contract is not equal to its expected value.
This is true. Translation is required.
I think it’s worse than that. Suppose we’re in a futarchy and there’s a proposal to build an innovative new kind of nuclear reactor in the heart of Cardiff. According to our state-determined utility function, this has positive utility if and only if the risk of meltdown within 10 years is less than .00001. But regardless of what people actually think, the price of CardiffReactor.Meltdown.Before2022 has a hard floor: the price where it’s not worth it to anyone on the planet to short-sell, because there are better investment opportunities elsewhere. If this floor is greater than .00001 (and how could it not be?), the market provides no usable information.
The difficulty here is one of constructing markets and, critically, derivatives that make trading based on that sort of information feasible. This is the sort of thing that would require infrastructure in place to allow large amounts of complicated trading. If we were actually in a serious futarchy this kind of thing could and probably would be traded on, albeit indirectly in a huge sea of conditional probability payoff systems.
For us, given that we are not in a mature futarchy (and do not otherwise have access to an advanced and heavily traded prediction market), we are of course unable to use a market to directly answer that kind of question.