A quick retrospective on the four Manifold markets on the day, focusing on the possibility of good or bad incentives flowing out of the markets. Several people expressed concerns that the Manifold markets would cause LessWrong’s home page to go down, or go down earlier, due to incentives. People also placed bets and limit orders to try to mute those bad incentives. Some examples:
“This is a classical example where having a prediction market creates really bad incentives.”
“I think the right thing to do here is buy yes so that it’s more beneficial for others to buy and hold no.”
“I’m not wild about this market existing, but given that it does exist I’m strongly in favour of making it profitable for others to not press the button.”
I decided to look into this. I want to say up front that I think all the incentives were very small relative to perceived stakes, and I have no suspicions of anyone after writing this comment. I am also not going to give any bettor names.
By the end of the YES/NO market, several users had bet large sums on YES and were consequently incentivized to blow up the home page. One user had m26,002 YES shares (USD $260). Another user had 7328 NO shares (USD $73) and was incentivized not to blow up the site. But this doesn’t show the whole picture, because there were also limit orders. As was explained to me:
The choice of whether to leave (public) limit orders on one side or the other is the way you incentivize action.
At various points during the day there were large (m1000, etc) YES limit orders in play. This is completely useless as an incentive for an individual LessWrong user. Sure, if I was the only one who might press the button, I could bet through those limit orders, not press the button, and collect my bounty. But many people could press the button, and this would have lost me mana. They did count as an incentive for LessWrong coders! They could quietly introduce a bug preventing the button from being pressed, bet NO, and collect their winnings. Nobody took up that incentive.
I didn’t see any large NO limit orders. A NO limit order on a market like this is inherently risky, because if the button is pressed the offer will definitely be taken, but if the button is not pressed it probably won’t. If there had been then that could have added to the incentive some bettors had to blow up the home page, but as far as I can tell there weren’t.
The positions in the WHEN market were much smaller, with the largest positions coming in at 552 shares in LATE/NEVER and 325 shares in EARLY. The current design of range markets on Manifold is fun to play with, but is not very “swingy”. In particular, there is no option for outsized winnings if you can predict exactly when something will happen. If a user was planning to blow up the home page in the last few hours they could have bet a combination of LATE/NEVER in this market and YES in the binary market to maximize their gains, but I don’t see any evidence of this. Because there are fewer high karma users, large EARLY bets in this market could slightly limit their incentive to bet EARLY and blow up the home page early. Unlike the YES/NO market, those bets didn’t happen.
If someone was willing to anonymously blow up the home page, then a pattern of placing huge bets on that happening might look suspicious and damage their anonymity. So generally I would expect these incentives to only be effective for someone who was willing to blow up the home page and take credit for it. You can bet on that happening in Will anyone try to take credit for nuking LessWrong?. Or, place a limit NO order to incentivize someone to bet YES and then take the credit (and the mana).
Finally mkualquiera made a market—Will my friend agree to defect on Petrov Day?. This market is evidence that two people were thinking about being incentivized to blow up the home page by betting markets. But it seems like this was more about a fun game than making mana. The largest position was 301 shares on YES, and the market resolved NO, so whatever incentive those shares provided, they weren’t enough to make it happen.
In the end I think there is a low (10%) chance that anyone’s behavior was significantly shifted by prediction market incentives. Feel free to reply to this post with how your behavior was shifted so I can update.
I was hopeful that people might shift their behavior based on the prediction market predictions—specifically that the high probability placed on the home page being blown up would lead to design changes. However, this retrospective clarifies that Petrov Day 2022 was a social experiment, so the prediction would have just shown it was expected to work as designed.
… the primary design of the exercises – seeing how long it’d take for the site to go down, even if we were pretty sure that it would.
It’s also possible that people saw the high probability that the home page would be blown up and altered their plans. They may have not blown it up because they expected someone else would (especially if they weren’t confident in their anonymity being preserved). Or they may have blown it up because “if it’s going to happen, it may as well be me”. I think there is a higher chance (25%) that this was an effect, but I don’t know what the net direction would have been. Again, feel free to reply if you think the prediction markets helped you make decisions.
Overall I think the prediction markets were a positive addition to a negative event. I think the main incentive in play was the possibility of seeing, or not seeing, the LessWrong home page being blown up. And I think we all have a lot to figure out about prediction markets.
To clarify, me and my friend were 100% going to press the button, but we were discouraged by the false alarm. There was no fun at that point, and it made me lose like 1⁄3 of my total mana. I had to close all my positions to stop the losses, and we went to sleep. When we woke up it was already too late for it to be noteworthy or fun.
A quick retrospective on the four Manifold markets on the day, focusing on the possibility of good or bad incentives flowing out of the markets. Several people expressed concerns that the Manifold markets would cause LessWrong’s home page to go down, or go down earlier, due to incentives. People also placed bets and limit orders to try to mute those bad incentives. Some examples:
I decided to look into this. I want to say up front that I think all the incentives were very small relative to perceived stakes, and I have no suspicions of anyone after writing this comment. I am also not going to give any bettor names.
By the end of the YES/NO market, several users had bet large sums on YES and were consequently incentivized to blow up the home page. One user had m26,002 YES shares (USD $260). Another user had 7328 NO shares (USD $73) and was incentivized not to blow up the site. But this doesn’t show the whole picture, because there were also limit orders. As was explained to me:
At various points during the day there were large (m1000, etc) YES limit orders in play. This is completely useless as an incentive for an individual LessWrong user. Sure, if I was the only one who might press the button, I could bet through those limit orders, not press the button, and collect my bounty. But many people could press the button, and this would have lost me mana. They did count as an incentive for LessWrong coders! They could quietly introduce a bug preventing the button from being pressed, bet NO, and collect their winnings. Nobody took up that incentive.
I didn’t see any large NO limit orders. A NO limit order on a market like this is inherently risky, because if the button is pressed the offer will definitely be taken, but if the button is not pressed it probably won’t. If there had been then that could have added to the incentive some bettors had to blow up the home page, but as far as I can tell there weren’t.
The positions in the WHEN market were much smaller, with the largest positions coming in at 552 shares in LATE/NEVER and 325 shares in EARLY. The current design of range markets on Manifold is fun to play with, but is not very “swingy”. In particular, there is no option for outsized winnings if you can predict exactly when something will happen. If a user was planning to blow up the home page in the last few hours they could have bet a combination of LATE/NEVER in this market and YES in the binary market to maximize their gains, but I don’t see any evidence of this. Because there are fewer high karma users, large EARLY bets in this market could slightly limit their incentive to bet EARLY and blow up the home page early. Unlike the YES/NO market, those bets didn’t happen.
If someone was willing to anonymously blow up the home page, then a pattern of placing huge bets on that happening might look suspicious and damage their anonymity. So generally I would expect these incentives to only be effective for someone who was willing to blow up the home page and take credit for it. You can bet on that happening in Will anyone try to take credit for nuking LessWrong?. Or, place a limit NO order to incentivize someone to bet YES and then take the credit (and the mana).
When I created my market—Will LessWrong change their Petrov Day 2022 plan and reduce the chance of the button being pressed? - it was swiftly bet down to a low probability, creating a small incentive for the LessWrong admins to bet YES and then change their plans. Nobody took up that incentive, and the market didn’t attract large limit orders.
Finally mkualquiera made a market—Will my friend agree to defect on Petrov Day?. This market is evidence that two people were thinking about being incentivized to blow up the home page by betting markets. But it seems like this was more about a fun game than making mana. The largest position was 301 shares on YES, and the market resolved NO, so whatever incentive those shares provided, they weren’t enough to make it happen.
In the end I think there is a low (10%) chance that anyone’s behavior was significantly shifted by prediction market incentives. Feel free to reply to this post with how your behavior was shifted so I can update.
I was hopeful that people might shift their behavior based on the prediction market predictions—specifically that the high probability placed on the home page being blown up would lead to design changes. However, this retrospective clarifies that Petrov Day 2022 was a social experiment, so the prediction would have just shown it was expected to work as designed.
It’s also possible that people saw the high probability that the home page would be blown up and altered their plans. They may have not blown it up because they expected someone else would (especially if they weren’t confident in their anonymity being preserved). Or they may have blown it up because “if it’s going to happen, it may as well be me”. I think there is a higher chance (25%) that this was an effect, but I don’t know what the net direction would have been. Again, feel free to reply if you think the prediction markets helped you make decisions.
Overall I think the prediction markets were a positive addition to a negative event. I think the main incentive in play was the possibility of seeing, or not seeing, the LessWrong home page being blown up. And I think we all have a lot to figure out about prediction markets.
This is great! You should make a top-level post. Please!
To clarify, me and my friend were 100% going to press the button, but we were discouraged by the false alarm. There was no fun at that point, and it made me lose like 1⁄3 of my total mana. I had to close all my positions to stop the losses, and we went to sleep. When we woke up it was already too late for it to be noteworthy or fun.