[disclaimer: I have only been using the site seriously for around 5 months]
I was looking at the growth of predictionbook.com recently, and there has been a pretty stable addition of about 5 new public predictions per day since 2012 (that is counting only new predictions, not including additional wagers on existing predictions). I was curious why the site did not seem to be growing, and how little it is mentioned or linked to on lesswrong and related blogs.
(sidebar: Total predictions (based on the IDs of the public predictions) are growing at about double that rate although there was huge growth around 2015 (graph) that I assume was either a script generating automated predictions, or just testing by the devs maybe—does anyone know what caused this?)
Personally I find predictionbook to be very useful for
reducing hindsight bias
revealing planning fallacy
making me more objective, reducing effects of narrative fallacy
forcing me to think through questions more thoroughly by considering base rates, what the world would need to look like now for the prediction to come to pass, noticing composite predictions and considering each part individually, etc.
making me more aware of other people’s failure at prediction, or when they are careful to make hard to verify predictions.
making me more wary of post-hoc rationalization of events I would not have predicted
fun
Gwern covers many other benefits of making and tracking predictions here
I would expect predictionbook to be more popular, since I am not aware of any similar services, and I find predictions to be so useful. I was therefore wondering:
who on lesswrong tracks their predictions outside of predictionbook, and their thoughts on that method
who is not tracking their predictions at all, and why they made that decision
I use Metaculus a lot, and have made predictions on the /r/SpaceX subreddit which I need to go back and make a calibration graph for.
(They regularly bet donations of reddit gold, and have occasional prediction threads, like just before large SpaceX announcements. They would make an excellent target audience for better prediction tools.)
I’ve toyed with the idea of making a bot which searched for keywords on Reddit/LW, and tracked people’s predictions for them. However, since LW is moving away from the reddit code base, I’m not sure if building such a bot would make sense right now.
I don’t think that the dev’s touched predictionbook in quite a while. In general discovery of interesting public predictions doesn’t work well because it’s not easy to search and there are no tags.
I don’t think that the dev’s touched predictionbook in quite a while. In general discovery of interesting public predictions doesn’t work well because it’s not easy to search and there are no tags.
Yes, this is true. I believe they accept pull requests but there has not been much development for a long time and the site is quite basic.
Yes, this is true. I believe they accept pull requests but there has not been much development for a long time and the site is quite basic.
That’s might be true, but from my perspective, a few pull requests won’t change the fate of predictionbook significantly. It’s likely that another service will sooner or later overtake it.
I like the Android App at it’s current state but don’t know how much time it’s creator plans to invest in it in total. For personal prediction tracking the Android App is already more handy for me than predictionbook as I caryy my mobile phone around and it’s relatively straightforward to add new predictions when I have nothing else to do at the moment while traveling from place A to B.
Arbital might sooner or later build it’s own prediction tracking.
It’s also possible that someone starts a completely new project.
who on lesswrong tracks their predictions outside of predictionbook, and their thoughts on that method
Just adding to the other responses: I also use Metaculus and like it a lot. In another thread, I posted a rough note about its community’s calibration.
Compared to PredictionBook, the major limitation of Metaculus is that users cannot create and predict on arbitrary questions, because questions are curated. This is an inherent limitation/feature for a website like Metaculus because they want the community to focus on a set of questions of general interest. In Metaculus’s case, ‘general interest’ translates mostly to ‘science and technology’; for questions on politics, I suggest taking a look at GJ Open instead.
(some previous discussion of predictionbook.com here)
[disclaimer: I have only been using the site seriously for around 5 months]
I was looking at the growth of predictionbook.com recently, and there has been a pretty stable addition of about 5 new public predictions per day since 2012 (that is counting only new predictions, not including additional wagers on existing predictions). I was curious why the site did not seem to be growing, and how little it is mentioned or linked to on lesswrong and related blogs.
(sidebar: Total predictions (based on the IDs of the public predictions) are growing at about double that rate although there was huge growth around 2015 (graph) that I assume was either a script generating automated predictions, or just testing by the devs maybe—does anyone know what caused this?)
Personally I find predictionbook to be very useful for
reducing hindsight bias
revealing planning fallacy
making me more objective, reducing effects of narrative fallacy
forcing me to think through questions more thoroughly by considering base rates, what the world would need to look like now for the prediction to come to pass, noticing composite predictions and considering each part individually, etc.
making me more aware of other people’s failure at prediction, or when they are careful to make hard to verify predictions.
making me more wary of post-hoc rationalization of events I would not have predicted
fun
Gwern covers many other benefits of making and tracking predictions here
I would expect predictionbook to be more popular, since I am not aware of any similar services, and I find predictions to be so useful. I was therefore wondering:
who on lesswrong tracks their predictions outside of predictionbook, and their thoughts on that method
who is not tracking their predictions at all, and why they made that decision
I use Metaculus a lot, and have made predictions on the /r/SpaceX subreddit which I need to go back and make a calibration graph for.
(They regularly bet donations of reddit gold, and have occasional prediction threads, like just before large SpaceX announcements. They would make an excellent target audience for better prediction tools.)
I’ve toyed with the idea of making a bot which searched for keywords on Reddit/LW, and tracked people’s predictions for them. However, since LW is moving away from the reddit code base, I’m not sure if building such a bot would make sense right now.
I don’t think that the dev’s touched predictionbook in quite a while. In general discovery of interesting public predictions doesn’t work well because it’s not easy to search and there are no tags.
There’s https://www.metaculus.com and https://www.gjopen.com/ for curated public predictions. For private prediction there’s a new Android App (still very much in Beta and in development): https://play.google.com/store/apps/details?id=squirrelinhell.lwpredictions
I also did a bunch of prediction tracking in less structured ways. Our LW dojo had a while a shared Workflowy for predictions.
Yes, this is true. I believe they accept pull requests but there has not been much development for a long time and the site is quite basic.
I didn’t know about this one, thank you!
That’s might be true, but from my perspective, a few pull requests won’t change the fate of predictionbook significantly. It’s likely that another service will sooner or later overtake it.
I like the Android App at it’s current state but don’t know how much time it’s creator plans to invest in it in total. For personal prediction tracking the Android App is already more handy for me than predictionbook as I caryy my mobile phone around and it’s relatively straightforward to add new predictions when I have nothing else to do at the moment while traveling from place A to B.
Arbital might sooner or later build it’s own prediction tracking.
It’s also possible that someone starts a completely new project.
Just adding to the other responses: I also use Metaculus and like it a lot. In another thread, I posted a rough note about its community’s calibration.
Compared to PredictionBook, the major limitation of Metaculus is that users cannot create and predict on arbitrary questions, because questions are curated. This is an inherent limitation/feature for a website like Metaculus because they want the community to focus on a set of questions of general interest. In Metaculus’s case, ‘general interest’ translates mostly to ‘science and technology’; for questions on politics, I suggest taking a look at GJ Open instead.