Are they planning on adding features relevant to a prediction market (apart from betting money)? E.g., tracking better reputation/score based on success or transitive trust; or tracking the overall predicted value of a prediction with many bets, weighed by the success/reputation/… of the betters.
Official answer: Eliezer’s right. If we see traffic growing we’ll invest in further development.
We can think of many things we could do to make the site better… but those users who currently use it don’t use it enough, and if they tell their friends about it their friends don’t become regular users (often enough).
Hosting the current code is very cheap and easy, so the site’s in little danger of being shut down, but we won’t be developing it further unless you guys and gals (and your friends, and their friends) pile on the love.
Just out of curiosity, are you a startup, a non profit or a guy doing a side project?
I predict the site’s userbase will not explode overnight but will escalate in the shape of a hockey stick. That’s how these things usually happen. You will have to keep improving it even while the userbase is still low, otherwise people will think the site is dying and they will stop showing up. Interesting things need to already be happening on the site before a larger audience will keep coming back to it, not vice versa.
Also, you need to add documentation no matter how simple and intuitive you think the sites features are. They don’t seem as intuitive from the outside. By ‘documentation’ I mean a short and EXPLICIT description of what each feature does. I like the ‘help’ button near the timeframe for the prediction. You could add help buttons next to everything. Also a faq would be nice.
Overall I think the site has great potential. Keep up the good work.
Just out of curiosity, are you a startup, a non profit or a guy doing a side project?
We’re Investling, which is a handfull of startups and an IT consultancy. We’re for-profit, with some non-profit projects on the side (in part because we’ll make more profits if we can help save the world from surprise conversion to paperclips). The majority of our non-profit work is SIAI related.
I predict the site’s userbase will not explode overnight but will escalate in the shape of a hockey stick. […]
Some projects follow that pattern. Some projects never hockey-stick. How can you tell which curve you’re riding?
We have many projects running: some have maintained exponential growth since we became involved; some are too young to judge; and some are on the low end of a curve that may be a hockey stick and may just be a project that doesn’t have any legs. I very much hope that the LW crowd will latch on to PBook (keep coming back, tell your friends, etc.). If you do (we do—several of us are very keen LWers) and we see traffic growing, we’ll flood more resources into the project. If it languishes we’ll continue to host it and may even open source it, but it seems more sensible to flood our resources into projects that are winning. I really don’t want to see PBook die, but I’m trying to count warm fuzzies consciously.
Also, you need to add documentation […]
We know the documentation is sparse (or, more precisely, the user interface isn’t intuitive—documentation is evidence of a UI failure and good design is self-documenting). If you guys are still around in 14 days we should talk about more dev resources.
Agreed on the UI being incredibly confusing (and slow).
In terms of usability, if they just moved the judgment buttons down below, added text like “Render final judgment on this prediction” to make it obvious what judgment does, and changed “My 2 cents” to “Submit Estimate” or something like that, it would be a huge improvement over the current. These sorts of very minor cosmetic UI changes would be trivial to make.
I just signed up and did a bunch of predictions. Here are my initial impressions:
The majority of our non-profit work is SIAI related.
A tool like PB is like spaced repetition flash card programs or writing Wikipedia articles—a long-term tool. Some benefits appear quickly, yes, but the bulk of the benefits arrive over years or decades. (PB is somewhat like Long Bets.)
As the saying goes, “In the long run, the utility of all non-Free software approaches zero. All non-Free software is a dead end.” If I invest time in PB, what guarantee do I have that I will be able to get my data out of PB when* it dies, especially for topics I didn’t write? Are you guys going to license the content under a CC license?
(You should do it early, while there still isn’t too much content—once Wikipedia got large, it took years and years and a unique one-time exemption by the FSF to liberate its content from the GFDL into a CC license.)
* And it will die eventually. Every site either dies or evolves out of recognition.
** My data is vastly more important to me than the website software. If I had to, I could run a personal PB in just a flat text file, after all.
2) comments are ridiculously constrained. I dunno if you guys were trying for some sort of auto-Twitter compatibility, but it’s really annoying. If you need to dump comments on Twitter and they’re too long, then just truncate them.
3) I just judged a Michael Jackson-related prediction wrong, with a citation that the predicted event happened in the wrong year. But in the history section, my comment never appeared! My current workaround is to make a 0 or 50% prediction (wrong/right), explain my reasoning as best as I can in so short a space, and then separately mark it wrong/right. This is unfair to my score, since obviously I can choose 0 or 100% and always be right.
4) The black boxes on prediction pages (eg. “Join this prediction”) are horrible. I was convinced for the longest time that they were buttons to push, and that they were disabled by some JavaScript pokery until I went and read the page source.
5) Newlines in comments do not get translated to a space or two in the comment; they get translated to nothing whatsoever.
6) No apparent way to edit ‘due dates’ for predictions; many unjudged predictions can’t be judged at all because they seem to have been created expired.
7) On userpages, the most recent prediction/action gets split in half by the statistics graph in my Firefox; screenshot.
8) Years get interpreted badly. ’2029′ becomes—somehow − 2 hours from right now, as opposed to 19 years. screenshot
9) The in-browser JS date checker seems to be quite inaccurate. I’ve been entering all my dates as ‘1 January 2024’ and the like, which it has never validated—but which turn into the right date when actually submitted.
10) The site is slow. And it seems to be on the server itself. I’m the only user right now, and yet predictions can take as much as 10 seconds to enter. I don’t understand how it can be so slow, given that a prediction is a 4-tuple of (date,prediction,owner, user-confidence) which probably adds up to less than a kilobyte of data.
It’s our intention to Open Source the PredictionBook code… and has been for at least six months, but we keep not quite getting around to it. It’s also my intention to write a top level post about why I think PBook isn’t getting much traffic (it being slow is only one reason).
Any one with a reputation on this site that wants access to the code before we get it open sourced is welcome to contact me directly. The code’s on github.com and is written in Ruby on Rails.
(gwern, if you want access send me your promise that you’ll behave responsibly and your github username.)
It’s also my intention to write a top level post about why I think PBook isn’t getting much traffic (it being slow is only one reason).
I have my own theories (mostly that people aren’t very interested in truth-seeking, pace Hanson, and that the benefits are too long-term, cf. SRS flashcards), but that’s just my perspective as a user.
gwern, if you want access send me your promise that you’ll behave responsibly and your github username.
Do you mean access to the data? As I said, I’d like to edit the dates on some of the predictions...
No. People have private predictions in there, so I don’t think I can in clear conscience give you access to anyone’s predictions but your own (and giving you access to only your own is about half as much work as properly open sourcing the project). I mean the code… and you didn’t quite send me your promise that you’ll behave responsibly yet.
Well, alright. I see I didn’t specifically say ‘public data’. That’s what I want.
and you didn’t quite send me your promise that you’ll behave responsibly yet.
I think it’s kind of silly to ask for such a promise, but for what it’s worth, you have it. (What irresponsible things could I do with just the codebase? I’m no cracker to find security holes and exploit them on the live site.)
This is quite interesting & exciting.
Are they planning on adding features relevant to a prediction market (apart from betting money)? E.g., tracking better reputation/score based on success or transitive trust; or tracking the overall predicted value of a prediction with many bets, weighed by the success/reputation/… of the betters.
Whether they add features will depend on whether people seem interested in using it, they say.
Official answer: Eliezer’s right. If we see traffic growing we’ll invest in further development.
We can think of many things we could do to make the site better… but those users who currently use it don’t use it enough, and if they tell their friends about it their friends don’t become regular users (often enough).
Hosting the current code is very cheap and easy, so the site’s in little danger of being shut down, but we won’t be developing it further unless you guys and gals (and your friends, and their friends) pile on the love.
Just out of curiosity, are you a startup, a non profit or a guy doing a side project?
I predict the site’s userbase will not explode overnight but will escalate in the shape of a hockey stick. That’s how these things usually happen. You will have to keep improving it even while the userbase is still low, otherwise people will think the site is dying and they will stop showing up. Interesting things need to already be happening on the site before a larger audience will keep coming back to it, not vice versa.
Also, you need to add documentation no matter how simple and intuitive you think the sites features are. They don’t seem as intuitive from the outside. By ‘documentation’ I mean a short and EXPLICIT description of what each feature does. I like the ‘help’ button near the timeframe for the prediction. You could add help buttons next to everything. Also a faq would be nice.
Overall I think the site has great potential. Keep up the good work.
We’re Investling, which is a handfull of startups and an IT consultancy. We’re for-profit, with some non-profit projects on the side (in part because we’ll make more profits if we can help save the world from surprise conversion to paperclips). The majority of our non-profit work is SIAI related.
Some projects follow that pattern. Some projects never hockey-stick. How can you tell which curve you’re riding?
We have many projects running: some have maintained exponential growth since we became involved; some are too young to judge; and some are on the low end of a curve that may be a hockey stick and may just be a project that doesn’t have any legs. I very much hope that the LW crowd will latch on to PBook (keep coming back, tell your friends, etc.). If you do (we do—several of us are very keen LWers) and we see traffic growing, we’ll flood more resources into the project. If it languishes we’ll continue to host it and may even open source it, but it seems more sensible to flood our resources into projects that are winning. I really don’t want to see PBook die, but I’m trying to count warm fuzzies consciously.
We know the documentation is sparse (or, more precisely, the user interface isn’t intuitive—documentation is evidence of a UI failure and good design is self-documenting). If you guys are still around in 14 days we should talk about more dev resources.
Yes yes yes. Four times yes.
Right now the UI is so slow / bad that I couldn’t see myself using it.
Agreed on the UI being incredibly confusing (and slow).
In terms of usability, if they just moved the judgment buttons down below, added text like “Render final judgment on this prediction” to make it obvious what judgment does, and changed “My 2 cents” to “Submit Estimate” or something like that, it would be a huge improvement over the current. These sorts of very minor cosmetic UI changes would be trivial to make.
I just signed up and did a bunch of predictions. Here are my initial impressions:
A tool like PB is like spaced repetition flash card programs or writing Wikipedia articles—a long-term tool. Some benefits appear quickly, yes, but the bulk of the benefits arrive over years or decades. (PB is somewhat like Long Bets.)
As the saying goes, “In the long run, the utility of all non-Free software approaches zero. All non-Free software is a dead end.” If I invest time in PB, what guarantee do I have that I will be able to get my data out of PB when* it dies, especially for topics I didn’t write? Are you guys going to license the content under a CC license?
(You should do it early, while there still isn’t too much content—once Wikipedia got large, it took years and years and a unique one-time exemption by the FSF to liberate its content from the GFDL into a CC license.)
* And it will die eventually. Every site either dies or evolves out of recognition.
** My data is vastly more important to me than the website software. If I had to, I could run a personal PB in just a flat text file, after all.
2) comments are ridiculously constrained. I dunno if you guys were trying for some sort of auto-Twitter compatibility, but it’s really annoying. If you need to dump comments on Twitter and they’re too long, then just truncate them.
3) I just judged a Michael Jackson-related prediction wrong, with a citation that the predicted event happened in the wrong year. But in the history section, my comment never appeared!
My current workaround is to make a 0 or 50% prediction (wrong/right), explain my reasoning as best as I can in so short a space, and then separately mark it wrong/right. This is unfair to my score, since obviously I can choose 0 or 100% and always be right.
4) The black boxes on prediction pages (eg. “Join this prediction”) are horrible. I was convinced for the longest time that they were buttons to push, and that they were disabled by some JavaScript pokery until I went and read the page source.
5) Newlines in comments do not get translated to a space or two in the comment; they get translated to nothing whatsoever.
6) No apparent way to edit ‘due dates’ for predictions; many unjudged predictions can’t be judged at all because they seem to have been created expired.
7) On userpages, the most recent prediction/action gets split in half by the statistics graph in my Firefox; screenshot.
8) Years get interpreted badly. ’2029′ becomes—somehow − 2 hours from right now, as opposed to 19 years. screenshot
9) The in-browser JS date checker seems to be quite inaccurate. I’ve been entering all my dates as ‘1 January 2024’ and the like, which it has never validated—but which turn into the right date when actually submitted.
10) The site is slow. And it seems to be on the server itself. I’m the only user right now, and yet predictions can take as much as 10 seconds to enter. I don’t understand how it can be so slow, given that a prediction is a 4-tuple of (date,prediction,owner, user-confidence) which probably adds up to less than a kilobyte of data.
It’s our intention to Open Source the PredictionBook code… and has been for at least six months, but we keep not quite getting around to it. It’s also my intention to write a top level post about why I think PBook isn’t getting much traffic (it being slow is only one reason).
Any one with a reputation on this site that wants access to the code before we get it open sourced is welcome to contact me directly. The code’s on github.com and is written in Ruby on Rails.
(gwern, if you want access send me your promise that you’ll behave responsibly and your github username.)
I know the feeling.
I have my own theories (mostly that people aren’t very interested in truth-seeking, pace Hanson, and that the benefits are too long-term, cf. SRS flashcards), but that’s just my perspective as a user.
Do you mean access to the data? As I said, I’d like to edit the dates on some of the predictions...
I’ve signed up at http://github.com/gwern
No. People have private predictions in there, so I don’t think I can in clear conscience give you access to anyone’s predictions but your own (and giving you access to only your own is about half as much work as properly open sourcing the project). I mean the code… and you didn’t quite send me your promise that you’ll behave responsibly yet.
Well, alright. I see I didn’t specifically say ‘public data’. That’s what I want.
I think it’s kind of silly to ask for such a promise, but for what it’s worth, you have it. (What irresponsible things could I do with just the codebase? I’m no cracker to find security holes and exploit them on the live site.)