Sites like stackexchange, r/askscience, quora, and lesswrong.com/questions all have a similar format. One person asks a question. Everyone gives their own answer. You can vote on answers to (hopefully) raise the best answer to the top. When the answer becomes out of date, if you’re lucky, the original author will update their comment.
There are some benefits with this approach.
Competition
Each answer competes with the other answers for top position. This competition does a decent job at sorting out the good from the bad answers.
Reward
These sites usually have some sort of karma system where you get cred for writing better answers. Maybe that reward system motivates people to answer questions more often? Seems at least possible.
Ease
The format is very easy to understand. If you have a question, ask it. If you know an answer to a question and want to help out, you give your answer. There is no complicated coordination required. If you’re reading through the answers and think one is best, upvote.
There are also some key problems with this approach.
Duplicate questions
All those sites have many duplicate questions. Even stackexchange only flags duplicates without removing them. Duplicate questions make searching for answers inefficient. They also make writing answers inefficient, as you may be covering the same ground someone else has on a duplicate question.
Duplicate answers
Even on a single question post, you’ll have many different answers. One per commenter. The asker of the question or (more importantly) the person who finds this post from google, is left to sort through the answers and make a judgement call on which one is best. That shouldn’t be the job of the person who writes or reads the question. Presumably they’re not experts on the topic… as they’re the ones who don’t know the answer.
There is the voting mechanism, true, but it’s not perfect. I often find the best answer on a stackoverflow post is the 5th one down, mostly because it’s more up to date. Which brings me to...
Stale questions and answers
Answerers often can update their answers over time, but most don’t (god bless the few who do). But it’s not the answerers’ fault for not keeping their comment up to date. They just tried to answer someone’s question, they didn’t sign on for maintaining their post for the rest of time. Because answers are very often stale, you can’t really trust any one answer too much, even assuming the top answer was very high quality when it was written.
Only one person can update an answer
This one may be harder to see as a problem. The problem of stale answers is due in large part because only one person can edit each answer. If we could all edit the top answer, stale answers would be more likely to be updated.
But there are other problems an answer might have besides being stale. It might be a bit strangely worded, too short or too long, inaccurate, or missing important details. Often people will comment suggested changes to answers. Sometimes the answerer updates their answer and sometimes not. If not, you may not see the crucial flaw in the answer unless you read all the comments. Often it would be better if people could just update the answer directly.
[note: Nathan Arthurpointed out in the comments that on stackexchange people can actually update other people’s answers]
The Question Wiki
To me a much more natural format for collective question answering is a wiki. The question would be the title of the page, and the body of the page would be the “answer”. Any duplicate questions would be merged. Any duplicate answers would be merged.
The page could more easily be kept up to date as anyone could update it. And anyone could improve it, even in very minor ways. Because there would only be one response to the question, that forces us to synthesize different people’s arguments, evidence, and opinions so the future reader doesn’t have to. Where views diverge, we’d lay out the different viewpoints and how they relate to each other.
Some subset of the questions would be open questions. In that case each page would track relevant sources, related questions, possible/proposed answers, possible experiments to decide between the answers etc. Basically the page would distill where the frontier of knowledge is on that question, and possible ways to push that frontier forward.
You could still have individual comments, just in the discussion page related to the question. I’m imagining this would be essentially the same setup as the polymath project, only completely general (that is, not limited to math questions).
I think there are some downsides to this format, mainly the costs of coordination. You’ll probably need more bureaucracy, more rules and norms. My guess is that it would be worth it though.
So I made this wiki: openquestions.wiki. The wiki is already useful, in the sense that I’ve been getting use out of it as a way to organize my own questions I’m interested in. Whether the thing gets much use from people besides me is yet to be seen. I give it about a 15% chance that more than a few hundred people will regularly use it.
Not everything is worked out yet. What questions should be allowed on the site? Should the site continue to exist or be merged with other similar projects (maybe Wikiversity)? What should the rules be? How do we know when to merge two question pages? These and many more questions will need to be answered. My idea is that those just become their own wiki pages to be answered collectively.
A Wiki for Questions
Q&A sites
Sites like stackexchange, r/askscience, quora, and lesswrong.com/questions all have a similar format. One person asks a question. Everyone gives their own answer. You can vote on answers to (hopefully) raise the best answer to the top. When the answer becomes out of date, if you’re lucky, the original author will update their comment.
There are some benefits with this approach.
Competition
Each answer competes with the other answers for top position. This competition does a decent job at sorting out the good from the bad answers.
Reward
These sites usually have some sort of karma system where you get cred for writing better answers. Maybe that reward system motivates people to answer questions more often? Seems at least possible.
Ease
The format is very easy to understand. If you have a question, ask it. If you know an answer to a question and want to help out, you give your answer. There is no complicated coordination required. If you’re reading through the answers and think one is best, upvote.
There are also some key problems with this approach.
Duplicate questions
All those sites have many duplicate questions. Even stackexchange only flags duplicates without removing them. Duplicate questions make searching for answers inefficient. They also make writing answers inefficient, as you may be covering the same ground someone else has on a duplicate question.
Duplicate answers
Even on a single question post, you’ll have many different answers. One per commenter. The asker of the question or (more importantly) the person who finds this post from google, is left to sort through the answers and make a judgement call on which one is best. That shouldn’t be the job of the person who writes or reads the question. Presumably they’re not experts on the topic… as they’re the ones who don’t know the answer.
There is the voting mechanism, true, but it’s not perfect. I often find the best answer on a stackoverflow post is the 5th one down, mostly because it’s more up to date. Which brings me to...
Stale questions and answers
Answerers often can update their answers over time, but most don’t (god bless the few who do). But it’s not the answerers’ fault for not keeping their comment up to date. They just tried to answer someone’s question, they didn’t sign on for maintaining their post for the rest of time. Because answers are very often stale, you can’t really trust any one answer too much, even assuming the top answer was very high quality when it was written.
Only one person can update an answer
This one may be harder to see as a problem. The problem of stale answers is due in large part because only one person can edit each answer. If we could all edit the top answer, stale answers would be more likely to be updated.
But there are other problems an answer might have besides being stale. It might be a bit strangely worded, too short or too long, inaccurate, or missing important details. Often people will comment suggested changes to answers. Sometimes the answerer updates their answer and sometimes not. If not, you may not see the crucial flaw in the answer unless you read all the comments. Often it would be better if people could just update the answer directly.
[note: Nathan Arthur pointed out in the comments that on stackexchange people can actually update other people’s answers]
The Question Wiki
To me a much more natural format for collective question answering is a wiki. The question would be the title of the page, and the body of the page would be the “answer”. Any duplicate questions would be merged. Any duplicate answers would be merged.
The page could more easily be kept up to date as anyone could update it. And anyone could improve it, even in very minor ways. Because there would only be one response to the question, that forces us to synthesize different people’s arguments, evidence, and opinions so the future reader doesn’t have to. Where views diverge, we’d lay out the different viewpoints and how they relate to each other.
Some subset of the questions would be open questions. In that case each page would track relevant sources, related questions, possible/proposed answers, possible experiments to decide between the answers etc. Basically the page would distill where the frontier of knowledge is on that question, and possible ways to push that frontier forward.
You could still have individual comments, just in the discussion page related to the question. I’m imagining this would be essentially the same setup as the polymath project, only completely general (that is, not limited to math questions).
I think there are some downsides to this format, mainly the costs of coordination. You’ll probably need more bureaucracy, more rules and norms. My guess is that it would be worth it though.
So I made this wiki: openquestions.wiki. The wiki is already useful, in the sense that I’ve been getting use out of it as a way to organize my own questions I’m interested in. Whether the thing gets much use from people besides me is yet to be seen. I give it about a 15% chance that more than a few hundred people will regularly use it.
Not everything is worked out yet. What questions should be allowed on the site? Should the site continue to exist or be merged with other similar projects (maybe Wikiversity)? What should the rules be? How do we know when to merge two question pages? These and many more questions will need to be answered. My idea is that those just become their own wiki pages to be answered collectively.