Once things have stabilized and things like inline annotations are there, I’d love to see the following: (1) An easy way to add and remove yourself from a pool of available feedback providers. (Checkbox in settings?) And (2) a way for anyone (or nearly anyone—e.g. non-negative karma) to request brief / “basic feedback” on their posts, by automatically matching people from the pool to posts based on e.g. post tags and front page tag weights.
On (1): I have proofread a couple thousand pages by now, and while I’m usually pretty busy, in a slow week I’d be happy to proof-read a bunch of drafts. However, if that involves messaging someone to be added to a list and then messaging someone again to be taken off that list, that’s quite a lot of overhead—I probably wouldn’t bother and just look for other stuff to do. So I suspect automating that part might greatly increase the amount of available reviewers for feedback.
On (2): With some extra capacity from (1), I expect the main bottleneck for providing more reviews is matching reviewers to posts. If that’s automated, the only cost is the time spent reviewing. With a scheme like “review two/three posts to get one post reviewed by two/three people”, the reviewer pool should be even bigger (so with (1) you might even go “review 2 to get 3″) and things should remain relatively fair. With multiple reviews, you should get some decent feedback even if one reviewer writes complete nonsense or doesn’t understand anything.
At that point, the human overhead for having this extra “basic feedback” system should be near-zero, apart from from maybe having to manually filter people trying to abuse the system—no clue how prevalent that is. And looking at myself, (a) I probably wouldn’t bother manually asking others for reviews, and (b) knowing that I can get guaranteed feedback, no questions asked, would make it more likely to actually start writing. (While I can’t say for sure whether that translates into actual posts, I can clearly see that there are lots of other “very important” things, some of which only barely win out because they’re less headache-inducing / uncertain.)
Once things have stabilized and things like inline annotations are there, I’d love to see the following: (1) An easy way to add and remove yourself from a pool of available feedback providers. (Checkbox in settings?) And (2) a way for anyone (or nearly anyone—e.g. non-negative karma) to request brief / “basic feedback” on their posts, by automatically matching people from the pool to posts based on e.g. post tags and front page tag weights.
On (1): I have proofread a couple thousand pages by now, and while I’m usually pretty busy, in a slow week I’d be happy to proof-read a bunch of drafts. However, if that involves messaging someone to be added to a list and then messaging someone again to be taken off that list, that’s quite a lot of overhead—I probably wouldn’t bother and just look for other stuff to do. So I suspect automating that part might greatly increase the amount of available reviewers for feedback.
On (2): With some extra capacity from (1), I expect the main bottleneck for providing more reviews is matching reviewers to posts. If that’s automated, the only cost is the time spent reviewing. With a scheme like “review two/three posts to get one post reviewed by two/three people”, the reviewer pool should be even bigger (so with (1) you might even go “review 2 to get 3″) and things should remain relatively fair. With multiple reviews, you should get some decent feedback even if one reviewer writes complete nonsense or doesn’t understand anything.
At that point, the human overhead for having this extra “basic feedback” system should be near-zero, apart from from maybe having to manually filter people trying to abuse the system—no clue how prevalent that is. And looking at myself, (a) I probably wouldn’t bother manually asking others for reviews, and (b) knowing that I can get guaranteed feedback, no questions asked, would make it more likely to actually start writing. (While I can’t say for sure whether that translates into actual posts, I can clearly see that there are lots of other “very important” things, some of which only barely win out because they’re less headache-inducing / uncertain.)
I plan to attempt making inline annotations happen soon.
The rest of what you describe sounds very cool. Maybe! If there’s the demand and supply for it, we could probably build it.