This touches directly on work I’m doing. Here is my burning question: Could an open-source optimization algorithm be workable?
I’m thinking of a wikipedia-like system for open-edit regulation of the optimization factors, weights, etc. Could full direct democratization of the attention economy be the solution to the arms race problem?
Jim Whales (the guy who started Wikipedia) tried that. He couldn’t get enough users to justify it.
I don’t see much of an advantage to have it open source, and it allows people to actually see the algorithm when they’re taking advantage of it. It might even be possible to change the algorithm to help them.
Neither of those is large enough scale for people to try to take advantage of the algorithm.
Come to think of it, in both cases, people could still use it for learning the algorithms in general. Knowing how Wikia Search worked would teach you a thing or two about search engine optimization, and knowing the specific vulnerabilities ClamAV protects against can tell you what you can take advantage of. It would be impossible to trace either of these effects back to the source, so we can’t be sure it hasn’t happened.
knowing the specific vulnerabilities ClamAV protects against can tell you what you can take advantage of.
In this vein, you can do even better with binary updates for vulnerabilities (such as Windows Update). They can be automatically rewritten into exploits: “Automatic Patch-Based Exploit Generation” (LtU).
You may be a dreamer, but so am I. Perhaps we should talk. :)
As it happens, I do have in mind a desgin of a distributed, open source approach that should circumvent this problem, at least in the area of social news. I am not sure however if the Less Wrong crowd would find it relevant for me to discuss that in an article.
I’d love to discuss my concept. It’s inspired in no small part by what I learned from LessWrong, and by my UI designer’s lens. I don’t have the karma points to post about it yet, but in a nutshell it’s about distributing social, preference and history data, but also distributing the processing of aggregates, cross-preferencing, folksonomy, and social clustering.
The grand scheme is to repurpose every web paradigm that has improved semantic and behavioral optimization, but distribute out the evil centralization in each of them. I’m thinking of an architecture akin to FreeNet, with randomized redundancies and cross-checking, to circumvent individual nodes from gaming the ruleset.
But we do crowd-source the ruleset, and distribute its governance as well. Using a system not unlike LW’s karma (but probably a bit more complex), we weigh individual users’ “influence.” The factors on which articles, comments, and users can be rated is one of the tough questions I’m struggling with. I firmly believe that given a usable yet potentially deep and wide range of evaluation factors, many people will bother to offer nuanced ratings and opinions… Especially if the effort is rewarded by growth in their own “influence”.
So, through cross-influencing, we recreate online the networks of reputation and influence that exist in the real social world… but with less friction, and based more on your words and deeds than in institutional, authority, and character bias.
I’m hoping this has the potential to encourage more of a meritocracy of ideas. Although to be honest, I envision a system that can be used to filter the internet any way you want. You can decide to view only the most influential ideas from people who think like you, or from people who agree with Rush Limbaugh, or from people who believe in the rapture… and you will see that. You can find the most influential cute kitty video among cute kitty experts.
That’s the grand vision in a nutshell, and it’s incredibly ambitious of course, yet I’m thinking of bootstrapping it as an agile startup, eventually open-sourcing it all and providing a hosted free service as an alternative to running a client node. If I can find an honest and non-predatory way to cover my living expenses out of it, it would be nice, but that’s definitely not the primary concern.
I’m looking for partners to build a tool, but also for advisors to help set the right value-optimizing architecture… “seed” value-adding behavior into the interface, as it were. I hope I can get some help from the LessWrong community. If this works, it could end up being a pretty influential bit of technology! I’d like it to be a net positive for humanity in the long term.
Good point! I assume we’ll have decay built into the system, based on age of the data points… some form of that is built into the architecture of FreeNet I believe, where less-accessed content eventually drops out from the network altogether.
I wasn’t even thinking about old people… I was more thinking about letting errors of youth not follow you around for your whole life… but at the same time, valuable content (that which is still attracting new readers who mark it as valuable) doesn’t disappear.
That said, longevity on the system means you’ve had more time to contribute… But if your contributions are generally rated as crappy, time isn’t going to help your influence without a significant ongoing improvement to your contributions’ quality.
But if you’re a cranky old nutjob, and there are people out there who like what you say, you can become influential in the nutjob community, if at the expense of your influence in other circles. You can be considered a leading light by a small group of people, but an idiot by the world at large.
I’m still not quite getting how this is going to work.
Lets say I am a spam blog bot. What it does is take popular (for a niche) articles and reposts automated summaries. So lets say it does this for cars. These aren’t very good, but aren’t very bad either. Perhaps it makes automatic word changes to real peoples summaries. It gets lots of other spam bots of this type and they form self-supportive networks (each up voting each other) and also liking popular things to do with cars. People come across these links and up vote them, because they go somewhere interesting. They gain lots of karma in these communities and then start pimping car related products or spreading FUD about rival companies. Automated astro-turf if you want.
Does anyone regulate the creation of new users?
How long before they stop being interesting to the car people? Or how much effort would it be to track them down and remove them from the circle of people you are interested in.
Also who keeps track of these votes? Can people ballot stuff?
I’ve thought a long these lines before and realised it is a non-trivial problem.
Authentication and identity are an interesting issue. My concept is to allow anonymous users, with a very low initial influence level. But there would be many ways for users to strengthen their “identity score” (credit card verification, address verification via snail-mailed verif code, etc.), which would greatly and rapidly increase their influence score. A username that is tied to a specific person, and therefore wields much more influence, could undo the efforts of 100 bots with a single downvote.
But if you want to stay anonymous, you can. You’ll just have to patiently work on earning the same level of trust that is awarded to people who put their real-life reputation on the line.
I’m also conceiving of a richly semantic system, where simply “upvoting” or facebook-liking are the least influential actions one can take. Up from there, you can rate content on many factors, comment on it, review it, tag it, share it, reference it, relate it to other content. The more editorial and cerebral actions would probably do more to change one’s influence than a simple thumbs up. If a bot can compete with a human in writing content that gets rated high on “useful”, “factual”, “verifiable”, “unbiased”, AND “original” (by people who have high influence score in these categories), then I think the bot deserves a good influence score, because it’s a benevolent AI. ;)
Another concept, which would reduce incentives to game the system, is vouching. You can vouch for other users’ identity, integrity, maturity, etc. If you vouched for a bot, and the bot’s influence gets downgraded by the community, your influence will take a hit as well.
I see this happening throughout the system: Every time you exert your influence, you take responsibility for that action, as anyone may now rate/review/downvote your action. If you stand behind your judgement of Rush Limbaugh as truthful, enough people will disagree with you that from that point on, anytime you rate something as “truthful”, that rating will count for very little.
Hi avalot, thank you for the detailed discussion. I suspect the system I have in mind is simpler but should satisfy the same principles. In fact it has been eerie reading your post, as on principle we are in 95% agreement, to excruciation detail, and to a large extent on technical behaviour. I guess my one explicit difference is that I cannot let go of the profit motive. If I make a substantial contribution, I would like to be properly rewarded, if only to be able to materialize other ideas and contribute to causes I find worthy. That of course does not imply going to facebook’s lengths to squeeze the last drop of value out of its system, nor should it take precedence over openness and distribution. But to the extent that it can fit, I would like it to be there. Two questions for you:
Second, assuming the number of connections increase exponentially with a linear increase in users, the processing load will also rise much quicker than the number of users. How will a system like this operate at web-scale?
Not surprised that we’re thinking along the same lines, if we both read this blog! ;)
I love your questions. Let’s do this:
Keynesian Beauty Contest: I don’t have a silver bullet for it, but a lot of mitigation tactics. First of all, I envision offering a cascading set of progressively more fine-grained rating attributes, so that, while you can still upvote or downvote, or rate something with starts, you can also rate it on truthfulness, entertainment value, fairness, rationality (and countless other attributes)… More nuanced ratings would probably carry more influence (again, subject to others’ cross-rating). Therefore, to gain the highest levels of influence, you’d need to be nuanced in your ratings of content… gaming the system with nuanced, detailed opinions might be effectively the same as providing value to the system. I don’t mind someone trying to figure out the general population’s nuanced preferences… that’s actually a valuable service!
Secondly, your ratings are also cross-related to the semantic metadata (folksonomy of tags) of the content, so that your influence is limited to the topic at hand. Gaining a high influence score as a fashion celebrity doesn’t put your political or scientific opinions at the top of search results. Hopefully, this works as a sort of structural Palin-filter. ;)
The third mitigation has to do with your second question: How do we handle the processing of millions of real-time preference data points, when all of them should (in theory) get cross-related to all others, with (theoretically) endless recursion?
The typical web-based service approach of centralized crunching doesn’t make sense. I’m envisioning a distributed system where each influence node talks with a few others (a dozen?), and does some cross-processing with a them to agree on some temporary local normals, means and averages. That cluster does some more higher-level processing in consort with other close-by clusters, and they negotiate some “regional” aggregates… that gets propagated back down into the local level, and up to the next level of abstraction… up until you reach some set of a dozen superclusters that span the globe, and who trade in high-level aggregates.
All that is regulated, in terms of clock ticks, by activity: Content that is being rated/shared/commented on by many people will be accessed and cached by more local nodes, and processed by more clusters, and its cross-processing will be accelerated because it’s “hot”. Whereas one little opinion on one obscure item might not get processed by servers on the other side of the world until someone there requests it. We also decay data this way: If nobody cares, the system eventually forgets. (Your personal node will remember your preferences, but the network, after having consumed their influence effects, might forget their data points.)
A distributed, propagation system, batch-processed, not real-time, not atomic but aggregated. That means you can’t go back and change old ratings, and individual data points, because they get consumed by the aggregates. That means you can’t inspect what made your scored go up and down at the atomic level. That means your score isn’t the same everywhere on the planet at the same time. So gaming the system is harder because there’s no real-time feedback loop, there’s no single source of absolute truth (truth is local and propagates lazily), and there’s no auditing trail of the individual effects of your influence.
All of this hopefully makes the system so fluid that it holds innumerable beauty contests, always ongoing, always local, and the results are different depending on when and where you are. Hopefully this makes the search for the Nash equilibrium a futile exercise, and people give up and just say what they actually think is valuable to others, as opposed to just expected by others.
That’s my wishful thinking at the point. Am I fooling myself?
I’d create a simplified evolutionary model of the system using a GA to create the agents. If groups can find a way to game your system to create infinite interesting-ness/insightful-ness for specific topics, that then you need to change it.
You’re right: A system like that could be genetically evolved for optimization.
On the other hand, I was hoping to create an open optimization algorithm, governable by the community at large… based on their influence scores in the field of “online influence governance.” So the community would have to notice abuse and gaming of the system, and modify policy (as expressed in the algorithm, in the network rules, in laws and regulations and in social mores) to respond to it. Kind of like democracy: Make a good set of rules for collaborative rule-making, give it to the people, and hope they don’t break it.
But of course the Huns could take over. I’m trusting us to protect ourselves. In some way this would be poetic justice: If crowds can’t be wise, even when given a chance to select and filter among the members for wisdom, then I’ll give up on bootstrapping humanity and wait patiently for the singularity. Until then, though, I’d like to see how far we could go if given a useful tool for collaboration, and left to our own devices.
I think you are closer to a strong solution than you realize. You have mentioned the pieces but I think you haven’t put them together yet. In short, the solution I see is to depend on local (individual) decisions rather than group ones. If each node has its own ranking algorithm and its own set of trust relations, there is no reason to create complex group-cooperation mechanisms. A user that spams gets negative feedback and therefore eventually gets isolated in the graph. Even if automated users outnumber real users, the best they can do is vote each other up and therefore end up with their own cluster of the network, with real users only strongly connected to each other. Of course, if a bot provides value, it can be incorporated in that graph. “sufficiently advanced spam...”, etc. etc. This also means that the graph splinters into various clusters depending on worldview. (your rush limbaugh example). This deals with keynesian beauty contests as there is no ‘average’ to aim at. Your values simply cluster you with people who share them. If you value quality, you go closer to quality. If you value ‘republican-ness’ you move closer to that. The price you pay is that there is no ‘objective’ view of the system. There is no ‘top 10 articles’, only ‘top 10 articles for user X’.
Another thing I see with your design is that it is complex and attempts to boil at least a few oceans. (emergent ontologies/folksonomies for one, distributing identity, storage, etc.). I have some experience with defining complex architectures for distributed systems (e.g. http://arxiv.org/abs/0907.2485 ) and the problem is that they need years of work by many people to reach some theoretical purity, and even then bootstrapping will be a bitch. The system I have in mind is extremely simple by comparison, definitely more pragmatic (and therefore makes compromises) and is based on established web technologies. As a result, it should bootstrap itself quite easily. I find myself not wanting to publicly share the full details until I can start working on the thing (I am currently writing up my PhD thesis and my deadline is Oct. 1. After that, I’m focusing on this project). If you want to talk more details, we should probably take this to a private discussion.
You are right: This needs to be a fully decentralized system, with no center, and processing happening at the nodes. I was conceiving of “regional” aggregates mostly as a guess as to what may relieve network congestion if every node calls out to thousands of others.
Thank you for setting me right: My thinking has been so influenced by over a decade of web app dev that I’m still working on integrating the full principles of decentralized systems.
As for boiling oceans… I wish you were wrong, but you probably are right. Some of these architectures are likely to be enormously hard to fine-tune for effectiveness. At the same time, I am also hoping to piggyback on existing standards and systems.
I am not sure however if the Less Wrong crowd would find it relevant for me to discuss that in an article.
I think it is relevant because better social-news sites on the web would lead to better conversations about advancing the art of human rationality, which is the core mission of Less Wrong.
Announce 2 iterated prize sequences. The big Red Prizes for the best optimization algorithm and the small Blue prizes for the best spam which can spoof the same. Don’t award a blue until the first red is awarded and then don’t award a red until the last blue one is awarded and so on. Keep escalating the price amounts until satisfactory performance is attained.
This touches directly on work I’m doing. Here is my burning question: Could an open-source optimization algorithm be workable?
I’m thinking of a wikipedia-like system for open-edit regulation of the optimization factors, weights, etc. Could full direct democratization of the attention economy be the solution to the arms race problem?
Or am I, as usual, a naive dreamer?
Jim Whales (the guy who started Wikipedia) tried that. He couldn’t get enough users to justify it.
I don’t see much of an advantage to have it open source, and it allows people to actually see the algorithm when they’re taking advantage of it. It might even be possible to change the algorithm to help them.
I would have thought the same of open source antivirus, but ClamAV is as good as any proprietary AV.
Neither of those is large enough scale for people to try to take advantage of the algorithm.
Come to think of it, in both cases, people could still use it for learning the algorithms in general. Knowing how Wikia Search worked would teach you a thing or two about search engine optimization, and knowing the specific vulnerabilities ClamAV protects against can tell you what you can take advantage of. It would be impossible to trace either of these effects back to the source, so we can’t be sure it hasn’t happened.
In this vein, you can do even better with binary updates for vulnerabilities (such as Windows Update). They can be automatically rewritten into exploits: “Automatic Patch-Based Exploit Generation” (LtU).
You may be a dreamer, but so am I. Perhaps we should talk. :)
As it happens, I do have in mind a desgin of a distributed, open source approach that should circumvent this problem, at least in the area of social news. I am not sure however if the Less Wrong crowd would find it relevant for me to discuss that in an article.
I’d love to discuss my concept. It’s inspired in no small part by what I learned from LessWrong, and by my UI designer’s lens. I don’t have the karma points to post about it yet, but in a nutshell it’s about distributing social, preference and history data, but also distributing the processing of aggregates, cross-preferencing, folksonomy, and social clustering.
The grand scheme is to repurpose every web paradigm that has improved semantic and behavioral optimization, but distribute out the evil centralization in each of them. I’m thinking of an architecture akin to FreeNet, with randomized redundancies and cross-checking, to circumvent individual nodes from gaming the ruleset.
But we do crowd-source the ruleset, and distribute its governance as well. Using a system not unlike LW’s karma (but probably a bit more complex), we weigh individual users’ “influence.” The factors on which articles, comments, and users can be rated is one of the tough questions I’m struggling with. I firmly believe that given a usable yet potentially deep and wide range of evaluation factors, many people will bother to offer nuanced ratings and opinions… Especially if the effort is rewarded by growth in their own “influence”.
So, through cross-influencing, we recreate online the networks of reputation and influence that exist in the real social world… but with less friction, and based more on your words and deeds than in institutional, authority, and character bias.
I’m hoping this has the potential to encourage more of a meritocracy of ideas. Although to be honest, I envision a system that can be used to filter the internet any way you want. You can decide to view only the most influential ideas from people who think like you, or from people who agree with Rush Limbaugh, or from people who believe in the rapture… and you will see that. You can find the most influential cute kitty video among cute kitty experts.
That’s the grand vision in a nutshell, and it’s incredibly ambitious of course, yet I’m thinking of bootstrapping it as an agile startup, eventually open-sourcing it all and providing a hosted free service as an alternative to running a client node. If I can find an honest and non-predatory way to cover my living expenses out of it, it would be nice, but that’s definitely not the primary concern.
I’m looking for partners to build a tool, but also for advisors to help set the right value-optimizing architecture… “seed” value-adding behavior into the interface, as it were. I hope I can get some help from the LessWrong community. If this works, it could end up being a pretty influential bit of technology! I’d like it to be a net positive for humanity in the long term.
I’m probably getting ahead of myself.
What are the sources and sinks of your value system? Will old people have huge amounts of whuffie because they have been around for ages?
Good point! I assume we’ll have decay built into the system, based on age of the data points… some form of that is built into the architecture of FreeNet I believe, where less-accessed content eventually drops out from the network altogether.
I wasn’t even thinking about old people… I was more thinking about letting errors of youth not follow you around for your whole life… but at the same time, valuable content (that which is still attracting new readers who mark it as valuable) doesn’t disappear.
That said, longevity on the system means you’ve had more time to contribute… But if your contributions are generally rated as crappy, time isn’t going to help your influence without a significant ongoing improvement to your contributions’ quality.
But if you’re a cranky old nutjob, and there are people out there who like what you say, you can become influential in the nutjob community, if at the expense of your influence in other circles. You can be considered a leading light by a small group of people, but an idiot by the world at large.
I’m still not quite getting how this is going to work.
Lets say I am a spam blog bot. What it does is take popular (for a niche) articles and reposts automated summaries. So lets say it does this for cars. These aren’t very good, but aren’t very bad either. Perhaps it makes automatic word changes to real peoples summaries. It gets lots of other spam bots of this type and they form self-supportive networks (each up voting each other) and also liking popular things to do with cars. People come across these links and up vote them, because they go somewhere interesting. They gain lots of karma in these communities and then start pimping car related products or spreading FUD about rival companies. Automated astro-turf if you want.
Does anyone regulate the creation of new users?
How long before they stop being interesting to the car people? Or how much effort would it be to track them down and remove them from the circle of people you are interested in.
Also who keeps track of these votes? Can people ballot stuff?
I’ve thought a long these lines before and realised it is a non-trivial problem.
There’s a few questions in there. Let’s see.
Authentication and identity are an interesting issue. My concept is to allow anonymous users, with a very low initial influence level. But there would be many ways for users to strengthen their “identity score” (credit card verification, address verification via snail-mailed verif code, etc.), which would greatly and rapidly increase their influence score. A username that is tied to a specific person, and therefore wields much more influence, could undo the efforts of 100 bots with a single downvote.
But if you want to stay anonymous, you can. You’ll just have to patiently work on earning the same level of trust that is awarded to people who put their real-life reputation on the line.
I’m also conceiving of a richly semantic system, where simply “upvoting” or facebook-liking are the least influential actions one can take. Up from there, you can rate content on many factors, comment on it, review it, tag it, share it, reference it, relate it to other content. The more editorial and cerebral actions would probably do more to change one’s influence than a simple thumbs up. If a bot can compete with a human in writing content that gets rated high on “useful”, “factual”, “verifiable”, “unbiased”, AND “original” (by people who have high influence score in these categories), then I think the bot deserves a good influence score, because it’s a benevolent AI. ;)
Another concept, which would reduce incentives to game the system, is vouching. You can vouch for other users’ identity, integrity, maturity, etc. If you vouched for a bot, and the bot’s influence gets downgraded by the community, your influence will take a hit as well.
I see this happening throughout the system: Every time you exert your influence, you take responsibility for that action, as anyone may now rate/review/downvote your action. If you stand behind your judgement of Rush Limbaugh as truthful, enough people will disagree with you that from that point on, anytime you rate something as “truthful”, that rating will count for very little.
Hi avalot, thank you for the detailed discussion. I suspect the system I have in mind is simpler but should satisfy the same principles. In fact it has been eerie reading your post, as on principle we are in 95% agreement, to excruciation detail, and to a large extent on technical behaviour. I guess my one explicit difference is that I cannot let go of the profit motive. If I make a substantial contribution, I would like to be properly rewarded, if only to be able to materialize other ideas and contribute to causes I find worthy. That of course does not imply going to facebook’s lengths to squeeze the last drop of value out of its system, nor should it take precedence over openness and distribution. But to the extent that it can fit, I would like it to be there. Two questions for you:
First, with everyone rating everyone, how do you avoid your system becoming a keynesian beauty contest? (http://en.wikipedia.org/wiki/Keynesian_beauty_contest)
Second, assuming the number of connections increase exponentially with a linear increase in users, the processing load will also rise much quicker than the number of users. How will a system like this operate at web-scale?
Alexandros,
Not surprised that we’re thinking along the same lines, if we both read this blog! ;)
I love your questions. Let’s do this:
Keynesian Beauty Contest: I don’t have a silver bullet for it, but a lot of mitigation tactics. First of all, I envision offering a cascading set of progressively more fine-grained rating attributes, so that, while you can still upvote or downvote, or rate something with starts, you can also rate it on truthfulness, entertainment value, fairness, rationality (and countless other attributes)… More nuanced ratings would probably carry more influence (again, subject to others’ cross-rating). Therefore, to gain the highest levels of influence, you’d need to be nuanced in your ratings of content… gaming the system with nuanced, detailed opinions might be effectively the same as providing value to the system. I don’t mind someone trying to figure out the general population’s nuanced preferences… that’s actually a valuable service!
Secondly, your ratings are also cross-related to the semantic metadata (folksonomy of tags) of the content, so that your influence is limited to the topic at hand. Gaining a high influence score as a fashion celebrity doesn’t put your political or scientific opinions at the top of search results. Hopefully, this works as a sort of structural Palin-filter. ;)
The third mitigation has to do with your second question: How do we handle the processing of millions of real-time preference data points, when all of them should (in theory) get cross-related to all others, with (theoretically) endless recursion?
The typical web-based service approach of centralized crunching doesn’t make sense. I’m envisioning a distributed system where each influence node talks with a few others (a dozen?), and does some cross-processing with a them to agree on some temporary local normals, means and averages. That cluster does some more higher-level processing in consort with other close-by clusters, and they negotiate some “regional” aggregates… that gets propagated back down into the local level, and up to the next level of abstraction… up until you reach some set of a dozen superclusters that span the globe, and who trade in high-level aggregates.
All that is regulated, in terms of clock ticks, by activity: Content that is being rated/shared/commented on by many people will be accessed and cached by more local nodes, and processed by more clusters, and its cross-processing will be accelerated because it’s “hot”. Whereas one little opinion on one obscure item might not get processed by servers on the other side of the world until someone there requests it. We also decay data this way: If nobody cares, the system eventually forgets. (Your personal node will remember your preferences, but the network, after having consumed their influence effects, might forget their data points.)
A distributed, propagation system, batch-processed, not real-time, not atomic but aggregated. That means you can’t go back and change old ratings, and individual data points, because they get consumed by the aggregates. That means you can’t inspect what made your scored go up and down at the atomic level. That means your score isn’t the same everywhere on the planet at the same time. So gaming the system is harder because there’s no real-time feedback loop, there’s no single source of absolute truth (truth is local and propagates lazily), and there’s no auditing trail of the individual effects of your influence.
All of this hopefully makes the system so fluid that it holds innumerable beauty contests, always ongoing, always local, and the results are different depending on when and where you are. Hopefully this makes the search for the Nash equilibrium a futile exercise, and people give up and just say what they actually think is valuable to others, as opposed to just expected by others.
That’s my wishful thinking at the point. Am I fooling myself?
I’d create a simplified evolutionary model of the system using a GA to create the agents. If groups can find a way to game your system to create infinite interesting-ness/insightful-ness for specific topics, that then you need to change it.
You’re right: A system like that could be genetically evolved for optimization.
On the other hand, I was hoping to create an open optimization algorithm, governable by the community at large… based on their influence scores in the field of “online influence governance.” So the community would have to notice abuse and gaming of the system, and modify policy (as expressed in the algorithm, in the network rules, in laws and regulations and in social mores) to respond to it. Kind of like democracy: Make a good set of rules for collaborative rule-making, give it to the people, and hope they don’t break it.
But of course the Huns could take over. I’m trusting us to protect ourselves. In some way this would be poetic justice: If crowds can’t be wise, even when given a chance to select and filter among the members for wisdom, then I’ll give up on bootstrapping humanity and wait patiently for the singularity. Until then, though, I’d like to see how far we could go if given a useful tool for collaboration, and left to our own devices.
I think you are closer to a strong solution than you realize. You have mentioned the pieces but I think you haven’t put them together yet. In short, the solution I see is to depend on local (individual) decisions rather than group ones. If each node has its own ranking algorithm and its own set of trust relations, there is no reason to create complex group-cooperation mechanisms. A user that spams gets negative feedback and therefore eventually gets isolated in the graph. Even if automated users outnumber real users, the best they can do is vote each other up and therefore end up with their own cluster of the network, with real users only strongly connected to each other. Of course, if a bot provides value, it can be incorporated in that graph. “sufficiently advanced spam...”, etc. etc. This also means that the graph splinters into various clusters depending on worldview. (your rush limbaugh example). This deals with keynesian beauty contests as there is no ‘average’ to aim at. Your values simply cluster you with people who share them. If you value quality, you go closer to quality. If you value ‘republican-ness’ you move closer to that. The price you pay is that there is no ‘objective’ view of the system. There is no ‘top 10 articles’, only ‘top 10 articles for user X’.
Another thing I see with your design is that it is complex and attempts to boil at least a few oceans. (emergent ontologies/folksonomies for one, distributing identity, storage, etc.). I have some experience with defining complex architectures for distributed systems (e.g. http://arxiv.org/abs/0907.2485 ) and the problem is that they need years of work by many people to reach some theoretical purity, and even then bootstrapping will be a bitch. The system I have in mind is extremely simple by comparison, definitely more pragmatic (and therefore makes compromises) and is based on established web technologies. As a result, it should bootstrap itself quite easily. I find myself not wanting to publicly share the full details until I can start working on the thing (I am currently writing up my PhD thesis and my deadline is Oct. 1. After that, I’m focusing on this project). If you want to talk more details, we should probably take this to a private discussion.
You are right: This needs to be a fully decentralized system, with no center, and processing happening at the nodes. I was conceiving of “regional” aggregates mostly as a guess as to what may relieve network congestion if every node calls out to thousands of others.
Thank you for setting me right: My thinking has been so influenced by over a decade of web app dev that I’m still working on integrating the full principles of decentralized systems.
As for boiling oceans… I wish you were wrong, but you probably are right. Some of these architectures are likely to be enormously hard to fine-tune for effectiveness. At the same time, I am also hoping to piggyback on existing standards and systems.
Anyway, let’s certainly talk offline!
I think it is relevant because better social-news sites on the web would lead to better conversations about advancing the art of human rationality, which is the core mission of Less Wrong.
I think an iterated tournament might work better.
Announce 2 iterated prize sequences. The big Red Prizes for the best optimization algorithm and the small Blue prizes for the best spam which can spoof the same. Don’t award a blue until the first red is awarded and then don’t award a red until the last blue one is awarded and so on. Keep escalating the price amounts until satisfactory performance is attained.