it’s people who were working on “Birdwatch” before Musk bought Twitter, and they use algorithms derived from PageRank.
these guys are not, even a little bit, “based.” Yet the Twitter userbase loves Community Notes!
if you have a capable team that firmly believes in “fairness”, in auditable, open, participatory processes that don’t put a top-down thumb on the scale on controversial issues, and they get to actually use the neutral algorithm instead of being pressured to make exceptions, you get solid results and community trust!
it’s not machine-learning based; it’s a version of GOFAI that’s formalizing the types of “tactics” or “moves” that a human goes through in a proof, trying to get the formalism right such that a computer proceduralization only has a modest, human-like amount of trial-and-error & backtracking rather than vast amounts of brute-force search.
“proof repair” is the problem of updating formally verified software; if you have a library of provably correct code, and you make any kind of software updates to the library, now you also have to update the proofs such that it’s still verifiably correct (assuming the update didn’t break anything.) Progress towards automating this.
https://web.stanford.edu/~jlmcc/ Jay McClelland, co-inventor (with Rumelhart of backprop fame) of the Parallel Distributed Processing theory of cognition, has been doing a lot with LLMs lately
https://web.stanford.edu/~jlmcc/papers/McClelland22CapturingAdvCogAbilitiesWithDeepNets humans learn human-made formal systems (like mathematics, computer science, logic) in order to solve certain kinds of difficult problems. perhaps AIs should also “go to school”, being trained on math problems, in particular with diagrams as well as text. also, goal-directed motivation may require fundamentally different architecture from the usual LLM transformer setup.
https://web.stanford.edu/~jlmcc/papers/NamMcC21RapidLearningGeneralizationInHumansArxiv.pdf human mTurkers are better at abstract problem-solving tasks if they’ve taken high school algebra and geometry (no effect for other educational variables). they split pretty bimodally into people who learn a strategy and people who guess at random. this points towards “basic math education teaches systematic thought.” also, small RNNs generalize much worse than humans, but who cares.
https://en.wikipedia.org/wiki/Answer_set_programming ASP is used for difficult search & combinatorics or optimization problems. I’m struggling to understand whether it is in wide industrial use or if it’s more of a research specialization.
if you have a capable team that firmly believes in “fairness”, in auditable, open, participatory processes that don’t put a top-down thumb on the scale on controversial issues, and they get to actually use the neutral algorithm instead of being pressured to make exceptions, you get solid results and community trust!
Then it is quite sad that the neutral algorithm was introduced as the same time as Xitter started losing popularity. (At least, it seems that it loses popularity? Maybe that’s just some bubble. I don’t know what to trust anymore.)
Could these things be related? It seems like the opposition against Xitter is mostly because Musk is hanging out with Trump recently. But hypothetically, it could be a combination of that and the fact that the Community Notes may be inconvenient for people who instead could have the content policed by members of their tribe.
Sorry for getting political, but at least until recently it seemed like one political tribe practically owned all the “mainstream” parts of the internet; not necessarily most of the users, but most of the mods and admins. They didn’t need to try finding a neutral ground, because instead, they could simply have it all.
I have seen a few attempts to make a neutral place where both sides could discuss, and those usually didn’t work well. The dominant tribe had no incentive to participate, if they could win the debate by avoiding the place and from outside declaring it to be full of horrible people who should be banned. You could only attract them by basically conceding to many of their demands (declaring their norms and taboos to be the rules of the group), which already made an equal debate impossible (stating your disagreement already meant breaking some of the rules), which made the debate kinda pointless (you could only make your point by diluting it to homeopathic levels, and then the other side yelled at you for a while, and then everyone congratulated themselves for being so tolerant and open-minded). I don’t want to give specific examples, but instead I will point to how Scott Alexander’s blog was handled e.g. by Wikipedia—despite the fact that most of its readers (and Scott himself) actually belonged to the dominant tribe, the fact that dissent was allowed was enough for some admins to call him names.
It is usually the weaker side that calls for fairness. Yes, it is amazing that you can implement it algorithmically, but the people who have the power to make this decisions, are usually not the ones who want it made.
So I wonder what will happen in future. Will more web platforms adopt the neutral algorithm? Or will it be instead something like “a neutral algorithm, but our trusted moderators can override its results if they don’t like them”?
links 1/7/25: https://roamresearch.com/#/app/srcpublic/page/01-07-2025
https://asteriskmag.com/issues/08/the-making-of-community-notes the team behind Community Notes describes how they do it
it’s people who were working on “Birdwatch” before Musk bought Twitter, and they use algorithms derived from PageRank.
these guys are not, even a little bit, “based.” Yet the Twitter userbase loves Community Notes!
if you have a capable team that firmly believes in “fairness”, in auditable, open, participatory processes that don’t put a top-down thumb on the scale on controversial issues, and they get to actually use the neutral algorithm instead of being pressured to make exceptions, you get solid results and community trust!
https://pmc.ncbi.nlm.nih.gov/articles/PMC7132523/ 90% of bronchitis is viral. doctors are cautioned not to give antibiotics for it, even for long-lasting coughs.
https://www.aafp.org/pubs/afp/issues/2010/1201/p1345.html no really. bronchitis symptoms typically last three weeks. it’s still not bacterial. antibiotics do not work.
https://harmonic.fun/ AI for formal mathematical reasoning
it’s Lean based! https://web.archive.org/web/20241230145058/https://www.nytimes.com/2024/09/23/technology/ai-chatbots-chatgpt-math.html
https://drive.google.com/file/d/1-FFa6nMVg18m1zPtoAQrFalwpx2YaGK4/view Tim Gowers’ research manifesto on AI for mathematics
it’s not machine-learning based; it’s a version of GOFAI that’s formalizing the types of “tactics” or “moves” that a human goes through in a proof, trying to get the formalism right such that a computer proceduralization only has a modest, human-like amount of trial-and-error & backtracking rather than vast amounts of brute-force search.
https://www.businesswire.com/news/home/20240514445088/en/Multiply-Labs-and-Retro-Biosciences-Announce-an-85-Million-Partnership-to-Advance-Cell-Therapy-Manufacturing-for-Age-Related-Diseases cell therapy manufacturing automation robots from Multiply Labs, used at Retro Bio.
https://gopiandcode.uk/pdfs/sisyphus-pldi23.pdf
“proof repair” is the problem of updating formally verified software; if you have a library of provably correct code, and you make any kind of software updates to the library, now you also have to update the proofs such that it’s still verifiably correct (assuming the update didn’t break anything.) Progress towards automating this.
https://openreview.net/pdf?id=RtTNbJthjV the Karp Dataset, 90 reduction-based NP-hardness proofs (in natural language) for training LLMs to write proofs.
https://web.stanford.edu/~jlmcc/ Jay McClelland, co-inventor (with Rumelhart of backprop fame) of the Parallel Distributed Processing theory of cognition, has been doing a lot with LLMs lately
https://web.stanford.edu/~jlmcc/papers/ChanEtAl22DataDistPropsDriveInContextLearning.pdf in-context learning works better when the training data is bursty and has lots of sparsely presented elements
https://web.stanford.edu/~jlmcc/papers/DasguptaLampinenEtAl22LMsShowHumanLikeContentEffectsInReasoning.pdf LLMs, like humans, show belief-bias effects on Wason and syllogism tests
https://web.stanford.edu/~jlmcc/papers/McClelland22CapturingAdvCogAbilitiesWithDeepNets humans learn human-made formal systems (like mathematics, computer science, logic) in order to solve certain kinds of difficult problems. perhaps AIs should also “go to school”, being trained on math problems, in particular with diagrams as well as text. also, goal-directed motivation may require fundamentally different architecture from the usual LLM transformer setup.
https://web.stanford.edu/~jlmcc/papers/NamEtAl22LearningToReasonWRelationalAbstractions.pdf an empirical example of “taking the AI to school”—fine-tuning on a McClelland-designed dataset intended to teach “relational abstractions” makes models perform better at math problems
https://web.stanford.edu/~jlmcc/papers/NamMcC21RapidLearningGeneralizationInHumansArxiv.pdf human mTurkers are better at abstract problem-solving tasks if they’ve taken high school algebra and geometry (no effect for other educational variables). they split pretty bimodally into people who learn a strategy and people who guess at random. this points towards “basic math education teaches systematic thought.” also, small RNNs generalize much worse than humans, but who cares.
https://en.wikipedia.org/wiki/Answer_set_programming ASP is used for difficult search & combinatorics or optimization problems. I’m struggling to understand whether it is in wide industrial use or if it’s more of a research specialization.
Then it is quite sad that the neutral algorithm was introduced as the same time as Xitter started losing popularity. (At least, it seems that it loses popularity? Maybe that’s just some bubble. I don’t know what to trust anymore.)
Could these things be related? It seems like the opposition against Xitter is mostly because Musk is hanging out with Trump recently. But hypothetically, it could be a combination of that and the fact that the Community Notes may be inconvenient for people who instead could have the content policed by members of their tribe.
Sorry for getting political, but at least until recently it seemed like one political tribe practically owned all the “mainstream” parts of the internet; not necessarily most of the users, but most of the mods and admins. They didn’t need to try finding a neutral ground, because instead, they could simply have it all.
I have seen a few attempts to make a neutral place where both sides could discuss, and those usually didn’t work well. The dominant tribe had no incentive to participate, if they could win the debate by avoiding the place and from outside declaring it to be full of horrible people who should be banned. You could only attract them by basically conceding to many of their demands (declaring their norms and taboos to be the rules of the group), which already made an equal debate impossible (stating your disagreement already meant breaking some of the rules), which made the debate kinda pointless (you could only make your point by diluting it to homeopathic levels, and then the other side yelled at you for a while, and then everyone congratulated themselves for being so tolerant and open-minded). I don’t want to give specific examples, but instead I will point to how Scott Alexander’s blog was handled e.g. by Wikipedia—despite the fact that most of its readers (and Scott himself) actually belonged to the dominant tribe, the fact that dissent was allowed was enough for some admins to call him names.
It is usually the weaker side that calls for fairness. Yes, it is amazing that you can implement it algorithmically, but the people who have the power to make this decisions, are usually not the ones who want it made.
So I wonder what will happen in future. Will more web platforms adopt the neutral algorithm? Or will it be instead something like “a neutral algorithm, but our trusted moderators can override its results if they don’t like them”?