Alex Turner introduced an exercise to test subjects’ ability to notice falsehoods: change factual statements in Wikipedia articles, hand the edited articles to subjects and see whether they notice the modifications.
I’ve spent a few hours making such modifications and testing the articles on my friend group. You can find the articles here. I describe my observations and thoughts below. The bottom line: it is hard to come up with good modifications / articles to modify, and this is the biggest crux for me.
The concept
Alex Turner explains the idea well here. The post is short, so I’m just copying it here:
Rationality exercise: Take a set of Wikipedia articles on topics which trainees are somewhat familiar with, and then randomly select a small number of claims to negate (negating the immediate context as well, so that you can’t just syntactically discover which claims were negated).
“By the time they are born, infants can recognize and have a preference for their mother’s voice suggesting some prenatal development of auditory perception.”
-> modified to
“Contrary to early theories, newborn infants are not particularly adept at picking out their mother’s voice from other voices. This suggests the absence of prenatal development of auditory perception.”
Sometimes, trainees will be given a totally unmodified article. For brevity, the articles can be trimmed of irrelevant sections.
Benefits:
Addressing key rationality skills. Noticing confusion; being more confused by fiction than fact; actually checking claims against your models of the world.
If you fail, either the article wasn’t negated skillfully (“5 people died in 2021” → “4 people died in 2021″ is not the right kind of modification), you don’t have good models of the domain, or you didn’t pay enough attention to your confusion.
Either of the last two are good to learn.
Features of good modifications
What does a good modification look like?
Let’s start by exploring some failure modes. Consider the following modifications:
“World War II or the Second World War (1 September 1939 – 2 September 1945) was...” → “World War II or the Second World War (31 August 1939 – 2 September 1945) was...
“In the wake of Axis defeat, Germany, Austria, Japan and Korea were occupied” → “In the wake of Allies defeat, United States, France and Great Britain were occupied”
“Operation Barbarossa was the invasion of the Soviet Union by...” → “Operation Bergenstein was the invasion of the Soviet Union by...”
Needless to say, these are obviously poor changes for more than one reason. Doing something which is not that, one gets at least the following desiderata for a good change:
The modifications shouldn’t be too obvious nor too subtle; both failure and success should be realistic outcomes.
The modification should have implications, rather than being an isolated fact, test of memorization or a mere change of labels.
The “intended solution” is based on general understanding of a topic, rather than memorization.
The change “The world population is 8 billion” → “The world population is 800,000” definitely has implications, and you could indirectly infer that the claim is false, but in practice people would think “I’ve previously read that the world population is 8 billion. This article gives a different number. This article is wrong.” Thus, this is a bad change.
Finally, let me add:
The topic is of general interest and importance.
While the focus is on general rationality skills rather than object-level information, I think you get better examples by having interesting and important topics, rather than something obscure.
Informally, an excellent modification is such that it’d just be very silly to actually believe the false claim made, in the sense of “How on Earth could a reasonable sane person believe that!? You can obviously see how this is false from X, Y or Z”, while still being non-trivial to notice.
Examples
I’ve created eight modified articles. You can find them at the Google Drive folder here.
(Details: For each article you can find the original article stripped for brevity, the edited article, an answer file and Wikipedia auxiliary files. You should download the whole folder and open the html file for the edited article in your browser. Tested on Ubuntu, should work on other systems as well.)
If you only have the energy to look at one article, see the Industrial Revolution one. If you want to look at two more, see the World economy and Price gouging articles. I think the articles steeply drop in quality (more on this below), but I put all articles I created in the folder anyways.
When I play-tested these with my friends, we usually took around 10 minutes per article for reading and writing down thoughts.
Takeaways and thoughts
The biggest challenge is coming up with claims to modify.
I quickly noticed that articles often don’t have lots of very clear-cut factual claims you can invert. To understand what I mean, look at the Wikipedia article on philosophy, for example. It just isn’t amenable to modifications satisfying the criteria laid out above.
Even if you have factual statements, it is often of the form of memorized facts (“The capital of France is Paris”) that are in themselves inconsequential. Or, if you have some very consequential facts, like “The population of Earth is 8 billion”, as opposed to 800,000 or 80 trillion, you have memorized those anyways.
Finding consequential statements that people haven’t memorized beforehand, or don’t immediately recognize as true or false, is not easy. I felt like many examples I came up with just weren’t good enough.
(Other source materials – scientific publications, perhaps – could be better in this regard than Wikipedia, though then one runs into issues of technicality and narrowness. I also thought of modifying the overall lean or bias of an article by selective focusing, withdrawal of information and “lies of omission”. Noticing such changes is a skill that clearly has real life applications, but more work is needed to produce articles.)
In any case, I haven’t spent that much effort on this, and feel like I lack imagination on the sorts of topics one could use.[1] Plausibly if you had more people spending more effort on this, you would come up with lots of other good examples and discover heuristics for finding good articles, types of modifications to make etc.
This remains the biggest crux for me: how difficult is it to come up with good modifications? If others find it as hard as I did, then it’s hard to get enough supply of articles. If others are more capable than I am, I’m much more optimistic: the idea is extremely scalable, allows for iteration on quantitative metrics, and one can filter for high-quality examples. So if you like the idea, consider taking some time to think about it and share your thoughts.
I did try to prompt GPT and Claude models to suggest me ideas, but their article suggestions were mediocre, and suggested modifications were rubbish. (I didn’t spend a lot of time optimizing the prompts, however.) At one point I did use Claude to write a fake paragraph in Wikipedia style for one of my modifications, and was happy with the result.
Brief notes on the Wikipedia game
Alex Turner introduced an exercise to test subjects’ ability to notice falsehoods: change factual statements in Wikipedia articles, hand the edited articles to subjects and see whether they notice the modifications.
I’ve spent a few hours making such modifications and testing the articles on my friend group. You can find the articles here. I describe my observations and thoughts below. The bottom line: it is hard to come up with good modifications / articles to modify, and this is the biggest crux for me.
The concept
Alex Turner explains the idea well here. The post is short, so I’m just copying it here:
Features of good modifications
What does a good modification look like?
Let’s start by exploring some failure modes. Consider the following modifications:
“World War II or the Second World War (1 September 1939 – 2 September 1945) was...” → “World War II or the Second World War (31 August 1939 – 2 September 1945) was...
“In the wake of Axis defeat, Germany, Austria, Japan and Korea were occupied” → “In the wake of Allies defeat, United States, France and Great Britain were occupied”
“Operation Barbarossa was the invasion of the Soviet Union by...” → “Operation Bergenstein was the invasion of the Soviet Union by...”
Needless to say, these are obviously poor changes for more than one reason. Doing something which is not that, one gets at least the following desiderata for a good change:
The modifications shouldn’t be too obvious nor too subtle; both failure and success should be realistic outcomes.
The modification should have implications, rather than being an isolated fact, test of memorization or a mere change of labels.
The “intended solution” is based on general understanding of a topic, rather than memorization.
The change “The world population is 8 billion” → “The world population is 800,000” definitely has implications, and you could indirectly infer that the claim is false, but in practice people would think “I’ve previously read that the world population is 8 billion. This article gives a different number. This article is wrong.” Thus, this is a bad change.
Finally, let me add:
The topic is of general interest and importance.
While the focus is on general rationality skills rather than object-level information, I think you get better examples by having interesting and important topics, rather than something obscure.
Informally, an excellent modification is such that it’d just be very silly to actually believe the false claim made, in the sense of “How on Earth could a reasonable sane person believe that!? You can obviously see how this is false from X, Y or Z”, while still being non-trivial to notice.
Examples
I’ve created eight modified articles. You can find them at the Google Drive folder here.
(Details: For each article you can find the original article stripped for brevity, the edited article, an answer file and Wikipedia auxiliary files. You should download the whole folder and open the html file for the edited article in your browser. Tested on Ubuntu, should work on other systems as well.)
If you only have the energy to look at one article, see the Industrial Revolution one. If you want to look at two more, see the World economy and Price gouging articles. I think the articles steeply drop in quality (more on this below), but I put all articles I created in the folder anyways.
When I play-tested these with my friends, we usually took around 10 minutes per article for reading and writing down thoughts.
Takeaways and thoughts
The biggest challenge is coming up with claims to modify.
I quickly noticed that articles often don’t have lots of very clear-cut factual claims you can invert. To understand what I mean, look at the Wikipedia article on philosophy, for example. It just isn’t amenable to modifications satisfying the criteria laid out above.
Even if you have factual statements, it is often of the form of memorized facts (“The capital of France is Paris”) that are in themselves inconsequential. Or, if you have some very consequential facts, like “The population of Earth is 8 billion”, as opposed to 800,000 or 80 trillion, you have memorized those anyways.
Finding consequential statements that people haven’t memorized beforehand, or don’t immediately recognize as true or false, is not easy. I felt like many examples I came up with just weren’t good enough.
(Other source materials – scientific publications, perhaps – could be better in this regard than Wikipedia, though then one runs into issues of technicality and narrowness. I also thought of modifying the overall lean or bias of an article by selective focusing, withdrawal of information and “lies of omission”. Noticing such changes is a skill that clearly has real life applications, but more work is needed to produce articles.)
In any case, I haven’t spent that much effort on this, and feel like I lack imagination on the sorts of topics one could use.[1] Plausibly if you had more people spending more effort on this, you would come up with lots of other good examples and discover heuristics for finding good articles, types of modifications to make etc.
This remains the biggest crux for me: how difficult is it to come up with good modifications? If others find it as hard as I did, then it’s hard to get enough supply of articles. If others are more capable than I am, I’m much more optimistic: the idea is extremely scalable, allows for iteration on quantitative metrics, and one can filter for high-quality examples. So if you like the idea, consider taking some time to think about it and share your thoughts.
I did try to prompt GPT and Claude models to suggest me ideas, but their article suggestions were mediocre, and suggested modifications were rubbish. (I didn’t spend a lot of time optimizing the prompts, however.) At one point I did use Claude to write a fake paragraph in Wikipedia style for one of my modifications, and was happy with the result.