There are an infinite number of hypotheses for what an ‘Awesome Triplet’ could be.
Here are some example hypotheses that could be true based on our initial evidence ‘2 4 6 is an awesome triplet’:
Any three integers
Any three integers in ascending order
Three successive multiples of the same number
The sequence ‘2 4 6’
Three integers not contained in the set ’512 231123 691 9834 91238 1′
We cannot falsify every possible hypothesis, so we need a strategy to falsify hypotheses, starting from the most likely. All hypotheses are not created equal.
I want to falsify as much of the hypotheses-space as possible (where simple hypthoses take up more space), so I design a test that should do so. My first test was ‘3 integers in descending order’, because it can falsify #1, the simplest hypothesis.
I find from this test that #1 is false.
My second test is to distinguish between #2 & #3; ‘3 integers in ascending order, but not successive multiples of the same number’, ‘1 2 5’
I find from this test that #2 is still plausible, but #3 is falsified.
You can continue falsifying smaller and smaller areas of the hypothesis-space with additional tests, up until you’re happy with your confidence level or you’re bored of testing.
For much better coverage of this entire area, see the following posts by Eliezer:
There is a real positive bias and this program helps confirm it.
Something that must be considered is whether the form of the test could have an influence on the outcome for reasons other than an intrinsic positive bias. More specifically, I note that the form of the question the participant has been given resembles that of question style that I have encountered many a time. In most of these cases I am expected to elicit the questioner’s intended meaning, usually something specific. Were I to give the answer “actually, it could be any integers in ascending order” I would expect less marks or mild dissaproval for being a smart ass.
The test is set up to confirm the positive bias without eliminating the possibility of simple cultural training on the test format and initial priming. I would like to look at alternate tests setups, perhaps including explicitly declared random triplets and some betting.
As it stands, the test strikes me as a little ironic!
“Susceptability to Trick Question Bias”
It is absolutely NOT a trick question.
There are an infinite number of hypotheses for what an ‘Awesome Triplet’ could be. Here are some example hypotheses that could be true based on our initial evidence ‘2 4 6 is an awesome triplet’:
Any three integers
Any three integers in ascending order
Three successive multiples of the same number
The sequence ‘2 4 6’
Three integers not contained in the set ’512 231123 691 9834 91238 1′
We cannot falsify every possible hypothesis, so we need a strategy to falsify hypotheses, starting from the most likely. All hypotheses are not created equal.
I want to falsify as much of the hypotheses-space as possible (where simple hypthoses take up more space), so I design a test that should do so. My first test was ‘3 integers in descending order’, because it can falsify #1, the simplest hypothesis. I find from this test that #1 is false. My second test is to distinguish between #2 & #3; ‘3 integers in ascending order, but not successive multiples of the same number’, ‘1 2 5’ I find from this test that #2 is still plausible, but #3 is falsified.
You can continue falsifying smaller and smaller areas of the hypothesis-space with additional tests, up until you’re happy with your confidence level or you’re bored of testing.
For much better coverage of this entire area, see the following posts by Eliezer:
What is Evidence?
The Lens That Sees Its Flaws
How Much Evidence Does It Take?
Occam’s Razor
For a good overview of additional related posts, see the list.
Edit: Learning Markdown, fixing style.
There is a real positive bias and this program helps confirm it.
Something that must be considered is whether the form of the test could have an influence on the outcome for reasons other than an intrinsic positive bias. More specifically, I note that the form of the question the participant has been given resembles that of question style that I have encountered many a time. In most of these cases I am expected to elicit the questioner’s intended meaning, usually something specific. Were I to give the answer “actually, it could be any integers in ascending order” I would expect less marks or mild dissaproval for being a smart ass.
The test is set up to confirm the positive bias without eliminating the possibility of simple cultural training on the test format and initial priming. I would like to look at alternate tests setups, perhaps including explicitly declared random triplets and some betting.
As it stands, the test strikes me as a little ironic!