Eliezer never said if he reversed the results a second time.
1 I found somewhat predictable
2 I found obvious
3 I didn’t know one way or the other
4 I found moderately unpredictable
5 I found moderately unpredictable
Overall, I’m completely unsure whether or not they’re reversed. Well, I suspect they’re only reversed once, but not on account of the data itself.
I was pretty certain about that economic one later on, but I’m good at economics. If I was wrong on that one, I would have looked for more data to back that up, and if I was indeed incorrect, it would make a huge impact on my political and economic opinions.
That vagueness is deliberate. The point is that Eliezer didn’t tell you inside the post which option was correct, because hindsight would then take over and make the results seem predictable. The idea is that then the reader is left with the uncertainty, and the problem of deciding what was the actual true answer. I went through each proposition individually and attempted to determine whether or not that proposition was true. I was correct four out of six times, which is probably a reasonable score.
I think readers should attempt to determine, on their own, their advance predictions (preferably written down, although I admit to skipping that step), before looking up the results of the actual study. The correct results are available, but I would recommend that the true results not be discussed or linked to on less wrong.
Eliezer never said if he reversed the results a second time.
1 I found somewhat predictable
2 I found obvious
3 I didn’t know one way or the other
4 I found moderately unpredictable
5 I found moderately unpredictable
Overall, I’m completely unsure whether or not they’re reversed. Well, I suspect they’re only reversed once, but not on account of the data itself.
I was pretty certain about that economic one later on, but I’m good at economics. If I was wrong on that one, I would have looked for more data to back that up, and if I was indeed incorrect, it would make a huge impact on my political and economic opinions.
That vagueness is deliberate. The point is that Eliezer didn’t tell you inside the post which option was correct, because hindsight would then take over and make the results seem predictable. The idea is that then the reader is left with the uncertainty, and the problem of deciding what was the actual true answer. I went through each proposition individually and attempted to determine whether or not that proposition was true. I was correct four out of six times, which is probably a reasonable score.
I think readers should attempt to determine, on their own, their advance predictions (preferably written down, although I admit to skipping that step), before looking up the results of the actual study. The correct results are available, but I would recommend that the true results not be discussed or linked to on less wrong.
The obvious thing to do would be to choose the reversal-parity randomly (by flipping a coin), independently for each claim.