Do you know why the error bars in the replication are smaller than the original one? (just more people?) And with which confidence is the null hypothesis (difference = 0) is rejected in both cases?
The political version of the question isn’t functionally the same as the skin cream version, because the former isn’t a randomized intervention—cities that decided to add gun control laws seem likely to have other crime-related events and law changes at the same time, which could produce a spurious result in either direction. So it’s quite reasonable to say “My opinion is determined by my priors and the evidence didn’t appreciably affect my position.”
A few glaring issues here: 1) Does the question imply causation or not? It shouldn’t. 2) Are these stats intended to be realistic such that I need to consider potential flaws and take a holistic view or just a toy scenario to test my numerical skills? If I believe it’s the former and I’m confident X and Y are positively correlated, a 2x2 grid showing X and Y negatively correlated should of course make me question the quality of your data proportionally. 3) Is this an adversarial question such that my response may be taken out of context or otherwise misused?
The sample interviews from Veritasium did not seem to address any of these issues: (1) They seemed to cut out the gun question, but the skin cream question implied causation, “Did the skin cream make the rash better or worse?” (2) One person mentioned “I Wouldn’t have expected that...” which implies he thought it was real data, (3) the last person clearly interpreted it adversarially.
In the original study, the question was stated as “cities that enacted a ban on carrying concealed handguns were more likely to have a decrease in crime.” This framing is not as bad, but still too close to implying causation in my opinion.
Enjoyed this video by Veritasium with data showing how Politics is the Mind Killer
While the broader message might be good, the study the video is about didn’t replicate.
Kicking myself for not making a fatebook about this. It definitely sounded like the kind of thing that wouldn’t replicate.
They replicated it within the video itself?
I watched the video and didn’t see any stats from their own experiment. Do you have a frame or a section?
I don’t remember them having the actual stats, not watching it again though. I wonder if they published those elsewhere
Do you know why the error bars in the replication are smaller than the original one? (just more people?) And with which confidence is the null hypothesis (difference = 0) is rejected in both cases?
The political version of the question isn’t functionally the same as the skin cream version, because the former isn’t a randomized intervention—cities that decided to add gun control laws seem likely to have other crime-related events and law changes at the same time, which could produce a spurious result in either direction. So it’s quite reasonable to say “My opinion is determined by my priors and the evidence didn’t appreciably affect my position.”
A few glaring issues here:
1) Does the question imply causation or not? It shouldn’t.
2) Are these stats intended to be realistic such that I need to consider potential flaws and take a holistic view or just a toy scenario to test my numerical skills? If I believe it’s the former and I’m confident X and Y are positively correlated, a 2x2 grid showing X and Y negatively correlated should of course make me question the quality of your data proportionally.
3) Is this an adversarial question such that my response may be taken out of context or otherwise misused?
The sample interviews from Veritasium did not seem to address any of these issues:
(1) They seemed to cut out the gun question, but the skin cream question implied causation, “Did the skin cream make the rash better or worse?”
(2) One person mentioned “I Wouldn’t have expected that...” which implies he thought it was real data,
(3) the last person clearly interpreted it adversarially.
In the original study, the question was stated as “cities that enacted a ban on carrying concealed handguns were more likely to have a decrease in crime.” This framing is not as bad, but still too close to implying causation in my opinion.