The last paragraph of 1 before the questions is just wrong all over.
You tested a hundred hypotheses before finding one that the data supported, ignoring every failed hypothesis.
Yes to the first part, but there’s nothing methodologically wrong with that. No to the second part. What is the alternative to all of these hypotheses—that you don’t actually have keys at all?
You really wanted each of these hypotheses in turn to be true, and made no attempt to avoid bias.
What sort of bias do you mean? The chances that you would conclude that you had actually found your keys when you had not is roughly zero.
You stopped collecting data the moment a hypothesis was confirmed.
Once you have your keys in your hand, you have strictly falsified all of the other hypotheses.
When you were running out of ideas to test, you frantically thought up some more.
… yes, and how is this methodologically unsound? When you have no explanations, it is time to invent some.
You repeated some failed experiments in the hope of getting a different result.
False negative rate is low but nonzero. False positive rate is multiple orders of magnitude lower, as noted. Repeat measurements are legit.
Basically, a vanishingly small false positive rate renders a lot of methodological ‘sins’ irrelevant.
~~~~
On 2a, sure. If the odds are 1 in a hundred million, the pot is probably decent (you didn’t specify, so I can’t estimate), and checking is cheap.
2b/c, A match between the numbers on the ticket and the numbers in the drawing is no more or less likely than the actual odds of winning. That doesn’t mean I won’t check carefully, but a match is very strong evidence. I would basically be confident at this point, though cognizant of the distant possibility of a screwup or fraud. This would be hard to quantify, since I do not have data on the subject.
I would basically be confident at this point, though cognizant of the distant possibility of a screwup or fraud.
Clearly people you know don’t take pranking seriously enough. Your friend sitting quietly at their laptop might have overheard your conversation and deployed a mirror website with fraudulent information to make your day more surreal (as I have on more than one occasion). Faking a lottery ticket would be too cruel, but I would totally fake a New York Times article “’Obama Calls on Somers to Stop F*#ing Around and Finish His Thesis Already”.
edit: This comment is not on topic, but for some reason I can’t delete it
It’s usually more ‘virtuous’ to replicate the positive (i.e. information-laden, surprising) results, while repeating negative results seems more like fishing for better p-values.
The last paragraph of 1 before the questions is just wrong all over.
Yes to the first part, but there’s nothing methodologically wrong with that. No to the second part. What is the alternative to all of these hypotheses—that you don’t actually have keys at all?
What sort of bias do you mean? The chances that you would conclude that you had actually found your keys when you had not is roughly zero.
Once you have your keys in your hand, you have strictly falsified all of the other hypotheses.
… yes, and how is this methodologically unsound? When you have no explanations, it is time to invent some.
False negative rate is low but nonzero. False positive rate is multiple orders of magnitude lower, as noted. Repeat measurements are legit.
Basically, a vanishingly small false positive rate renders a lot of methodological ‘sins’ irrelevant.
~~~~
On 2a, sure. If the odds are 1 in a hundred million, the pot is probably decent (you didn’t specify, so I can’t estimate), and checking is cheap.
2b/c, A match between the numbers on the ticket and the numbers in the drawing is no more or less likely than the actual odds of winning. That doesn’t mean I won’t check carefully, but a match is very strong evidence. I would basically be confident at this point, though cognizant of the distant possibility of a screwup or fraud. This would be hard to quantify, since I do not have data on the subject.
Clearly people you know don’t take pranking seriously enough. Your friend sitting quietly at their laptop might have overheard your conversation and deployed a mirror website with fraudulent information to make your day more surreal (as I have on more than one occasion). Faking a lottery ticket would be too cruel, but I would totally fake a New York Times article “’Obama Calls on Somers to Stop F*#ing Around and Finish His Thesis Already”.
edit: This comment is not on topic, but for some reason I can’t delete it
yes, actually. It’s a bit disappointing. Just a little.
Seconding this comment; I will just further add that
Is known as ‘repeatability’, and is normally considered virtuous to do.
It’s usually more ‘virtuous’ to replicate the positive (i.e. information-laden, surprising) results, while repeating negative results seems more like fishing for better p-values.