Since then I found a partially relevant, but very simple and effective “puzzle”.
There are four cards in front of you on the desk. It is known, that every card has a numerical digit on one side, and a letter from the English alphabet on the other side.
You have to verify the theory that “if one side of the card has a vowel, the other side has an even number”, and you are only allowed to flip two cards.
The cards in front of you are:
A T 7 2
Which cards will you flip?
(I wrote partially relevant because this is not an example for an unfalsifiable theory. The theory is falsifiable and the puzzle is solvable, the main point is that most people would pick the wrong answer because they will not try to falsify the theory)
If this would be enough to prove the effectiveness of rain-dancing, then we would develop 30 different styles of rain-dance, test each of them, and with a very high chance we would get p<0.05 on at least one of them.
Sadly, the medical industry is full of such publications, because publishing new ideas is rewarded more than reproducing already published experiments.