If you have someone trying to do something and be unsuccessful that tells you even less. I would not see it as indicator of not working. It is just no particular evidence.
It almost has to be evidence; even if it’s just evidence that the person isn’t doing it right, then you push this case through your prior for how often they do things wrong. Unless your prior is very high, you’re still getting half the impact or more of the evidence. Although you could argue from the selection effect of “correct in your expectations of success” that you’re almost guaranteed to only notice cases where they are doing it wrong, not cases where it doesn’t work (because you don’t expect it to work where it won’t).
A person using the techniques and not having increased success—weak evidence
People hating PUA—trivial evidence
A bunch of people active on a PUA forum having increased success in aggregagate—moderate evidence (would be strong but selection on dependent variable)
It almost has to be evidence; even if it’s just evidence that the person isn’t doing it right, then you push this case through your prior for how often they do things wrong. Unless your prior is very high, you’re still getting half the impact or more of the evidence. Although you could argue from the selection effect of “correct in your expectations of success” that you’re almost guaranteed to only notice cases where they are doing it wrong, not cases where it doesn’t work (because you don’t expect it to work where it won’t).
My take -
A person using the techniques and not having increased success—weak evidence
People hating PUA—trivial evidence
A bunch of people active on a PUA forum having increased success in aggregagate—moderate evidence (would be strong but selection on dependent variable)