I’d very much like to know more about this too. I distinctly recall at least three separate papers by separate authors in different domains that mentioned “adjusting for confounders” on things that I thought would be, so I thought “Oh, it’s fine then, they did their thinking properly and there is an effect!”. (At times like these I wish I remembered things or kept notes on research papers a fraction as diligently as Yvain or gwern...)
Then I read further because now I’m very interested in why, and upon finding the details of their adjustments, in small print under one of the many tables of Annex F, I discover that the “adjustment” was that they guessed at an effective rate for the confounders and plugged that percentage in their math. “Oh, there must be about 15% more rich people than poor people who go to the doctor for any given condition, so let’s adjust the results by that amount and see if things work!”
(I’m exaggerating for dramatization, these guess numbers are rarely “hidden” in this way and rarely this important, but even tiny examples of such piss me off and I get angry at the paper for it every time.)
I’d very much like to know more about this too. I distinctly recall at least three separate papers by separate authors in different domains that mentioned “adjusting for confounders” on things that I thought would be, so I thought “Oh, it’s fine then, they did their thinking properly and there is an effect!”. (At times like these I wish I remembered things or kept notes on research papers a fraction as diligently as Yvain or gwern...)
Then I read further because now I’m very interested in why, and upon finding the details of their adjustments, in small print under one of the many tables of Annex F, I discover that the “adjustment” was that they guessed at an effective rate for the confounders and plugged that percentage in their math. “Oh, there must be about 15% more rich people than poor people who go to the doctor for any given condition, so let’s adjust the results by that amount and see if things work!”
(I’m exaggerating for dramatization, these guess numbers are rarely “hidden” in this way and rarely this important, but even tiny examples of such piss me off and I get angry at the paper for it every time.)