Recapitulating something I’ve written about before:
You should first make a serious effort to formulate both the specific question you want answered, and why you want an answer. It may turn out surprisingly often that you don’t need to do all this work to evaluate the study.
Short of becoming an expert yourself, your best bet is then to learn how to talk to people in the field until you can understand what they think about the paper and why—and also how they think and talk about these things. This is roughly what Harry Collins calls “interactional” expertise. (He takes gravitational-wave scientist Joe Weber’s late work as an especially vivid example: “I can promise such lay readers that if they teach themselves a bit of elementary statistics and persevere with reading the paper, they will find it utterly convincing. Scientific papers are written to be utterly convincing; over the centuries their special language and style has been developed to make them read convincingly.… The only way to know that Weber’s paper is not to be read in the way it is written is to be a member of the ‘oral culture’ of the relevant specialist community.” The full passage is very good.)
If you only learn from papers (or even textbooks and papers), you won’t have any idea what you’re missing. A lot of expertise is bound up in individual tacit knowledge and group dynamics that never get written down. This isn’t to say that the ‘oral culture’ is always right, but if you don’t have a good grasp of it, you will make at best slow progress as an outsider.
This is the main thing holding me back from running the course I’ve half-written on layperson evaluation of science. Most of the time, the best thing is just to talk to people. (Cold emails are OK; be polite, concise, and ask a specific question. Grad students tend to be generous with their time if you have an interesting question or pizza and beer. And I’m glad to answer physics questions by LW message.)
Short of talking to people, you can often find blogs in the field of interest. More rarely, you can also find good journalism doing the above kind of work for you. (Quanta is typically good in physics, enough so that I more or less trust them on other subjects.)
There’s plenty to be said about primary source evaluation, which varies with field and which the other answers so far get at, but I think this lesson needs to come first.
Recapitulating something I’ve written about before:
You should first make a serious effort to formulate both the specific question you want answered, and why you want an answer. It may turn out surprisingly often that you don’t need to do all this work to evaluate the study.
Short of becoming an expert yourself, your best bet is then to learn how to talk to people in the field until you can understand what they think about the paper and why—and also how they think and talk about these things. This is roughly what Harry Collins calls “interactional” expertise. (He takes gravitational-wave scientist Joe Weber’s late work as an especially vivid example: “I can promise such lay readers that if they teach themselves a bit of elementary statistics and persevere with reading the paper, they will find it utterly convincing. Scientific papers are written to be utterly convincing; over the centuries their special language and style has been developed to make them read convincingly.… The only way to know that Weber’s paper is not to be read in the way it is written is to be a member of the ‘oral culture’ of the relevant specialist community.” The full passage is very good.)
If you only learn from papers (or even textbooks and papers), you won’t have any idea what you’re missing. A lot of expertise is bound up in individual tacit knowledge and group dynamics that never get written down. This isn’t to say that the ‘oral culture’ is always right, but if you don’t have a good grasp of it, you will make at best slow progress as an outsider.
This is the main thing holding me back from running the course I’ve half-written on layperson evaluation of science. Most of the time, the best thing is just to talk to people. (Cold emails are OK; be polite, concise, and ask a specific question. Grad students tend to be generous with their time if you have an interesting question or pizza and beer. And I’m glad to answer physics questions by LW message.)
Short of talking to people, you can often find blogs in the field of interest. More rarely, you can also find good journalism doing the above kind of work for you. (Quanta is typically good in physics, enough so that I more or less trust them on other subjects.)
There’s plenty to be said about primary source evaluation, which varies with field and which the other answers so far get at, but I think this lesson needs to come first.