I wanted to let you know about a recent article in New Idea in Psychology written by myself and Dr. Dario Krpan (assistant professor of psychology at London School of Economics) that discusses LessWrong and Scott Alexander (Siskind) as examples of amateurs (or communities of amateurs) that make valuable contributions to psychology. In the article, we argue that psychology and behavioral science can benefit from increased participation in knowledge work (hypothesizing, experimentation, observational research, etc.) by amateurs. We highlight several “blind spots” in academic psychology (long-term projects, observational research, speculation, interdisciplinary research, taboo or uncommon subjects, and aimless projects) that amateurs might profitably focus on and discuss how we can support and facilitate amateurs to do research in psychology. One point of clarification is the term “amateur”; this is not meant as derogatory in anyway (and indeed our title makes light of the denigrating phrase “amateur hour”), but is used to refer to anyone that is not a professional psychology researcher. Scott, although he is practicing psychiatrist, would still qualify as an amateur in our definition because he does not belong to an institution that pays him to do psychology research. I would also mention that this paper might the first in which an author lists his substack as his institution (Secretum Secretorum).
Below is the paragraph in which we discuss LessWrong.
Lastly, an example of a “quantified-self” amateur who has made valuable contributions in PBS is Alexey Guzey (2020), an independent researcher noted for conducting a self-experiment on the link between sleep and cognitive functioning. Guzey (2019) is also known for performing a rigorous fact-checking of the book Why We Sleep (Walker, 2017), which suggests that amateurs can also improve PBS by conducting thorough reviews of popular science books. Both reviews of popular science books and self-experimentation by amateurs can also be commonly found on LessWrong.com, a hub of the rationalist community; for example, one can find posts that detail self-experiments on the effect of chocolate on sleep, metacognitive training (e.g., using heuristics, noticing emotions), the relationship between work output and hours of work, or romantic techniques.2 Amateur self-experimentation has a long history in PBS, beginning perhaps with Herman Ebbinghaus’ ground-breaking work on memory that led to the discovery of the forgetting curve. Though he would eventually gain recognition as an academic psychologist, at the time of his experiments Ebbinghaus was an amateur—he did not have a university position and wanted to advance psychological knowledge by researching himself (Boneau, 1998; Slamecka, 1985; Woodworth, 1909).
In another section, we mention a specific post as an example of a long-term observational research project.
For example, the “slow scholarship” movement highlights how scholars face a general intensification in the pace of work and an increasing pressure to publish (Harland, 2016; Hartman & Darab, 2012). Research indicates that the average number of publications at time of hiring for science faculty positions has been steadily rising in recent years (Pennycook & Thompson, 2018; Reinero, 2019; Van Dijk, Manor, & Carey, 2014); trends like this may influence researchers, especially early career researchers, away from projects that require dedication over a long period of time. This suggests that long-term research projects are generally a neglected area in academia (i.e., a blind spot), and amateurs could do valuable work by focusing their efforts on research that may take a significant amount of time to yield results (Table 1) (Medin et al., 2017). This may involve spending decades to build rich and multilayered psychological theories, investigating psychological phenomena in greater detail, or conducting long-term observation. One example of an amateur conducting a long-term project in PBS is the post “Seven Years of Spaced Repetition Software in the Classroom” by user tanagrabeast (2015) on LessWrong.com, who investigated how spaced repetition of study material influenced high-school students’ academic performance.
I also wanted to let LessWrong know about a new scientific journal founded by myself and Dr. Krpan that grew out of discussion for this paper. Seeds of Science (theseedsofscience.org) publishes short scientific articles that are more speculative or non-traditional in some way. Peer review is community-based voting and commenting by our diverse network of “gardeners” from across science (the journal is 100% free and participation by gardeners is entirely at will). Our primary criterion is simple: does your article contain original ideas that have the potential to advance science? The goal is to be as open-minded as possible about what qualifies as a useful scientific contribution while also allowing for a diversity of writing styles and formats so that authors can express their ideas clearly and in an engaging manner. The openness of our format and the limited submission requirements (no cumbersome formatting rules) are designed to make the writing and reading of our papers a much easier and more enjoyable process than is typical for most scientific journals. You can read more about our criteria on the “How to Publish” page.
In some ways what we are trying to do is not too different from LessWrong – we want to create a community of intelligent like-minded individuals dedicated to writing and reviewing articles. The main differences between LessWrong and Seeds of Science are the general focus (we are more narrowly focused on science), the review structure (our review process is more formalized, we are not a forum), and the fact that we publish papers with DOIs and scholarly formatting. In particular, we hope to provide a platform for undergraduates, graduates, and amateurs to publish some of their more unusual ideas in a less restrictive format. It’s definitely an experiment in scientific publishing, but hopefully we can carve out a unique niche somewhere between rigorous academic journal and blog/forum.
We have published 2 articles so far and have also written two examples articles, one of which – Randomness in Science – I recently posted here. I’m happy to answer any questions about the journal (or the paper) and of course it goes without saying that we would love to have any of you join us as authors or gardeners. Again, it is free to join as a gardener and participation is entirely at will – we send you articles through email (think substack) and you can vote/comment or ignore without notification. Another unique feature of SoS is that we publish particularly interesting or helpful comments after the main text of the article.
LessWrong discussed in New Ideas in Psychology article
Hi LessWrong,
I wanted to let you know about a recent article in New Idea in Psychology written by myself and Dr. Dario Krpan (assistant professor of psychology at London School of Economics) that discusses LessWrong and Scott Alexander (Siskind) as examples of amateurs (or communities of amateurs) that make valuable contributions to psychology. In the article, we argue that psychology and behavioral science can benefit from increased participation in knowledge work (hypothesizing, experimentation, observational research, etc.) by amateurs. We highlight several “blind spots” in academic psychology (long-term projects, observational research, speculation, interdisciplinary research, taboo or uncommon subjects, and aimless projects) that amateurs might profitably focus on and discuss how we can support and facilitate amateurs to do research in psychology. One point of clarification is the term “amateur”; this is not meant as derogatory in anyway (and indeed our title makes light of the denigrating phrase “amateur hour”), but is used to refer to anyone that is not a professional psychology researcher. Scott, although he is practicing psychiatrist, would still qualify as an amateur in our definition because he does not belong to an institution that pays him to do psychology research. I would also mention that this paper might the first in which an author lists his substack as his institution (Secretum Secretorum).
Here is the official link to the paper: https://www.sciencedirect.com/science/article/abs/pii/S0732118X21000714
Here is the author copy which will be open access for the next 40 days or so:
https://authors.elsevier.com/c/1eAFI,GfNqbOWA
Below is the paragraph in which we discuss LessWrong.
In another section, we mention a specific post as an example of a long-term observational research project.
I also wanted to let LessWrong know about a new scientific journal founded by myself and Dr. Krpan that grew out of discussion for this paper. Seeds of Science (theseedsofscience.org) publishes short scientific articles that are more speculative or non-traditional in some way. Peer review is community-based voting and commenting by our diverse network of “gardeners” from across science (the journal is 100% free and participation by gardeners is entirely at will). Our primary criterion is simple: does your article contain original ideas that have the potential to advance science? The goal is to be as open-minded as possible about what qualifies as a useful scientific contribution while also allowing for a diversity of writing styles and formats so that authors can express their ideas clearly and in an engaging manner. The openness of our format and the limited submission requirements (no cumbersome formatting rules) are designed to make the writing and reading of our papers a much easier and more enjoyable process than is typical for most scientific journals. You can read more about our criteria on the “How to Publish” page.
In some ways what we are trying to do is not too different from LessWrong – we want to create a community of intelligent like-minded individuals dedicated to writing and reviewing articles. The main differences between LessWrong and Seeds of Science are the general focus (we are more narrowly focused on science), the review structure (our review process is more formalized, we are not a forum), and the fact that we publish papers with DOIs and scholarly formatting. In particular, we hope to provide a platform for undergraduates, graduates, and amateurs to publish some of their more unusual ideas in a less restrictive format. It’s definitely an experiment in scientific publishing, but hopefully we can carve out a unique niche somewhere between rigorous academic journal and blog/forum.
We have published 2 articles so far and have also written two examples articles, one of which – Randomness in Science – I recently posted here. I’m happy to answer any questions about the journal (or the paper) and of course it goes without saying that we would love to have any of you join us as authors or gardeners. Again, it is free to join as a gardener and participation is entirely at will – we send you articles through email (think substack) and you can vote/comment or ignore without notification. Another unique feature of SoS is that we publish particularly interesting or helpful comments after the main text of the article.