My composition teacher in college told me that in some pottery schools, the teacher holds up your pot, examines it, comments on it, and then smashes it on the floor. They do this for your first 100 pots.
In that spirit, this post's epistemic status is SMASH THIS POT.
When I’m doing research, I often start with a vague question based on an unsettled emotion, such as excitement, confusion, frustration, or need. Lately, I’ve been investigating visual imagery. I start off with a question that’s as hazy as “should I try and train my visual imagination so that I can be a better scientist?”
All too often, when I leap into searching Google Scholar with just a hazy question in mind, I get lost in a rabbit hole. My mind substitutes alternative metrics of success. How many papers have I read, and to what depth? What do they say, in general? How many papers can I find? How should I group them? Where to start research? Elizabeth think’s it’s with a question, not a book. Letting my chance findings on Google Scholar define my research agenda seems wrong, an example of lost purposes.
After taking detailed in-the-moment notes on how I investigated this question, I extricated some structure for my agile research process.
During this process, I’m glancing at titles, looking for perhaps 1-2 minutes at an abstract, or spending maybe 5 minutes looking at the text of an article.
1) Note the hazy idea that sparks your curiosity. Eg. “should I try and train my visual imagination?”
2) Try to decompose that into some sub-questions, which might require a paragraph or so to specify. Here are some examples:
I know we have different visual faculties for processing surface details vs. spatial relationships. People may have more developed skills or stronger innate potential for one or the other. Do most people tend to be average in both, or is it common to be much stronger in one than the other? Does one or the other correspond to feeling like you have a more vivid imagination?
Can visual imagination be improved with practice? If so, how?
What’s more helpful to a scientist/mathematician/engineer—a strong visual imagination, or skill in using visualization tools such as drawings, physical models, and software?
What’s more helpful to a scientist—a strong internalized knowledge of various skills, or knowing how to navigate a wide range of reference texts? I imagine investigating this question by testing them to see how well they can remember the content of textbooks or papers they’ve consulted in the last year. How much of what a scientist is doing is actually internalizing the content of a paper, versus doing an epistemic check on whether to accept the paper’s conclusions and just remembering a few sentence’s worth of actual content?
3) Search for a likely phrase on Google Scholar. Eg. “visual imagination.” Take away new questions from the titles that pop up. For example, “Four measures of visual imagination” made me ask how we’d even study and measure this phenomenon. Are the methods likely to produce useful evidence? Continue searching for related terms, building up the questions you’re interested in.
4) As you go along, bring your intuition to bear. Consider how the questions you identify are related. Write down your motivations for doing this research. Think about whether the idea you have is plausible. For example, I considered whether it seemed reasonable that the ability to see and control a mental picture is a key skill for STEM scientists, in an era of widely available visualization tools.
5) Note when superficially related questions are not actually relevant to your inquiry. For example, when I started finding papers on people’s ability to see creative visual patterns in a shadowy blob (sort of like a Rorschach), I realized that I needed to specify that I was interested in the ability to see and control a mental picture, not the ability to find creative associations in an image. Altering my search to “visual imagery” rather than “visual imagination” turned up more relevant papers. This process will help you define the domain of your inquiry.
6) Defining the domain of your research project should eventually allow you to prioritize research questions. In my project, I realized I was interested in two issues. First, is the ability to see and control a mental picture a broadly important skill for STEM scientists? If so, how does mental imagery work, and how can it be improved? Realizing that the latter question didn’t matter much to me if the answer to the first question was “no” helped me set aside many of the papers that popped up.
7) Prioritizing your research questions may lead you to realize that there’s preliminary research you should do before you even address the question you started with. In my case, I want to identify the most useful things I can do to build a career in science. Intuitively, is a big literature review to determine whether and how I should practice visualization a high priority, compared with things like studying for my classes, developing my research project, contacting grad programs, and so on? Maybe I should just keep practicing it intuitively and organically, rather than buttressing it with scientific research.
The outcome is a domain of specific, related research questions that are strongly linked with your original motivation to do research, and clarity on whether this is the right question-domain to pursue given your needs.
I think of this process as an example of “pre-research.”
Pre-research can take many forms. Sometimes, I create a spreadsheet to intuitively define large-scale possible directions of research before I do even a single search for literature. Other projects, like this one, involve much more back-and-forth between search and reflection.
The point is to prevent the books you’re finding from defining your agenda. Use the books, don’t let them use you. Only when your domain is defined should you move on to the next step of scholarship, which you should also figure out how to do efficiently.
One important outcome of pre-research is that if you commit to actually reading a paper or book, you know quite clearly why you’re doing so. It’s not for the purpose of open-ended exploration. It’s to get answers to a set of specific questions that are high-priority within your research domain. This way, a literature is less like running around the field and falling down a rabbit hole, and more like walking around the field and considering whether, why, and where to build your ivory tower.
An agile approach to pre-research
When I’m doing research, I often start with a vague question based on an unsettled emotion, such as excitement, confusion, frustration, or need. Lately, I’ve been investigating visual imagery. I start off with a question that’s as hazy as “should I try and train my visual imagination so that I can be a better scientist?”
All too often, when I leap into searching Google Scholar with just a hazy question in mind, I get lost in a rabbit hole. My mind substitutes alternative metrics of success. How many papers have I read, and to what depth? What do they say, in general? How many papers can I find? How should I group them? Where to start research? Elizabeth think’s it’s with a question, not a book. Letting my chance findings on Google Scholar define my research agenda seems wrong, an example of lost purposes.
After taking detailed in-the-moment notes on how I investigated this question, I extricated some structure for my agile research process.
During this process, I’m glancing at titles, looking for perhaps 1-2 minutes at an abstract, or spending maybe 5 minutes looking at the text of an article.
1) Note the hazy idea that sparks your curiosity. Eg. “should I try and train my visual imagination?”
2) Try to decompose that into some sub-questions, which might require a paragraph or so to specify. Here are some examples:
I know we have different visual faculties for processing surface details vs. spatial relationships. People may have more developed skills or stronger innate potential for one or the other. Do most people tend to be average in both, or is it common to be much stronger in one than the other? Does one or the other correspond to feeling like you have a more vivid imagination?
Can visual imagination be improved with practice? If so, how?
What’s more helpful to a scientist/mathematician/engineer—a strong visual imagination, or skill in using visualization tools such as drawings, physical models, and software?
What’s more helpful to a scientist—a strong internalized knowledge of various skills, or knowing how to navigate a wide range of reference texts? I imagine investigating this question by testing them to see how well they can remember the content of textbooks or papers they’ve consulted in the last year. How much of what a scientist is doing is actually internalizing the content of a paper, versus doing an epistemic check on whether to accept the paper’s conclusions and just remembering a few sentence’s worth of actual content?
3) Search for a likely phrase on Google Scholar. Eg. “visual imagination.” Take away new questions from the titles that pop up. For example, “Four measures of visual imagination” made me ask how we’d even study and measure this phenomenon. Are the methods likely to produce useful evidence? Continue searching for related terms, building up the questions you’re interested in.
4) As you go along, bring your intuition to bear. Consider how the questions you identify are related. Write down your motivations for doing this research. Think about whether the idea you have is plausible. For example, I considered whether it seemed reasonable that the ability to see and control a mental picture is a key skill for STEM scientists, in an era of widely available visualization tools.
5) Note when superficially related questions are not actually relevant to your inquiry. For example, when I started finding papers on people’s ability to see creative visual patterns in a shadowy blob (sort of like a Rorschach), I realized that I needed to specify that I was interested in the ability to see and control a mental picture, not the ability to find creative associations in an image. Altering my search to “visual imagery” rather than “visual imagination” turned up more relevant papers. This process will help you define the domain of your inquiry.
6) Defining the domain of your research project should eventually allow you to prioritize research questions. In my project, I realized I was interested in two issues. First, is the ability to see and control a mental picture a broadly important skill for STEM scientists? If so, how does mental imagery work, and how can it be improved? Realizing that the latter question didn’t matter much to me if the answer to the first question was “no” helped me set aside many of the papers that popped up.
7) Prioritizing your research questions may lead you to realize that there’s preliminary research you should do before you even address the question you started with. In my case, I want to identify the most useful things I can do to build a career in science. Intuitively, is a big literature review to determine whether and how I should practice visualization a high priority, compared with things like studying for my classes, developing my research project, contacting grad programs, and so on? Maybe I should just keep practicing it intuitively and organically, rather than buttressing it with scientific research.
The outcome is a domain of specific, related research questions that are strongly linked with your original motivation to do research, and clarity on whether this is the right question-domain to pursue given your needs.
I think of this process as an example of “pre-research.”
Pre-research can take many forms. Sometimes, I create a spreadsheet to intuitively define large-scale possible directions of research before I do even a single search for literature. Other projects, like this one, involve much more back-and-forth between search and reflection.
The point is to prevent the books you’re finding from defining your agenda. Use the books, don’t let them use you. Only when your domain is defined should you move on to the next step of scholarship, which you should also figure out how to do efficiently.
One important outcome of pre-research is that if you commit to actually reading a paper or book, you know quite clearly why you’re doing so. It’s not for the purpose of open-ended exploration. It’s to get answers to a set of specific questions that are high-priority within your research domain. This way, a literature is less like running around the field and falling down a rabbit hole, and more like walking around the field and considering whether, why, and where to build your ivory tower.