I can share my personal experience, but I’m an outlier on many axes.
I identify strongly with a lot of the things you wrote about liking tests.
I studied for and took the LSAT in the mid-2000s. I scored either a 166 or a 168 (can’t remember) on my first practice exam, which I took cold. Nearly all of the points I lost were on the games section.
Incidentally, this was a pretty emotionally difficult experience for me, because I had never done this poorly on a 200-800 (120-180) scored standardized test before. This one felt especially like an IQ test, so not being able to crack the 700 (170) barrier was very threatening to my self-esteem.
Your guess that most LSAT prep focuses on developing written systems for the games section is correct. Powerscore was the most highly regarded producer of LSAT prep materials for high scorers when I took the LSAT, and their reputation was larlgey due to their succeess in teaching people how to create accurate and efficient diagrams for the games. I think they also produced fairly well-regarded materials for the other sections too, but I didn’t investigate those because my deficiency was the games section.
My preparation consisted of working through the Powerscore games materials and taking lots of practice tests. LSAC published old LSATs in books of ten tests each, and three such books had been published at the time I was studying, so I bought them all and just took those 30 LSATs with a stopwatch. I ran out of old LSATs prior to the test, so I went online and found a torrent of dubiously legal low-quality scans of more recent LSATs, though I only had time to take one of those before the exam because the school year started.
On the logical reasoning and reading comprehension sections, I would guess that my mean missed questions per section prior to studying was about 3, with a standard deviation of about 1. After studying, it was probably about a mean of 2 with a standard deviation of maybe 0.7. I typically missed questions because I would read too quickly and miss something subtle and important. I did not improve significantly at this, even though I did a lot of practice. I did not use symbolic logic or any other systematized written system to answer any logical reasoning or reading comprehension questions; it’s possible that this would have helped.
On the games sections, my pre-studying mean missed questions per section was probably about 10, with a standard deviation of 3. After studying, the mean was probably 1 with a standard deviation of 1. Games went from being my weakest to my strongest section once I learned how to make and use good diagrams, and it became a fairly regular occurrence for me to finish the entire section with time to spare and complete certainty in the correctness of all of my answers.
Doing some of the older tests before I mastered the technique for the games, I scored as low as, IIRC, 158. This resulted in much hyperventilating and cruel self-talk. As I improved at the games and worked my way up through more recent tests, my scores improved to the point where I was consistently in the 170s, and achieved a few 180s. That one torrented practice test that I took the week before the actual test was the last practice test I took, and I got a 169 on it somehow, which was, of course, terrible for my confidence going into the exam.
I scored a 175 on the actual test.
The entire reason the LSAT is hard is the time pressure. I think that, given enough time, any 150s-scorer could arrive at the right answer to any LSAT question. I agree with ThisSpaceAvailable that the games section is very much a “can-you-follow-instructions-under-time-pressure” test, and my experience was like ioshva’s in that my missed logical reasoning questions were due to lapses in focus while reading under time pressure rather than inability to grasp the logic.
I do not believe the process of studying for the LSAT inculcated any rationalist habits of mind in me. For me, because of my strengths and weaknesses, the process of studying for the LSAT was largely the process of learning how to complete contrived little puzzles that are isomorphic to sudoku. I suppose there might be some Lumosity-type minor positive effect on working memory, but I doubt the LSAT is much better for that than any other mentally engaging task. The research you cited in the OP just makes me want a control group of people playing Lumosity games. I can imagine that, for someone whose weakness was the logical reasoning section and who was missing lots of logical reasoning questions because of a deficit in understanding of the basic mechanics of logic, studying for the LSAT would be one way to correct that deficiency. But I suspect that ioshva is correct in that just learning logic directly would be a more effective way of achieving the same result.
I do not believe the process of studying for the LSAT helped me in law school. I think the correlation between LSAT score and first-year law school GPA exists largely because high-IQ and high-conscientiousness people tend to do well on both. I performed abysmally in law school despite my relatively high LSAT score because of pathologically low conscientiousness. I was only able to overcome low conscientiousness during the LSAT preparation process because I was living with my parents at the time and they put considerable pressure on me to study hard.
LSAT score is certainly a huge factor in law school admissions, and I was admitted to a moderately prestigious law school solely on the strength of my LSAT score and in spite of a truly awful undergraduate GPA. In general, LSAT score, undergraduate GPA, and underrepresented racial minority status are really the only meaningful factors in the law school admission process. This holds except in weird edge cases where people have distinguished themselves in exceptional ways, like former pro athletes or people who have founded successful businesses or nonprofits. Things like recommendation letters and essays can potentially keep you out if they’re bad, but will not get you in except, again, in weird edge cases like having a Supreme Court justice write you a recommendation insisting that you’re the brightest mind of your generation.
Incidentally, on the “think like a lawyer” point raised by MaximumLiberty—in my experience, this usually refers to a couple of specific analytic skills that lawyers are trained to develop: (1) reasoning by analogy and distinction, as with case law analysis, and (2) thinking strategically about the way systems of rules constrain or fail to constrain the behavior of agents within them, as with advising a client on how to avoid punishment. I’m not sure that this sort of software specialization would be reflected in the hardware imaged by an fMRI.
I wanted to thank you for your response and to apologize for not getting to read it sooner (I am in the throes of final exams, project due dates, et c.). The Lumosity control group or some similar intervention is a great idea and probably the only way to know for sure if LSAT prep had any unique effects.
I can share my personal experience, but I’m an outlier on many axes.
I identify strongly with a lot of the things you wrote about liking tests.
I studied for and took the LSAT in the mid-2000s. I scored either a 166 or a 168 (can’t remember) on my first practice exam, which I took cold. Nearly all of the points I lost were on the games section.
Incidentally, this was a pretty emotionally difficult experience for me, because I had never done this poorly on a 200-800 (120-180) scored standardized test before. This one felt especially like an IQ test, so not being able to crack the 700 (170) barrier was very threatening to my self-esteem.
Your guess that most LSAT prep focuses on developing written systems for the games section is correct. Powerscore was the most highly regarded producer of LSAT prep materials for high scorers when I took the LSAT, and their reputation was larlgey due to their succeess in teaching people how to create accurate and efficient diagrams for the games. I think they also produced fairly well-regarded materials for the other sections too, but I didn’t investigate those because my deficiency was the games section.
My preparation consisted of working through the Powerscore games materials and taking lots of practice tests. LSAC published old LSATs in books of ten tests each, and three such books had been published at the time I was studying, so I bought them all and just took those 30 LSATs with a stopwatch. I ran out of old LSATs prior to the test, so I went online and found a torrent of dubiously legal low-quality scans of more recent LSATs, though I only had time to take one of those before the exam because the school year started.
On the logical reasoning and reading comprehension sections, I would guess that my mean missed questions per section prior to studying was about 3, with a standard deviation of about 1. After studying, it was probably about a mean of 2 with a standard deviation of maybe 0.7. I typically missed questions because I would read too quickly and miss something subtle and important. I did not improve significantly at this, even though I did a lot of practice. I did not use symbolic logic or any other systematized written system to answer any logical reasoning or reading comprehension questions; it’s possible that this would have helped.
On the games sections, my pre-studying mean missed questions per section was probably about 10, with a standard deviation of 3. After studying, the mean was probably 1 with a standard deviation of 1. Games went from being my weakest to my strongest section once I learned how to make and use good diagrams, and it became a fairly regular occurrence for me to finish the entire section with time to spare and complete certainty in the correctness of all of my answers.
Doing some of the older tests before I mastered the technique for the games, I scored as low as, IIRC, 158. This resulted in much hyperventilating and cruel self-talk. As I improved at the games and worked my way up through more recent tests, my scores improved to the point where I was consistently in the 170s, and achieved a few 180s. That one torrented practice test that I took the week before the actual test was the last practice test I took, and I got a 169 on it somehow, which was, of course, terrible for my confidence going into the exam.
I scored a 175 on the actual test.
The entire reason the LSAT is hard is the time pressure. I think that, given enough time, any 150s-scorer could arrive at the right answer to any LSAT question. I agree with ThisSpaceAvailable that the games section is very much a “can-you-follow-instructions-under-time-pressure” test, and my experience was like ioshva’s in that my missed logical reasoning questions were due to lapses in focus while reading under time pressure rather than inability to grasp the logic.
I do not believe the process of studying for the LSAT inculcated any rationalist habits of mind in me. For me, because of my strengths and weaknesses, the process of studying for the LSAT was largely the process of learning how to complete contrived little puzzles that are isomorphic to sudoku. I suppose there might be some Lumosity-type minor positive effect on working memory, but I doubt the LSAT is much better for that than any other mentally engaging task. The research you cited in the OP just makes me want a control group of people playing Lumosity games. I can imagine that, for someone whose weakness was the logical reasoning section and who was missing lots of logical reasoning questions because of a deficit in understanding of the basic mechanics of logic, studying for the LSAT would be one way to correct that deficiency. But I suspect that ioshva is correct in that just learning logic directly would be a more effective way of achieving the same result.
I do not believe the process of studying for the LSAT helped me in law school. I think the correlation between LSAT score and first-year law school GPA exists largely because high-IQ and high-conscientiousness people tend to do well on both. I performed abysmally in law school despite my relatively high LSAT score because of pathologically low conscientiousness. I was only able to overcome low conscientiousness during the LSAT preparation process because I was living with my parents at the time and they put considerable pressure on me to study hard.
LSAT score is certainly a huge factor in law school admissions, and I was admitted to a moderately prestigious law school solely on the strength of my LSAT score and in spite of a truly awful undergraduate GPA. In general, LSAT score, undergraduate GPA, and underrepresented racial minority status are really the only meaningful factors in the law school admission process. This holds except in weird edge cases where people have distinguished themselves in exceptional ways, like former pro athletes or people who have founded successful businesses or nonprofits. Things like recommendation letters and essays can potentially keep you out if they’re bad, but will not get you in except, again, in weird edge cases like having a Supreme Court justice write you a recommendation insisting that you’re the brightest mind of your generation.
Incidentally, on the “think like a lawyer” point raised by MaximumLiberty—in my experience, this usually refers to a couple of specific analytic skills that lawyers are trained to develop: (1) reasoning by analogy and distinction, as with case law analysis, and (2) thinking strategically about the way systems of rules constrain or fail to constrain the behavior of agents within them, as with advising a client on how to avoid punishment. I’m not sure that this sort of software specialization would be reflected in the hardware imaged by an fMRI.
I wanted to thank you for your response and to apologize for not getting to read it sooner (I am in the throes of final exams, project due dates, et c.). The Lumosity control group or some similar intervention is a great idea and probably the only way to know for sure if LSAT prep had any unique effects.