A year ago, I started trying to deliberate practice skills that would “help people figure out the answers to confusing, important questions.” I experimented with Thinking Physics questions, GPQA questions, Puzzle Games , Strategy Games, and a stupid twitchy reflex game I had struggled to beat for 8 years[1]. Then I went back to my day job and tried figuring stuff out there too.
The most important skill I was trying to learn was Metastrategic Brainstorming – the skill of looking at a confusing, hopeless situation, and nonetheless brainstorming useful ways to get traction or avoid wasted motion.
Normally, when you want to get good at something, it’s great to stand on the shoulders of giants and copy all the existing techniques. But this is challenging if you’re trying to solve important, confusing problems because there probably isn’t (much) established wisdom on how to solve it. You may need to discover techniques that haven’t been invented yet, or synthesize multiple approaches that haven’t previously been combined. At the very least, you may need to find an existing technique buried in the internet somewhere, which hasn’t been linked to your problem with easy-to-search keywords, without anyone to help you.
In the process of doing this, I found a few skills that came up over and over again.
I didn’t invent the following skills, but I feel like I “won” them in some sense via a painstaking “throw myself into the deep end” method. I feel slightly wary of publishing them in a list here, because I think it was useful to me to have to figure out for myself that they were the right tool for the job. And they seem like kinda useful “entry level” techniques, that you’re more likely to successfully discover for yourself.
But, I think this is hard enough, and forcing people to discover everything for themselves seems unlikely to be worth it.
The skills that seemed most general, in both practice and on my day job, are:
Taking breaks/naps
Working Memory facility
Patience
Knowing what confusion/deconfusion feels like
Actually Fucking Backchain
Asking “what is my goal?”
Having multiple plans
There were other skills I already was tracking, like Noticing, or Focusing. There were also somewhat more classic “How to Solve It” style tools for breaking down problems. There are also a host of skills I need when translating this all into my day-job, like “setting reminders for myself” and “negotiating with coworkers.”
But the skills listed above feel like they stood out in some way as particularly general, and particularly relevant for “solve confusing problems.”
Taking breaks, or naps
Difficult intellectual labor is exhausting. During the two weeks I was working on solving Thinking Physics problems, I worked for like 5 hours a day and then was completely fucked up in the evenings. Other researchers I’ve talked to report similar things.
During my workshops, one of the most useful things I recommended people was “actually go take a nap. If you don’t think you can take a real nap because you can’t sleep, go into a pitch black room and lie down for awhile, and the worst case scenario is your brain will mull over the problem in a somewhat more spacious/relaxed way for awhile.”
Practical tips: Get yourself a sleeping mask, noise machine (I prefer a fan or air purifier), and access to a nearby space where you can rest. Leave your devices outside the room.
Working Memory facility
Often a topic feels overwhelming. This is often because it’s just too complicated to grasp with your raw working memory. But, there are various tools (paper, spreadsheets, larger monitors, etc) that can improve this. And, you can develop the skill of noticing “okay this isn’t fitting in my head, or even on my big monitor – what would let it fit in my head?”.
The “eye opening” example of this for me was trying to solve a physics problem that included 3 dimensions (but one of the dimensions was “time”). I tried drawing it out but grasping the time-progression was still hard. I came up with the idea of using semi-translucent paper, where I would draw a diagram of what each step looked like on separate pages, and then I could see where different elements were pointed.
I’ve also found “spreadsheet literacy” a recurring skill – google sheets is very versatile but you have to know what all the functions are, have a knack for arranging elements in an easy-to-parse way, etc.
Practical Tips: Have lots of kinds of paper, whiteboards and writing supplies around.
On google sheets:
You can make collapsible sections, which help with making complex models while also being able to hide away the complexity of sub-parts you aren’t modeling. (hotkey: alt-shift-rightarrow)
switch between “display formulas” and the default “display the result” mode (hotkey: ctrl-backtic)
Patience
If I’m doing something confusingly hard, there are times when it feels painful to sit with it, and I’m itchy to pick some solution and get moving. This comes up in two major areas:
Deliberate/purposeful practice. A key thing here is to be practicing the form perfectly, which requires somehow slowing things down such that you have time to get each moment correct. The urge to rush can undo the practice you just did, by training mistakes, or prevent you from actually successfully practicing at all.
Launching into a plan, or declaring yourself done, when you are still confused. Sitting with the uncomfortableness feels very itchy. But vague plans can be completely wrong, resting on confused assumptions.
There is of course a corresponding virtue of “just get moving, build up momentum and start learning through iteration.” The wisdom to tell the difference between “I’m still confused and need to orient more” and “I need to get moving” is important. But, an important skill there is at least being capable of sitting with impatient discomfort, in the situations where that’s the right call.
Practical tips: I dunno I still kinda suck at this one, but I find taking deep breaths, and deliberately reminding myself “Slow is smooth, smooth is fast”.
Know what deconfusion, or “having a crisp understanding” feels like
A skill from both Thinking Physics and Baba is You.
When I first started Thinking Physics, I would get to a point where “I dunno, I feel pretty sure, and I can’t think of more things to do to resolve my confusion”, and then impatiently roll the dice on checking the answer. Sometimes I’d be right, more often I’d be wrong.
Eventually I had a breakthrough where I came up with a crisp model of the problem, and was like “oh, man, now it would actually be really surprising if any of the other answers were true.” From then on… well, I’d still sometimes got things wrong (mostly due to impatience). But, I could tell when I still had pieces of my model that were vague and unprincipled.
Similarly in Baba is You: when people don’t have a crisp understanding of the puzzle, they tend to grasp and straws and motivatedly-reason their way into accepting sketchy sounding premises. But, the true solution to a level often feels very crisp and clear and inevitable.
Learning to notice this difference in qualia is quite valuable.
Practical tips: This is where Noticing and Focusingare key, but are worthwhile for helping you notice subtle differences in how an idea feels in your mind.
Try either making explicit numerical predictions about whether you’ve solved an exercise before you look up the answer; or, write down a qualitative sentence like “I feel like I really deeply understand the answer” or “this seems probably right but I feel some niggling doubts.”
Actually Fucking Backchain
From Baba is You, I got the fear-of-god put in me seeing how easy it was to spin my wheels, tinkering around with stuff that was nearby/accessible/easy-to-iterate-with, and how that often turned out to not be at all relevant to beating a level.
I had much less wasted motion when I thought through “What would the final stages of beating this level need to look like? What are the stages just before those?”, and focusing my attention on things that could help me get to that point.
One might say “well, Baba is You is a game optimized for being counterintuitive and weird.” I think for many people with a goal like “build a successful startup”, it can sometimes be fine to just be forward chaining with stuff that feels promising, rather than trying to backchain from complex goals.
But, when I eyeball the realworld problems I’m contending with (i.e. x-risk) they really do seem like there’s a relatively narrow set of victory conditions that plausibly work. And, many of the projects I feel tempted to start don’t actually really seem that relevant.
(I also think great startup founders are often doing a mix of forward and backward chaining. i.e. I bet Jeff Bezos was like “okay I bet I could make an online bookstore that worked”, was also thinking “but, what if I ultimately wanted the Everything Store? What are obstacles that I’d eventually need to deal”)
Practical tips: First, come up with at least one concrete story of what the world would look like, if you succeeded at your goals. Try hard to come up with 2 other worlds, so you aren’t too anchored on your first idea.
Then, try to concretely imagine the steps that would come a little bit earlier in the chain from the end.
Don’t worry about mapping out all the different possible branches of the future (that’s impossible). But, for a complex plan, have at least one end-to-end plan that connects all the dots from the resources you have now to the victory condition at the end.
Meanwhile, while doing most of your work, notice when it starts to feel like you’ve lost the plot (try just making a little tally-mark whenever you notice yourself rabbitholing in a way that feels off). And ask “what is my goal? is what I’m currently doing helping”
Ask “What’s My Goal?”
Actually, having just written the previous section, I’m recalling a simpler, more commonly useful skill, which is simply to ask “what is my goal?”.
Often, doing this throws into relief that you’re not sure what your goal is. Sometimes, asking the question immediately prompts me to notice a key insight I’d been glossing over.
If you’re not sure what your goal is, try babbling some things that seem like they might be a goal, and then ask yourself “does this feel like what I’m most trying to achieve right now?”
It’s okay if it turns out your goal is different or more embarrassing-sounding than you thought. You might say “Actually, you know what? I do care more about showing off and sounding smart, than actually learning something right now.” (But, you might also realize “okay, I separately care about learning something and sounding smart”, and then be more intentional about finding a tactic that accomplishes both)
Once you remember (or figure out) your goal, as you brainstorm strategies, ask yourself “would I be surprised if this didn’t help me achieve my goals?”, and then prioritize strategies that you viscerally expect to work.
This one is important enough to be it’s own post. (I guess, probably most of these are important enough to be a full post? But, this one especially)
But, listing here for completeness:
Whether you are solving a puzzle, or figuring out how to solve a puzzle, or deciding what your team should do next week, try to have multiple hypotheses. (I usually say “try to have at least 3 plans”, but a plan is basically a special case – a hypothesis about “doing X is the best way to achieve goal Y”).
They each need to be a hypothesis you actually believe in.
I say “at least 3”, because I think it gets you “fully intellectually agile.” If you only have one plan, it’s easy to get tunnel vision on it and not notice that it’s doomed. Two ideas helps free up your mind, but then you might still evaluate all evidence in terms of “does this support idea 1 or idea 2?”. If you have 3 different hypotheses, it’s much more natural to keep generating more hypotheses, and to pivot around in a multiple dimensional space of possibility.
This wasn’t practice for “solving confusing problems”, but it was practice for “accomplish anything at all through purposeful practice.” It took 40 hours despite me being IMO very fucking clever about it.
Skills from a year of Purposeful Rationality Practice
A year ago, I started trying to deliberate practice skills that would “help people figure out the answers to confusing, important questions.” I experimented with Thinking Physics questions, GPQA questions, Puzzle Games , Strategy Games, and a stupid twitchy reflex game I had struggled to beat for 8 years[1]. Then I went back to my day job and tried figuring stuff out there too.
The most important skill I was trying to learn was Metastrategic Brainstorming – the skill of looking at a confusing, hopeless situation, and nonetheless brainstorming useful ways to get traction or avoid wasted motion.
Normally, when you want to get good at something, it’s great to stand on the shoulders of giants and copy all the existing techniques. But this is challenging if you’re trying to solve important, confusing problems because there probably isn’t (much) established wisdom on how to solve it. You may need to discover techniques that haven’t been invented yet, or synthesize multiple approaches that haven’t previously been combined. At the very least, you may need to find an existing technique buried in the internet somewhere, which hasn’t been linked to your problem with easy-to-search keywords, without anyone to help you.
In the process of doing this, I found a few skills that came up over and over again.
I didn’t invent the following skills, but I feel like I “won” them in some sense via a painstaking “throw myself into the deep end” method. I feel slightly wary of publishing them in a list here, because I think it was useful to me to have to figure out for myself that they were the right tool for the job. And they seem like kinda useful “entry level” techniques, that you’re more likely to successfully discover for yourself.
But, I think this is hard enough, and forcing people to discover everything for themselves seems unlikely to be worth it.
The skills that seemed most general, in both practice and on my day job, are:
Taking breaks/naps
Working Memory facility
Patience
Knowing what confusion/deconfusion feels like
Actually Fucking Backchain
Asking “what is my goal?”
Having multiple plans
There were other skills I already was tracking, like Noticing, or Focusing. There were also somewhat more classic “How to Solve It” style tools for breaking down problems. There are also a host of skills I need when translating this all into my day-job, like “setting reminders for myself” and “negotiating with coworkers.”
But the skills listed above feel like they stood out in some way as particularly general, and particularly relevant for “solve confusing problems.”
Taking breaks, or naps
Difficult intellectual labor is exhausting. During the two weeks I was working on solving Thinking Physics problems, I worked for like 5 hours a day and then was completely fucked up in the evenings. Other researchers I’ve talked to report similar things.
During my workshops, one of the most useful things I recommended people was “actually go take a nap. If you don’t think you can take a real nap because you can’t sleep, go into a pitch black room and lie down for awhile, and the worst case scenario is your brain will mull over the problem in a somewhat more spacious/relaxed way for awhile.”
Practical tips: Get yourself a sleeping mask, noise machine (I prefer a fan or air purifier), and access to a nearby space where you can rest. Leave your devices outside the room.
Working Memory facility
Often a topic feels overwhelming. This is often because it’s just too complicated to grasp with your raw working memory. But, there are various tools (paper, spreadsheets, larger monitors, etc) that can improve this. And, you can develop the skill of noticing “okay this isn’t fitting in my head, or even on my big monitor – what would let it fit in my head?”.
The “eye opening” example of this for me was trying to solve a physics problem that included 3 dimensions (but one of the dimensions was “time”). I tried drawing it out but grasping the time-progression was still hard. I came up with the idea of using semi-translucent paper, where I would draw a diagram of what each step looked like on separate pages, and then I could see where different elements were pointed.
I’ve also found “spreadsheet literacy” a recurring skill – google sheets is very versatile but you have to know what all the functions are, have a knack for arranging elements in an easy-to-parse way, etc.
Practical Tips: Have lots of kinds of paper, whiteboards and writing supplies around.
On google sheets:
You can make collapsible sections, which help with making complex models while also being able to hide away the complexity of sub-parts you aren’t modeling. (hotkey: alt-shift-rightarrow)
switch between “display formulas” and the default “display the result” mode
(hotkey: ctrl-backtic)
Patience
If I’m doing something confusingly hard, there are times when it feels painful to sit with it, and I’m itchy to pick some solution and get moving. This comes up in two major areas:
Deliberate/purposeful practice. A key thing here is to be practicing the form perfectly, which requires somehow slowing things down such that you have time to get each moment correct. The urge to rush can undo the practice you just did, by training mistakes, or prevent you from actually successfully practicing at all.
Launching into a plan, or declaring yourself done, when you are still confused. Sitting with the uncomfortableness feels very itchy. But vague plans can be completely wrong, resting on confused assumptions.
There is of course a corresponding virtue of “just get moving, build up momentum and start learning through iteration.” The wisdom to tell the difference between “I’m still confused and need to orient more” and “I need to get moving” is important. But, an important skill there is at least being capable of sitting with impatient discomfort, in the situations where that’s the right call.
Practical tips: I dunno I still kinda suck at this one, but I find taking deep breaths, and deliberately reminding myself “Slow is smooth, smooth is fast”.
Know what deconfusion, or “having a crisp understanding” feels like
A skill from both Thinking Physics and Baba is You.
When I first started Thinking Physics, I would get to a point where “I dunno, I feel pretty sure, and I can’t think of more things to do to resolve my confusion”, and then impatiently roll the dice on checking the answer. Sometimes I’d be right, more often I’d be wrong.
Eventually I had a breakthrough where I came up with a crisp model of the problem, and was like “oh, man, now it would actually be really surprising if any of the other answers were true.” From then on… well, I’d still sometimes got things wrong (mostly due to impatience). But, I could tell when I still had pieces of my model that were vague and unprincipled.
Similarly in Baba is You: when people don’t have a crisp understanding of the puzzle, they tend to grasp and straws and motivatedly-reason their way into accepting sketchy sounding premises. But, the true solution to a level often feels very crisp and clear and inevitable.
Learning to notice this difference in qualia is quite valuable.
Practical tips: This is where Noticing and Focusing are key, but are worthwhile for helping you notice subtle differences in how an idea feels in your mind.
Try either making explicit numerical predictions about whether you’ve solved an exercise before you look up the answer; or, write down a qualitative sentence like “I feel like I really deeply understand the answer” or “this seems probably right but I feel some niggling doubts.”
Actually Fucking Backchain
From Baba is You, I got the fear-of-god put in me seeing how easy it was to spin my wheels, tinkering around with stuff that was nearby/accessible/easy-to-iterate-with, and how that often turned out to not be at all relevant to beating a level.
I had much less wasted motion when I thought through “What would the final stages of beating this level need to look like? What are the stages just before those?”, and focusing my attention on things that could help me get to that point.
One might say “well, Baba is You is a game optimized for being counterintuitive and weird.” I think for many people with a goal like “build a successful startup”, it can sometimes be fine to just be forward chaining with stuff that feels promising, rather than trying to backchain from complex goals.
But, when I eyeball the realworld problems I’m contending with (i.e. x-risk) they really do seem like there’s a relatively narrow set of victory conditions that plausibly work. And, many of the projects I feel tempted to start don’t actually really seem that relevant.
(I also think great startup founders are often doing a mix of forward and backward chaining. i.e. I bet Jeff Bezos was like “okay I bet I could make an online bookstore that worked”, was also thinking “but, what if I ultimately wanted the Everything Store? What are obstacles that I’d eventually need to deal”)
Practical tips: First, come up with at least one concrete story of what the world would look like, if you succeeded at your goals. Try hard to come up with 2 other worlds, so you aren’t too anchored on your first idea.
Then, try to concretely imagine the steps that would come a little bit earlier in the chain from the end.
Don’t worry about mapping out all the different possible branches of the future (that’s impossible). But, for a complex plan, have at least one end-to-end plan that connects all the dots from the resources you have now to the victory condition at the end.
Meanwhile, while doing most of your work, notice when it starts to feel like you’ve lost the plot (try just making a little tally-mark whenever you notice yourself rabbitholing in a way that feels off). And ask “what is my goal? is what I’m currently doing helping”
Ask “What’s My Goal?”
Actually, having just written the previous section, I’m recalling a simpler, more commonly useful skill, which is simply to ask “what is my goal?”.
Often, doing this throws into relief that you’re not sure what your goal is. Sometimes, asking the question immediately prompts me to notice a key insight I’d been glossing over.
If you’re not sure what your goal is, try babbling some things that seem like they might be a goal, and then ask yourself “does this feel like what I’m most trying to achieve right now?”
It’s okay if it turns out your goal is different or more embarrassing-sounding than you thought. You might say “Actually, you know what? I do care more about showing off and sounding smart, than actually learning something right now.” (But, you might also realize “okay, I separately care about learning something and sounding smart”, and then be more intentional about finding a tactic that accomplishes both)
Once you remember (or figure out) your goal, as you brainstorm strategies, ask yourself “would I be surprised if this didn’t help me achieve my goals?”, and then prioritize strategies that you viscerally expect to work.
Always[2] try to have 3 hypotheses
This one is important enough to be it’s own post. (I guess, probably most of these are important enough to be a full post? But, this one especially)
But, listing here for completeness:
Whether you are solving a puzzle, or figuring out how to solve a puzzle, or deciding what your team should do next week, try to have multiple hypotheses. (I usually say “try to have at least 3 plans”, but a plan is basically a special case – a hypothesis about “doing X is the best way to achieve goal Y”).
They each need to be a hypothesis you actually believe in.
I say “at least 3”, because I think it gets you “fully intellectually agile.” If you only have one plan, it’s easy to get tunnel vision on it and not notice that it’s doomed. Two ideas helps free up your mind, but then you might still evaluate all evidence in terms of “does this support idea 1 or idea 2?”. If you have 3 different hypotheses, it’s much more natural to keep generating more hypotheses, and to pivot around in a multiple dimensional space of possibility.
This wasn’t practice for “solving confusing problems”, but it was practice for “accomplish anything at all through purposeful practice.” It took 40 hours despite me being IMO very fucking clever about it.
Okay not literally always, but, whenever you’re about to spend a large chunk of timing on a project or figuring something out.