Katja is widely respected amongst the rationalists and, according to Hive, she is one of the most followed/respected EA accounts[1]. But she doesn’t give off the same vibe as many impact olympians. She doesn’t have iron self-will, nor does she manage a huge team. She’s hasn’t got all the facts at her fingertips. But she has got something, I’m confident of that. How can I be more like her?
To understand her impact, let’s consider the top things she’s done:
She ran surveys on AI researchers well before they were needed and has continued to run them
She wrote an early blog on how we could slow down AI. This blog, I’ve heard, played a part in encouraging the Musk AI letter, which in turn inspired the “Existential Risks” AI letter.
She thought about AI long before it was vogue, since about 2010
She has a large track record of predictions
These actions seem impactful to me. And I guess someone should have paid $10mn in hindsight for the first 2, maybe more.
To me, Katja has a very low tolerance for incomplete stories. When she sees something that she doesn’t quite understand or that seems a bit off she struggles to pretend otherwise, so she says “how does that work?”. She doesn’t accept handwaving when discussing something, whether it be the simulation argument, how efficient flight is or the plot of Dune, part 2[2]. She wants an unbroken chain of arguments she can repeat[3].
She also doesn’t mind admitting she doesn’t know the answer. In her living room she will turn to her friend Joe Carlsmith and ask “Wait, why are we worried about AI, again?” even though she’s been thinking about this for 15 years. Because at that moment it doesn’t fit for her and she has a high tolerance for embarrassed[4] when it comes to truth. There is an deep resolve here—she doesn’t get it, so she will ask until she does.
She works on the most important thing, slowly. If you are Elon Musk, maybe you can just work all the time. But I am not. And much as I love her, neither is Katja. She does not get an abnormal amount of work done per day. Instead, month in, month out, Katja works on what she thinks is most important. And eventually she gets the survey done, years ahead of when it’s needed.
There are lessons we can take from this. Just as we often talk about learning to code, or task management, I can become better at saying “wait that doesn’t work”. Here are some strategies that let me be more like Katja:
Write it down—it’s harder to fool myself into thinking something makes sense if I have to read it rather than speak it
“What is the best thing? How do I do that?”—this is hard to put into practice but an underrated prompt
Give more examples—one feature of Katja’s writing is she loves to list things. I think more people should list every example in favour of their argument and every counterexample they can think of. Spend 5 minutes on each.
Can I make a forecast of that? - I find fatebook.io useful for this. As I forecast more I learn how poor my judgement is. And I think it’s improving.
Know when I am not capable—Katja is good at knowing when something is beyond her. When she hasn’t thought about something or when it’s a quantitative problem and she hasn’t worked on it carefully enough. She doesn’t always like the hard work but she knows when it needs to be done.
If you have the right answer you can afford to be slow—in a world of often lurching acceleration, it’s easy to forget that if I just knew the right thing, then I could probably take years over it. More output is (usually) more better, but so is more accuracy.
Have a distinction between what you currently understand and what you’d bet on—If Peter Wildeford and I disagree, he’s probably right, but that doesn’t mean I now understand. It is worth tracking the difference between what I actually think and what I would say if I had to bet on it. They might be very different.
For balance, there are costs to this mindset. Katja actively cultivates a feeling that if she makes errors there may be grave suffering, but that she must go on regardless. Sometimes she dreads meetings, or struggles with migraines. At times she has said she would prefer the most painful experiences of her life[5] if her work would do itself that day. This breaks my heart.
I don’t see these costs as central but they are worth mentioning[6]. If you are someone who struggles with overwork or neuroticism, maybe choose a different role model. Or ideally, take the good without falling into this trap.
Because for me, Katja is a wise guide on epistemics. Just as I can work more hours or use automation, I can take steps to ensure the work I do actually has good results. I can carefully look for the best thing and do it. And perhaps, I would recommend developing an inner Katja, to turn to you while you are mid-flow and, kindly, say “Sorry, I don’t understand that step.”
Her comment on this section “I feel like in some sense I do accept it here, but the consequence is that I can barely think about the topic. You may disagree about ‘acceptance’, thanks for answering a lot of questions about the plot of Dune, part 2 :P”
Personally this is a bit of a struggle, since I am at times a bit of a vague thinker. She’ll turn to me with a sort of innocent, concerned expression and ask for an example, which often I don’t have
An earlier draft said she wasn’t embarrassed. She says she is, but that she can’t give up because of that. She is embarrassed to still care about covid, but she thinks the arguments are good, so she does.
Nor are they central to her—this piece misses her singing, her love of donkeys and her stories about growing up in a restaurant in an abandoned movie set
Be More Katja
Katja is widely respected amongst the rationalists and, according to Hive, she is one of the most followed/respected EA accounts[1]. But she doesn’t give off the same vibe as many impact olympians. She doesn’t have iron self-will, nor does she manage a huge team. She’s hasn’t got all the facts at her fingertips. But she has got something, I’m confident of that. How can I be more like her?
To understand her impact, let’s consider the top things she’s done:
She ran surveys on AI researchers well before they were needed and has continued to run them
She wrote an early blog on how we could slow down AI. This blog, I’ve heard, played a part in encouraging the Musk AI letter, which in turn inspired the “Existential Risks” AI letter.
She thought about AI long before it was vogue, since about 2010
She has a large track record of predictions
These actions seem impactful to me. And I guess someone should have paid $10mn in hindsight for the first 2, maybe more.
To me, Katja has a very low tolerance for incomplete stories. When she sees something that she doesn’t quite understand or that seems a bit off she struggles to pretend otherwise, so she says “how does that work?”. She doesn’t accept handwaving when discussing something, whether it be the simulation argument, how efficient flight is or the plot of Dune, part 2[2]. She wants an unbroken chain of arguments she can repeat[3].
She also doesn’t mind admitting she doesn’t know the answer. In her living room she will turn to her friend Joe Carlsmith and ask “Wait, why are we worried about AI, again?” even though she’s been thinking about this for 15 years. Because at that moment it doesn’t fit for her and she has a high tolerance for embarrassed[4] when it comes to truth. There is an deep resolve here—she doesn’t get it, so she will ask until she does.
She works on the most important thing, slowly. If you are Elon Musk, maybe you can just work all the time. But I am not. And much as I love her, neither is Katja. She does not get an abnormal amount of work done per day. Instead, month in, month out, Katja works on what she thinks is most important. And eventually she gets the survey done, years ahead of when it’s needed.
There are lessons we can take from this. Just as we often talk about learning to code, or task management, I can become better at saying “wait that doesn’t work”. Here are some strategies that let me be more like Katja:
Write it down—it’s harder to fool myself into thinking something makes sense if I have to read it rather than speak it
“What is the best thing? How do I do that?”—this is hard to put into practice but an underrated prompt
Give more examples—one feature of Katja’s writing is she loves to list things. I think more people should list every example in favour of their argument and every counterexample they can think of. Spend 5 minutes on each.
Can I make a forecast of that? - I find fatebook.io useful for this. As I forecast more I learn how poor my judgement is. And I think it’s improving.
Know when I am not capable—Katja is good at knowing when something is beyond her. When she hasn’t thought about something or when it’s a quantitative problem and she hasn’t worked on it carefully enough. She doesn’t always like the hard work but she knows when it needs to be done.
If you have the right answer you can afford to be slow—in a world of often lurching acceleration, it’s easy to forget that if I just knew the right thing, then I could probably take years over it. More output is (usually) more better, but so is more accuracy.
Have a distinction between what you currently understand and what you’d bet on—If Peter Wildeford and I disagree, he’s probably right, but that doesn’t mean I now understand. It is worth tracking the difference between what I actually think and what I would say if I had to bet on it. They might be very different.
For balance, there are costs to this mindset. Katja actively cultivates a feeling that if she makes errors there may be grave suffering, but that she must go on regardless. Sometimes she dreads meetings, or struggles with migraines. At times she has said she would prefer the most painful experiences of her life[5] if her work would do itself that day. This breaks my heart.
I don’t see these costs as central but they are worth mentioning[6]. If you are someone who struggles with overwork or neuroticism, maybe choose a different role model. Or ideally, take the good without falling into this trap.
Because for me, Katja is a wise guide on epistemics. Just as I can work more hours or use automation, I can take steps to ensure the work I do actually has good results. I can carefully look for the best thing and do it. And perhaps, I would recommend developing an inner Katja, to turn to you while you are mid-flow and, kindly, say “Sorry, I don’t understand that step.”
Article originally posted at https://nathanpmyoung.substack.com/
Katja and I date, so yes, I am biased, but I really think that’s a pretty unimportant fact about her
Her comment on this section “I feel like in some sense I do accept it here, but the consequence is that I can barely think about the topic. You may disagree about ‘acceptance’, thanks for answering a lot of questions about the plot of Dune, part 2 :P”
Personally this is a bit of a struggle, since I am at times a bit of a vague thinker. She’ll turn to me with a sort of innocent, concerned expression and ask for an example, which often I don’t have
An earlier draft said she wasn’t embarrassed. She says she is, but that she can’t give up because of that. She is embarrassed to still care about covid, but she thinks the arguments are good, so she does.
Operations without anaesthetic
Nor are they central to her—this piece misses her singing, her love of donkeys and her stories about growing up in a restaurant in an abandoned movie set