I’m not very new, but I’ve been mostly lurking, so I think I’ll introduce myself. (Note: 1k words.)
Basic information: high school student, fairly socially clueless. Probably similar to how most people here were as teenagers (if not now) - smart, nerdy, a bit of a loner, etc. I’m saying this because I think it’s relevant enough in limited ways. (My age is relevant to the life plans I describe. The social cluelessness tells you that you should ignore any odd signals I send between the lines, because I didn’t intend to send them. Is there a conversational code, similar in type to Crocker’s rules, that says “I will send as much important information explicitly as possible, please err on the side of ignoring implicit signals”?)
My first introduction to the rationalist community was through Scott Alexander. A few years ago, I was in an online discussion about gender, it turned to tolerance, and somebody linked I Can Tolerate Anything Except The Outgroup. I got hooked on SSC’s clarity and novel ideas, and eventually that led me to LW. I’ve read HPMOR, the Sequences, and so on. It’s taken a while for it to sink in, but it’s had a large effect on my views. Prior to this, I was a fairly typical young atheist nerd, so the changes aren’t very drastic, but I often find myself using the ideas I’ve gotten through the Sequences and the mindset of analytic truthseeking. The object-level belief changes I’ve had are the obvious ones: many worlds, cryonics, intelligence explosion, effective altruism, etc. I’m a humanist transhumanist reductionist materialist atheist, like almost everyone else here. That’s a cluster-membership description, not tribe-membership. Language doesn’t make it easy to Keep Your Identity Small.
I’ve always wanted to go into a career in STEM. I’ve loved mathematics from a young age, and I’ve done pretty well at it too. I started taking university calculus in middle school, and upper-division university mathematics in high school. (I’m not saying that to brag—that wouldn’t even be effective here anyway—but to show that I’m not just a one-in-ten “good at math” person who thinks very highly of themselves. I think I’m at the one-in-ten-thousand level, but I don’t have high confidence in that estimate.) A few months ago, I decided that the best way to achieve my goals was to work in the field of AI risk research. I think I can make progress in that field, and AI risk is probably the most important field in history, so it’s the best choice. (Humanity needs to solve AI risk soon. My 50% estimate for the Singularity is 2040-2060, and the default is we all die. But you’ve heard this before.) I aim to work at MIRI, or a similar organization if that doesn’t work out. It’s a rather high goal I’ve set for myself, but if I can’t have immodest ambitions here, where can I have them? I’ve been accepted to a (roughly) top 10 university for Math/CS, and I read somewhere (80k Hours?) that’s the rough talent level necessary to do good AI risk research, so I don’t think it’s an impossible goal.
My biggest failure point is my inability to carry out goals. That’s what my inner Murphy says would cause my failure to get into MIRI and do good work. That’s probably the most important thing I’m currently trying to get out of LW—the Hammertime sequence looks promising. If anyone has any good recommendations for people who can’t remember to focus, I’d love them.
In fact, any recommendations would be greatly appreciated. Such as: what would you say to someone who’s going into college? What would you say to someone who wants to work in AI risk?
I’m currently working through the MIRI research guide, starting with Halmos’s Naive Set Theory. If anyone else is doing this and would like a study partner, we should study together.
I’ve read some things about AI risk, both through the popularizations available on LW/SSC and through a couple of papers. I’ve had a few ideas already. My Outside View is sane, and I know there’s a very low chance that I’ve seen something that everyone else missed. Should I write a post on LW about it anyway?
To give an example, here’s the idea on my mind right now: it’s probably not possible to encode all our values explicitly into an AI. The obvious solution is to build into it the ability to learn values. This means it’ll start in a state of “moral ignorance”, and learn what it “should want to do” by looking at people. I’m not saying it’ll copy people’s actions, I’m saying its actions will have to be somehow entangled with what humans are and do. Information theory and so on. The crucial point: before it “opens its eyes”, this AI is not a classical consequentialist, right? Classical consequentialists have a ranking over worlds that doesn’t vary by each world. This AI’s terminal goals change depending on which possible world it’s in! I want to explore the implications—are there pitfalls? Does this help us solve problems? What is the best way to build this kind of agent? Should we even build it this way or not? I also want to formalize this kind of agent. It seems very similar to UDT in a sense, so perhaps it’s a simple extension of UDT. But there’s probably complications, and it’s worth turning the fuzzy ideas into math.
Some questions I have: does this seem like a possibly fruitful direction to look? Has someone already done something like this? What advice do you have for someone trying to do what I’m doing? Is there a really good AI risk paper that I could look at and try to mimic in terms of “this is how you formalize things, these are the sorts of questions you need to answer, etc.”? Is there anyone who’d be interested in mentoring a young person who’s interested in the field? (Connotation clarification: communicating, giving advice, kind of a back-and-forth maybe? I don’t know what’s okay to ask for and what’s not, because I’m a young clueless person. But I’m really interested and motivated, and Asking For Help is important, so I’ll put this out there and hope people interpret it charitably.)
My biggest failure point is my inability to carry out goals. That’s what my inner Murphy says would cause my failure to get into MIRI and do good work. That’s probably the most important thing I’m currently trying to get out of LW—the Hammertime sequence looks promising. If anyone has any good recommendations for people who can’t remember to focus, I’d love them.
If you elaborate on your productivity issues, maybe we can offer specific recommendations. What’s the nature of your difficulty focusing?
My Outside View is sane, and I know there’s a very low chance that I’ve seen something that everyone else missed.
I was a computer science undergraduate at a top university. The outside view is that for computer science students taking upper division classes, assisting professors with research is nothing remarkable. Pure math is different, because there is so much already and you need to climb to the top before contributing. But AI safety is a very young field.
The thing you’re describing sounds similar to other proposals I’ve seen. But I’d suggest developing it independently for a while. A common piece of research advice: If you read what others write, you think the same thoughts they’re thinking, which decreases your odds of making an original contribution. (Once you run out of steam, you can survey the literature, figure out how your idea is different, and publish the delta.)
I would suggest playing with ideas without worrying a lot about whether they’re original. Independently re-inventing something can still be a rewarding experience. See also. You miss all the shots you don’t take.
Final note: When I was your age, I suffered from the halo effect when thinking about MIRI. It took me years to realize MIRI has blind spots just like everyone else. I wish I had realized this sooner. They say science advances one funeral at a time. If AI safety is to progress faster than that, we’ll need willingness to disregard the opinions of senior people while they’re still alive. A healthy disrespect for authority is a good thing to have.
You lucked out in terms of your journey, the shortcut through SSC may have saved you a number of years ;)
My only advice on careers would be to strongly consider doing some ML capabilities work rather than pure AI risk. This will make it much easier to get enough qualifications and experience to get to work on risk later on. The risk field is so much smaller (and is less well received in academia) that setting yourself that goal may be too much of a stretch. You can always try to pick thesis topics which are as close to the intersection of risk and capabilities as possible.
I’m not very new, but I’ve been mostly lurking, so I think I’ll introduce myself. (Note: 1k words.)
Basic information: high school student, fairly socially clueless. Probably similar to how most people here were as teenagers (if not now) - smart, nerdy, a bit of a loner, etc. I’m saying this because I think it’s relevant enough in limited ways. (My age is relevant to the life plans I describe. The social cluelessness tells you that you should ignore any odd signals I send between the lines, because I didn’t intend to send them. Is there a conversational code, similar in type to Crocker’s rules, that says “I will send as much important information explicitly as possible, please err on the side of ignoring implicit signals”?)
My first introduction to the rationalist community was through Scott Alexander. A few years ago, I was in an online discussion about gender, it turned to tolerance, and somebody linked I Can Tolerate Anything Except The Outgroup. I got hooked on SSC’s clarity and novel ideas, and eventually that led me to LW. I’ve read HPMOR, the Sequences, and so on. It’s taken a while for it to sink in, but it’s had a large effect on my views. Prior to this, I was a fairly typical young atheist nerd, so the changes aren’t very drastic, but I often find myself using the ideas I’ve gotten through the Sequences and the mindset of analytic truthseeking. The object-level belief changes I’ve had are the obvious ones: many worlds, cryonics, intelligence explosion, effective altruism, etc. I’m a humanist transhumanist reductionist materialist atheist, like almost everyone else here. That’s a cluster-membership description, not tribe-membership. Language doesn’t make it easy to Keep Your Identity Small.
I’ve always wanted to go into a career in STEM. I’ve loved mathematics from a young age, and I’ve done pretty well at it too. I started taking university calculus in middle school, and upper-division university mathematics in high school. (I’m not saying that to brag—that wouldn’t even be effective here anyway—but to show that I’m not just a one-in-ten “good at math” person who thinks very highly of themselves. I think I’m at the one-in-ten-thousand level, but I don’t have high confidence in that estimate.) A few months ago, I decided that the best way to achieve my goals was to work in the field of AI risk research. I think I can make progress in that field, and AI risk is probably the most important field in history, so it’s the best choice. (Humanity needs to solve AI risk soon. My 50% estimate for the Singularity is 2040-2060, and the default is we all die. But you’ve heard this before.) I aim to work at MIRI, or a similar organization if that doesn’t work out. It’s a rather high goal I’ve set for myself, but if I can’t have immodest ambitions here, where can I have them? I’ve been accepted to a (roughly) top 10 university for Math/CS, and I read somewhere (80k Hours?) that’s the rough talent level necessary to do good AI risk research, so I don’t think it’s an impossible goal.
My biggest failure point is my inability to carry out goals. That’s what my inner Murphy says would cause my failure to get into MIRI and do good work. That’s probably the most important thing I’m currently trying to get out of LW—the Hammertime sequence looks promising. If anyone has any good recommendations for people who can’t remember to focus, I’d love them.
In fact, any recommendations would be greatly appreciated. Such as: what would you say to someone who’s going into college? What would you say to someone who wants to work in AI risk?
I’m currently working through the MIRI research guide, starting with Halmos’s Naive Set Theory. If anyone else is doing this and would like a study partner, we should study together.
I’ve read some things about AI risk, both through the popularizations available on LW/SSC and through a couple of papers. I’ve had a few ideas already. My Outside View is sane, and I know there’s a very low chance that I’ve seen something that everyone else missed. Should I write a post on LW about it anyway?
To give an example, here’s the idea on my mind right now: it’s probably not possible to encode all our values explicitly into an AI. The obvious solution is to build into it the ability to learn values. This means it’ll start in a state of “moral ignorance”, and learn what it “should want to do” by looking at people. I’m not saying it’ll copy people’s actions, I’m saying its actions will have to be somehow entangled with what humans are and do. Information theory and so on. The crucial point: before it “opens its eyes”, this AI is not a classical consequentialist, right? Classical consequentialists have a ranking over worlds that doesn’t vary by each world. This AI’s terminal goals change depending on which possible world it’s in! I want to explore the implications—are there pitfalls? Does this help us solve problems? What is the best way to build this kind of agent? Should we even build it this way or not? I also want to formalize this kind of agent. It seems very similar to UDT in a sense, so perhaps it’s a simple extension of UDT. But there’s probably complications, and it’s worth turning the fuzzy ideas into math.
Some questions I have: does this seem like a possibly fruitful direction to look? Has someone already done something like this? What advice do you have for someone trying to do what I’m doing? Is there a really good AI risk paper that I could look at and try to mimic in terms of “this is how you formalize things, these are the sorts of questions you need to answer, etc.”? Is there anyone who’d be interested in mentoring a young person who’s interested in the field? (Connotation clarification: communicating, giving advice, kind of a back-and-forth maybe? I don’t know what’s okay to ask for and what’s not, because I’m a young clueless person. But I’m really interested and motivated, and Asking For Help is important, so I’ll put this out there and hope people interpret it charitably.)
Hey, welcome! Glad you made this post.
If you elaborate on your productivity issues, maybe we can offer specific recommendations. What’s the nature of your difficulty focusing?
I was a computer science undergraduate at a top university. The outside view is that for computer science students taking upper division classes, assisting professors with research is nothing remarkable. Pure math is different, because there is so much already and you need to climb to the top before contributing. But AI safety is a very young field.
The thing you’re describing sounds similar to other proposals I’ve seen. But I’d suggest developing it independently for a while. A common piece of research advice: If you read what others write, you think the same thoughts they’re thinking, which decreases your odds of making an original contribution. (Once you run out of steam, you can survey the literature, figure out how your idea is different, and publish the delta.)
I would suggest playing with ideas without worrying a lot about whether they’re original. Independently re-inventing something can still be a rewarding experience. See also. You miss all the shots you don’t take.
Final note: When I was your age, I suffered from the halo effect when thinking about MIRI. It took me years to realize MIRI has blind spots just like everyone else. I wish I had realized this sooner. They say science advances one funeral at a time. If AI safety is to progress faster than that, we’ll need willingness to disregard the opinions of senior people while they’re still alive. A healthy disrespect for authority is a good thing to have.
You lucked out in terms of your journey, the shortcut through SSC may have saved you a number of years ;)
My only advice on careers would be to strongly consider doing some ML capabilities work rather than pure AI risk. This will make it much easier to get enough qualifications and experience to get to work on risk later on. The risk field is so much smaller (and is less well received in academia) that setting yourself that goal may be too much of a stretch. You can always try to pick thesis topics which are as close to the intersection of risk and capabilities as possible.