[Note: this one, steelman, and feedback on proposals all have very similar input spaces. I think I would ideally mix them as one in an actual product, but I’m keeping them separate for now]
Task: Obvious EA/Alignment Advice
Context: There are several common mental motions that the EA community does which are usefully applied to alignment. Ex. “Differential Impact”, “Counterfactual Impact”, “Can you clearly tell a story on how this reduces x-risk?”, and “Truly Optimizing for X”. A general “obvious advice” is useful for general capabilities as well, but this is intended to have a strong, EA tilt to it.
Input Type: A plan for reducing x-risk, either a research direction, upskilling, or field-building
Output Type: Useful exploratory questions
Instance 1
Input:
I’m thinking about taking up a job at Google to upskill in machine learning before I apply to Anthropic or Redwood Research.
Output:
Isn’t there a more direct path to upskilling in machine learning? Like taking a course or doing a bunch of dumb, fun projects? Additionally, is programming/machine learning your comparative advantage?
Instance 2
Input:
I’m working on robustness to distributional shift, specifically identifying when an input is out-of-distribution so it can default to a no-op or asking for human feedback on that input.
Output:
How neglected is this field? If you didn’t work in this field, would someone else have done the work you’d do, but a year or two later?
Instance 3
Input:
I’m building an interpretability tool that can tell you what the model was thinking when it was making a classification, for instance, what part of the input it was using to make that classification.
Output:
Let’s say you succeed in making this tool; can you tell a story on how this tool can reduces AI x-risk?
I actually happen to already have taught elicit to give helpful/obvious advice (not alignment specific, but close enough given the examples were inspired by thinking that lots of the advice I give in my day job as an advisor is obvious)! You can play with it here if you have an elicit account.
Edit:
Here’s the training data
Life problem
I need to think of a research topic but I’ve only had two ideas and both of them aren’t that great.
Obvious but useful advice.
Have you tried setting a 5-minute timer and, by-the-clock, just trying to write down as many ideas as possible? This can work surprisingly well!
Life problem
I’ve been putting off writing this email, and now every time I think about it I feel bad and don’t want to do anything at all, especially write the email!
Obvious but useful advice.
This seems like a pretty urgent problem to solve, as it’s one that will just keep getting worse otherwise as you get negative reinforcement when you think about it. I have a few ideas for things to try: Can you get a friend to sit with you while you write it, or even to write it for you? If you make it your number one priority, can you get it done right now? Is there a way you can not send it, for example by just emailing to say ‘sorry, can’t reply now, will explain later’?
Life problem
I’m thinking about quitting my job in finance in order to self-study ML and switch to working on alignment. How can I make the final decision?
Obvious but useful advice.
That’s an exciting decision to be making! It might be worth writing up the pros and cons of both options in a googledoc, and sharing it with some friends with comment access enabled. Getting your thoughts sorted in a way which is clear to others might be helpful itself, and then also your friends might have useful suggestions or additional considerations!
Life problem
I’m giving a talk tomorrow, but I’m worried that I’m going to be asked a question that’s difficult to answer and then say something which seems stupid if it gets taken out of context.
Obvious but useful advice.
Remember that it’s fine to say “I don’t know”, or some variant like “I’m not sure I can give a clear enough answer to that without thinking about it for longer”, when people ask you questions! You can even write this on your hand if you need to.
Life problem
I’m a bit worried about the next week, there’s a few pretty important things happening and I’ve tried to plan as best I can for them but I’m worried it’s all going to go horribly wrong.
Obvious but useful advice.
It’s ok to be worried before important things! One option you have, as long as it wouldn’t be too stressful, is writing a premortem. This is where you think about a specific thing that could go wrong, then decide whether there’s any action you can take to prevent or mitigate that scenario, then repeat. At the end of this process, you can then say to yourself something like “I’m pretty sure I’ve taken all of the actions it’s reasonable to take”, which might be reassuring. Even if it’s not immediately emotionally reassuring, if the actions were sensible, it’s now less likely things will go wrong!
Life problem
I don’t know how to go about planning my career.
Obvious but useful advice.
My suggestion would be to write down a few of the most important things you’d like your career to look like in two or three years, and for each of them write down concrete steps which you think will make progress towards achieving that. Then you can start seeing what sorts of jobs / collaborations might be feasible. If one thing that’s important to you for your career is impact, you could check out the resources on 80000hours.org, where there’s lots of helpful guidance, and you can even book a call with an advisor.
Life problem
I’m trying to solve a difficult technical problem at work, but my mind is just blank.
Obvious but useful advice.
Do you have to solve the problem on your own? Asking a colleague or friend can be really helpful. You could even write up a description of the problem in a googledoc and share it with a few people asking them to comment with ideas. If you have to solve the problem on your own, you could try imagining a specific person who you think would be able to solve the problem, and asking yourself what they would try.
Life problem
I’ve taken on too many commitments and now I’m feeling really overwhelmed by them
Obvious but useful advice.
Remember that saying “no” is fine, there are a lot of good reasons to drop commitments, including simply not having time or being too busy. You don’t need to feel guilty about making this decision now, but when you’re feeling better it might be worth working out whether you could have noticed dropped some things earlier. If you’re finding yourself saying yes too much, it might be worth setting yourself a ‘saying no to things’ goal which you share with a friend.
Life problem
I’ve got a test next week and I don’t know how best to prepare.
Obvious but useful advice.
Do you have any practice tests you can sit and then mark to work out where you went wrong? The best way to prepare for a thing is usually just trying to do that thing!
Life problem
I’m struggling to maintain focus as I work. I find myself staring into space, and not really resting, but I feel too guilty to stop and take a break.
Obvious but useful advice.
If you’re too stressed or tired to do useful work, you should stop and rest! It’s better to fully rest and gain back some energy than keep struggling when you aren’t being productive. You could also try using the pomodoro technique of working for set periods of time and taking breaks in between.
[Note: this one, steelman, and feedback on proposals all have very similar input spaces. I think I would ideally mix them as one in an actual product, but I’m keeping them separate for now]
Task: Obvious EA/Alignment Advice
Context: There are several common mental motions that the EA community does which are usefully applied to alignment. Ex. “Differential Impact”, “Counterfactual Impact”, “Can you clearly tell a story on how this reduces x-risk?”, and “Truly Optimizing for X”. A general “obvious advice” is useful for general capabilities as well, but this is intended to have a strong, EA tilt to it.
Input Type: A plan for reducing x-risk, either a research direction, upskilling, or field-building
Output Type: Useful exploratory questions
Instance 1
Input:
Output:
Instance 2
Input:
Output:
Instance 3
Input:
Output:
I actually happen to already have taught elicit to give helpful/obvious advice (not alignment specific, but close enough given the examples were inspired by thinking that lots of the advice I give in my day job as an advisor is obvious)! You can play with it here if you have an elicit account.
Edit:
Here’s the training data
Life problem
I need to think of a research topic but I’ve only had two ideas and both of them aren’t that great.
Obvious but useful advice.
Have you tried setting a 5-minute timer and, by-the-clock, just trying to write down as many ideas as possible? This can work surprisingly well!
Life problem
I’ve been putting off writing this email, and now every time I think about it I feel bad and don’t want to do anything at all, especially write the email!
Obvious but useful advice.
This seems like a pretty urgent problem to solve, as it’s one that will just keep getting worse otherwise as you get negative reinforcement when you think about it. I have a few ideas for things to try: Can you get a friend to sit with you while you write it, or even to write it for you? If you make it your number one priority, can you get it done right now? Is there a way you can not send it, for example by just emailing to say ‘sorry, can’t reply now, will explain later’?
Life problem
I’m thinking about quitting my job in finance in order to self-study ML and switch to working on alignment. How can I make the final decision?
Obvious but useful advice.
That’s an exciting decision to be making! It might be worth writing up the pros and cons of both options in a googledoc, and sharing it with some friends with comment access enabled. Getting your thoughts sorted in a way which is clear to others might be helpful itself, and then also your friends might have useful suggestions or additional considerations!
Life problem
I’m giving a talk tomorrow, but I’m worried that I’m going to be asked a question that’s difficult to answer and then say something which seems stupid if it gets taken out of context.
Obvious but useful advice.
Remember that it’s fine to say “I don’t know”, or some variant like “I’m not sure I can give a clear enough answer to that without thinking about it for longer”, when people ask you questions! You can even write this on your hand if you need to.
Life problem
I’m a bit worried about the next week, there’s a few pretty important things happening and I’ve tried to plan as best I can for them but I’m worried it’s all going to go horribly wrong.
Obvious but useful advice.
It’s ok to be worried before important things! One option you have, as long as it wouldn’t be too stressful, is writing a premortem. This is where you think about a specific thing that could go wrong, then decide whether there’s any action you can take to prevent or mitigate that scenario, then repeat. At the end of this process, you can then say to yourself something like “I’m pretty sure I’ve taken all of the actions it’s reasonable to take”, which might be reassuring. Even if it’s not immediately emotionally reassuring, if the actions were sensible, it’s now less likely things will go wrong!
Life problem
I don’t know how to go about planning my career.
Obvious but useful advice.
My suggestion would be to write down a few of the most important things you’d like your career to look like in two or three years, and for each of them write down concrete steps which you think will make progress towards achieving that. Then you can start seeing what sorts of jobs / collaborations might be feasible. If one thing that’s important to you for your career is impact, you could check out the resources on 80000hours.org, where there’s lots of helpful guidance, and you can even book a call with an advisor.
Life problem
I’m trying to solve a difficult technical problem at work, but my mind is just blank.
Obvious but useful advice.
Do you have to solve the problem on your own? Asking a colleague or friend can be really helpful. You could even write up a description of the problem in a googledoc and share it with a few people asking them to comment with ideas. If you have to solve the problem on your own, you could try imagining a specific person who you think would be able to solve the problem, and asking yourself what they would try.
Life problem
I’ve taken on too many commitments and now I’m feeling really overwhelmed by them
Obvious but useful advice.
Remember that saying “no” is fine, there are a lot of good reasons to drop commitments, including simply not having time or being too busy. You don’t need to feel guilty about making this decision now, but when you’re feeling better it might be worth working out whether you could have noticed dropped some things earlier. If you’re finding yourself saying yes too much, it might be worth setting yourself a ‘saying no to things’ goal which you share with a friend.
Life problem
I’ve got a test next week and I don’t know how best to prepare.
Obvious but useful advice.
Do you have any practice tests you can sit and then mark to work out where you went wrong? The best way to prepare for a thing is usually just trying to do that thing!
Life problem
I’m struggling to maintain focus as I work. I find myself staring into space, and not really resting, but I feel too guilty to stop and take a break.
Obvious but useful advice.
If you’re too stressed or tired to do useful work, you should stop and rest! It’s better to fully rest and gain back some energy than keep struggling when you aren’t being productive. You could also try using the pomodoro technique of working for set periods of time and taking breaks in between.