I don’t really understand… Suppose I am a computer scientist who has just learned about AI risks and am eager to contribute, but I’m not sure where to start. Then the natural step would be to ask someone who does have experience in this area for their advice (e.g. what careers should I go into), so I can take advantage of what’s already known rather than starting from scratch.
My surface question is about careers, but my true/implicit question is actually asking for help on getting started contributing to AI risk. It’s not about wanting to know how to be considered a real member of the group? I would be annoyed by the first answer because it answers my superficial question while rebuffing my true question (I still don’t know what to do next!). I am asking you for help and advice, and yet your response is that you have no answer. I mean it is technically true and it would be a good answer for a peer, but in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Answer 3 is good because you are sharing your experience and offering help. There’s no need to make reference to culture. I would be be just as happy with an answer that says “no one really knows” (instead of “I don’t know”), because then you are telling me the current state of knowledge in the industry.
A similar example:
When I asked someone who was teaching me how to cook how much salt I should add to the dish, they answered “I don’t know”. I was annoyed because it sounded like they were refusing to help me (because they’re not answering my implicit question of how to decide how much salt to put when I cook next time) when they were the expert. It would have been better if they’d said that the saltiness varies based on the type of salt and the other ingredients and your own preferences, etc. so it’s not possible to have a known, fixed amount of salt. What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments. Nowadays I would know how to rephrase my question to get the answer I want, but I didn’t use to be able to.
I guess I’d say frustrated, worried, confused. I was somewhat surprised/alarmed by the conclusion that Alec was actually trying to request information on how to be considered part of the group.
It seems to me like a rather uncharitable interpretation of Alec’s response, to assume that he just wants to figure out how to belong, rather than genuinely desiring to find out how best to contribute.
We try to have a culture around here where there is no vetted-by-the-group answer to this; we instead try to encourage forming your own inside-view model of how AI risk might work, what paths through to a good future might be possible, etc.
I would be rather insulted by this response, because it implies that I am looking for a vetted-by-the-group answer, and also seems to be criticising me for asking Anna about opimal careers. Firstly, that was never my intent. Secondly, asking an expert for their opinion sounds like a perfectly reasonable course of action to me. However, this does assume that Alec shares my motivations and assumptions.
I’m not sure of my assumptions/beliefs/conclusions though. I might be missing context (e.g. I don’t know what bay area is like, or the cultural norms), and I didn’t really understand the essay (I found the example too distracting for me to focus on the concept of narrative syncing—I like the new examples much more).
>my true question (I still don’t know what to do next!)
(The following might be a strawman; sorry if so; but it is a genuine question about a phenomenon I’ve seen.) This part sounds like you’re saying, you’re asking someone what to do next. If so, what if it’s a mistake for you to be mentally executing the procedure 1. hear what someone says to do next, 2. do that thing? How would the person you’re asking communicate to you that it’s a mistake for you to execute that procedure, without you taking their attempted statements of fact as cryptic instructions for you to follow? (Assuming that’s a mistake you think you could plausibly be making, and assuming you’d want the mistake communicated to you if you were making it.)
>in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Someone’s role is a combination of their behavior and the behavior of others’ behavior and expectations. Suppose that Berry wants to invite Alec to a workshop with an offer to help, but *not* with any implied intent to play a role of advisor/expert. How could Berry communicate that to Alec?
>What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments.
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.
Hmm if CFAR organizes a workshop, then I would think it is reasonable to assume that the CFAR staff (assuming they are introduced as such) are there as experienced members of the AI risk community who are there to offer guidance to people who are new to the area.
Thus, if I ask them about career paths, I’m not asking for their opinions as individuals, I’m asking for their opinions as people with more expertise than I have.
Two possible motivations I can think of for consulting someone with expertise would be:
In school, I ask the teacher when I have a question so they tell me the answer. Likewise, if I’m not sure of the appropriate career move, I should ask an expert in AI risk and they can tell me what to do. If I follow their answer and it fails, then it’s their fault. I would feel betrayed and blame them for wasting my life.
Someone who has been working on AI risks would have more knowledge about the area. Thus, I should consult their opinions before deciding what to do. I would feel betrayed if I later find out that they gave me a misleading answer (e.g. telling me to study ML when that’s not what they actually think).
In both cases I’m trying to figure out what to do next, but in the first case I want to be told what to do next, and in the second case I just want more information so I can decide what to do next. (Expert doesn’t mean you bear responsibility for my life decisions, just responsibility for providing accurate information.)
If I’m not wrong, you’re asking about the first case? (My comment was trying to describe the second scenario.)
If it’s just a matter of ambiguity (e.g. I will accompany you just for moral support vs accompany you to provide advice), I would just state it explicitly (e.g. tell you “I won’t be helping you. If someone asks you something and you look to me for help, I’ll just look at you and smile.”) and then do precisely what I said I’ll do.
Otherwise, if it’s a mindset issue (thinking I should do what others tell me to), it’s a deeper problem. If it’s someone I’m close to, I would address the issue directly e.g. saying “If the GPS tells you to drive into a lake and you listen and drown, whose problem is it? I can tell you what to do, but if I’m wrong, you’re the one who will suffer, not me. Even if some very wise person tells you what to do, you still have to check if it makes sense and decide if you will do it, because you have the most to lose. If other people tell you the wrong thing, they can just go ‘Oops sorry! I didn’t realise.’ You, on the other hand, are stuck picking up the pieces. Make sure you think it is worth it before implementing anyone’s advice.” And if that fails, just… let them make their mistakes and eventually they’ll learn from their experiences?
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.
Not really, saying “I don’t know” is very different from saying “after years of research, we still don’t know”.
“I’m not sure” sounds like the kind of answer a newbie would give, so I’m not really learning anything new from the conversation. Even worse, “don’t know” sounds like you don’t really want to help me—surely an expert knows more than me, if they say they don’t know, then that must mean they just don’t want to share their knowledge.
In contrast, if you said that no one knows because the field is still too new and rapidly changing, then you are giving me an informed opinion. I now know that this is a problem even the experts haven’t solved and can then make decisions based on this information.
Anyone can say “I’m an expert”. There has to be some other way that you’re distinguishing who’s a newbie who’s just clueless, from an expert who’s ignorance is evidence of a general difficulty. From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason. E.g., at a gloss, Yudkowsky says “no one has a plan that could possibly work”, and Christiano says “no, I have a plan which, with some optimistic but not crazy assumptions, looks like it’ll probably work”, and Yudkowsky says something that people don’t understand, and Christiano responds in a way that may or may not address Yudkowsky’s critique, and who knows who’s an expert? Just going by who says “I have a plan” is helpful if someone has a plan that will work (although it also opens up social niches for con artists).
From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason.
Ah yes agreed. Alec doesn’t know that this is what’s happening though (judging from his response to answer 1). Personally I’d default to assuming that Berry will play the role of expert since he’s part of CFAR while I’m just a random computer scientist (from Berry’s perspective). I would switch to a more equal dynamic only if there’s a clear indicator that I should do so.
For example, if a student asks a professor a question, the professor may ask the student for their thoughts instead and then respond thoughtfully to their answer, like they would respond to a fellow professor. Or if a boss asks a new subordinate for their opinion but the subordinate thinks this is a fake question and tries to guess the cryptic instructions instead (because sometimes people ask you questions to hint that you should answer a certain way), the boss may ask someone else who’s been on the team longer. When the new member sees the senior member responding honestly and boss engaging thoughtfully with the response, then the new member would know that the question was genuine.
In the careers example, I can’t tell from the first answer that Anna is trying to engage me as a peer. (I’d imagine someone used to different norms might assume that as the default though.)
I don’t really understand… Suppose I am a computer scientist who has just learned about AI risks and am eager to contribute, but I’m not sure where to start. Then the natural step would be to ask someone who does have experience in this area for their advice (e.g. what careers should I go into), so I can take advantage of what’s already known rather than starting from scratch.
My surface question is about careers, but my true/implicit question is actually asking for help on getting started contributing to AI risk. It’s not about wanting to know how to be considered a real member of the group? I would be annoyed by the first answer because it answers my superficial question while rebuffing my true question (I still don’t know what to do next!). I am asking you for help and advice, and yet your response is that you have no answer. I mean it is technically true and it would be a good answer for a peer, but in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Answer 3 is good because you are sharing your experience and offering help. There’s no need to make reference to culture. I would be be just as happy with an answer that says “no one really knows” (instead of “I don’t know”), because then you are telling me the current state of knowledge in the industry.
A similar example:
When I asked someone who was teaching me how to cook how much salt I should add to the dish, they answered “I don’t know”. I was annoyed because it sounded like they were refusing to help me (because they’re not answering my implicit question of how to decide how much salt to put when I cook next time) when they were the expert. It would have been better if they’d said that the saltiness varies based on the type of salt and the other ingredients and your own preferences, etc. so it’s not possible to have a known, fixed amount of salt. What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments. Nowadays I would know how to rephrase my question to get the answer I want, but I didn’t use to be able to.
I am curious about how you felt when writing this bit:
I guess I’d say frustrated, worried, confused. I was somewhat surprised/alarmed by the conclusion that Alec was actually trying to request information on how to be considered part of the group.
It seems to me like a rather uncharitable interpretation of Alec’s response, to assume that he just wants to figure out how to belong, rather than genuinely desiring to find out how best to contribute.
I would be rather insulted by this response, because it implies that I am looking for a vetted-by-the-group answer, and also seems to be criticising me for asking Anna about opimal careers. Firstly, that was never my intent. Secondly, asking an expert for their opinion sounds like a perfectly reasonable course of action to me. However, this does assume that Alec shares my motivations and assumptions.
I’m not sure of my assumptions/beliefs/conclusions though. I might be missing context (e.g. I don’t know what bay area is like, or the cultural norms), and I didn’t really understand the essay (I found the example too distracting for me to focus on the concept of narrative syncing—I like the new examples much more).
>my true question (I still don’t know what to do next!)
(The following might be a strawman; sorry if so; but it is a genuine question about a phenomenon I’ve seen.) This part sounds like you’re saying, you’re asking someone what to do next. If so, what if it’s a mistake for you to be mentally executing the procedure 1. hear what someone says to do next, 2. do that thing? How would the person you’re asking communicate to you that it’s a mistake for you to execute that procedure, without you taking their attempted statements of fact as cryptic instructions for you to follow? (Assuming that’s a mistake you think you could plausibly be making, and assuming you’d want the mistake communicated to you if you were making it.)
>in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Someone’s role is a combination of their behavior and the behavior of others’ behavior and expectations. Suppose that Berry wants to invite Alec to a workshop with an offer to help, but *not* with any implied intent to play a role of advisor/expert. How could Berry communicate that to Alec?
>What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments.
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.
Hmm if CFAR organizes a workshop, then I would think it is reasonable to assume that the CFAR staff (assuming they are introduced as such) are there as experienced members of the AI risk community who are there to offer guidance to people who are new to the area.
Thus, if I ask them about career paths, I’m not asking for their opinions as individuals, I’m asking for their opinions as people with more expertise than I have.
Two possible motivations I can think of for consulting someone with expertise would be:
In school, I ask the teacher when I have a question so they tell me the answer. Likewise, if I’m not sure of the appropriate career move, I should ask an expert in AI risk and they can tell me what to do. If I follow their answer and it fails, then it’s their fault. I would feel betrayed and blame them for wasting my life.
Someone who has been working on AI risks would have more knowledge about the area. Thus, I should consult their opinions before deciding what to do. I would feel betrayed if I later find out that they gave me a misleading answer (e.g. telling me to study ML when that’s not what they actually think).
In both cases I’m trying to figure out what to do next, but in the first case I want to be told what to do next, and in the second case I just want more information so I can decide what to do next. (Expert doesn’t mean you bear responsibility for my life decisions, just responsibility for providing accurate information.)
If I’m not wrong, you’re asking about the first case? (My comment was trying to describe the second scenario.)
If it’s just a matter of ambiguity (e.g. I will accompany you just for moral support vs accompany you to provide advice), I would just state it explicitly (e.g. tell you “I won’t be helping you. If someone asks you something and you look to me for help, I’ll just look at you and smile.”) and then do precisely what I said I’ll do.
Otherwise, if it’s a mindset issue (thinking I should do what others tell me to), it’s a deeper problem. If it’s someone I’m close to, I would address the issue directly e.g. saying “If the GPS tells you to drive into a lake and you listen and drown, whose problem is it? I can tell you what to do, but if I’m wrong, you’re the one who will suffer, not me. Even if some very wise person tells you what to do, you still have to check if it makes sense and decide if you will do it, because you have the most to lose. If other people tell you the wrong thing, they can just go ‘Oops sorry! I didn’t realise.’ You, on the other hand, are stuck picking up the pieces. Make sure you think it is worth it before implementing anyone’s advice.” And if that fails, just… let them make their mistakes and eventually they’ll learn from their experiences?
Not really, saying “I don’t know” is very different from saying “after years of research, we still don’t know”.
“I’m not sure” sounds like the kind of answer a newbie would give, so I’m not really learning anything new from the conversation. Even worse, “don’t know” sounds like you don’t really want to help me—surely an expert knows more than me, if they say they don’t know, then that must mean they just don’t want to share their knowledge.
In contrast, if you said that no one knows because the field is still too new and rapidly changing, then you are giving me an informed opinion. I now know that this is a problem even the experts haven’t solved and can then make decisions based on this information.
Anyone can say “I’m an expert”. There has to be some other way that you’re distinguishing who’s a newbie who’s just clueless, from an expert who’s ignorance is evidence of a general difficulty. From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason. E.g., at a gloss, Yudkowsky says “no one has a plan that could possibly work”, and Christiano says “no, I have a plan which, with some optimistic but not crazy assumptions, looks like it’ll probably work”, and Yudkowsky says something that people don’t understand, and Christiano responds in a way that may or may not address Yudkowsky’s critique, and who knows who’s an expert? Just going by who says “I have a plan” is helpful if someone has a plan that will work (although it also opens up social niches for con artists).
Ah yes agreed. Alec doesn’t know that this is what’s happening though (judging from his response to answer 1). Personally I’d default to assuming that Berry will play the role of expert since he’s part of CFAR while I’m just a random computer scientist (from Berry’s perspective). I would switch to a more equal dynamic only if there’s a clear indicator that I should do so.
For example, if a student asks a professor a question, the professor may ask the student for their thoughts instead and then respond thoughtfully to their answer, like they would respond to a fellow professor. Or if a boss asks a new subordinate for their opinion but the subordinate thinks this is a fake question and tries to guess the cryptic instructions instead (because sometimes people ask you questions to hint that you should answer a certain way), the boss may ask someone else who’s been on the team longer. When the new member sees the senior member responding honestly and boss engaging thoughtfully with the response, then the new member would know that the question was genuine.
In the careers example, I can’t tell from the first answer that Anna is trying to engage me as a peer. (I’d imagine someone used to different norms might assume that as the default though.)