>my true question (I still don’t know what to do next!)
(The following might be a strawman; sorry if so; but it is a genuine question about a phenomenon I’ve seen.) This part sounds like you’re saying, you’re asking someone what to do next. If so, what if it’s a mistake for you to be mentally executing the procedure 1. hear what someone says to do next, 2. do that thing? How would the person you’re asking communicate to you that it’s a mistake for you to execute that procedure, without you taking their attempted statements of fact as cryptic instructions for you to follow? (Assuming that’s a mistake you think you could plausibly be making, and assuming you’d want the mistake communicated to you if you were making it.)
>in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Someone’s role is a combination of their behavior and the behavior of others’ behavior and expectations. Suppose that Berry wants to invite Alec to a workshop with an offer to help, but *not* with any implied intent to play a role of advisor/expert. How could Berry communicate that to Alec?
>What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments.
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.
Hmm if CFAR organizes a workshop, then I would think it is reasonable to assume that the CFAR staff (assuming they are introduced as such) are there as experienced members of the AI risk community who are there to offer guidance to people who are new to the area.
Thus, if I ask them about career paths, I’m not asking for their opinions as individuals, I’m asking for their opinions as people with more expertise than I have.
Two possible motivations I can think of for consulting someone with expertise would be:
In school, I ask the teacher when I have a question so they tell me the answer. Likewise, if I’m not sure of the appropriate career move, I should ask an expert in AI risk and they can tell me what to do. If I follow their answer and it fails, then it’s their fault. I would feel betrayed and blame them for wasting my life.
Someone who has been working on AI risks would have more knowledge about the area. Thus, I should consult their opinions before deciding what to do. I would feel betrayed if I later find out that they gave me a misleading answer (e.g. telling me to study ML when that’s not what they actually think).
In both cases I’m trying to figure out what to do next, but in the first case I want to be told what to do next, and in the second case I just want more information so I can decide what to do next. (Expert doesn’t mean you bear responsibility for my life decisions, just responsibility for providing accurate information.)
If I’m not wrong, you’re asking about the first case? (My comment was trying to describe the second scenario.)
If it’s just a matter of ambiguity (e.g. I will accompany you just for moral support vs accompany you to provide advice), I would just state it explicitly (e.g. tell you “I won’t be helping you. If someone asks you something and you look to me for help, I’ll just look at you and smile.”) and then do precisely what I said I’ll do.
Otherwise, if it’s a mindset issue (thinking I should do what others tell me to), it’s a deeper problem. If it’s someone I’m close to, I would address the issue directly e.g. saying “If the GPS tells you to drive into a lake and you listen and drown, whose problem is it? I can tell you what to do, but if I’m wrong, you’re the one who will suffer, not me. Even if some very wise person tells you what to do, you still have to check if it makes sense and decide if you will do it, because you have the most to lose. If other people tell you the wrong thing, they can just go ‘Oops sorry! I didn’t realise.’ You, on the other hand, are stuck picking up the pieces. Make sure you think it is worth it before implementing anyone’s advice.” And if that fails, just… let them make their mistakes and eventually they’ll learn from their experiences?
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.
Not really, saying “I don’t know” is very different from saying “after years of research, we still don’t know”.
“I’m not sure” sounds like the kind of answer a newbie would give, so I’m not really learning anything new from the conversation. Even worse, “don’t know” sounds like you don’t really want to help me—surely an expert knows more than me, if they say they don’t know, then that must mean they just don’t want to share their knowledge.
In contrast, if you said that no one knows because the field is still too new and rapidly changing, then you are giving me an informed opinion. I now know that this is a problem even the experts haven’t solved and can then make decisions based on this information.
Anyone can say “I’m an expert”. There has to be some other way that you’re distinguishing who’s a newbie who’s just clueless, from an expert who’s ignorance is evidence of a general difficulty. From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason. E.g., at a gloss, Yudkowsky says “no one has a plan that could possibly work”, and Christiano says “no, I have a plan which, with some optimistic but not crazy assumptions, looks like it’ll probably work”, and Yudkowsky says something that people don’t understand, and Christiano responds in a way that may or may not address Yudkowsky’s critique, and who knows who’s an expert? Just going by who says “I have a plan” is helpful if someone has a plan that will work (although it also opens up social niches for con artists).
From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason.
Ah yes agreed. Alec doesn’t know that this is what’s happening though (judging from his response to answer 1). Personally I’d default to assuming that Berry will play the role of expert since he’s part of CFAR while I’m just a random computer scientist (from Berry’s perspective). I would switch to a more equal dynamic only if there’s a clear indicator that I should do so.
For example, if a student asks a professor a question, the professor may ask the student for their thoughts instead and then respond thoughtfully to their answer, like they would respond to a fellow professor. Or if a boss asks a new subordinate for their opinion but the subordinate thinks this is a fake question and tries to guess the cryptic instructions instead (because sometimes people ask you questions to hint that you should answer a certain way), the boss may ask someone else who’s been on the team longer. When the new member sees the senior member responding honestly and boss engaging thoughtfully with the response, then the new member would know that the question was genuine.
In the careers example, I can’t tell from the first answer that Anna is trying to engage me as a peer. (I’d imagine someone used to different norms might assume that as the default though.)
>my true question (I still don’t know what to do next!)
(The following might be a strawman; sorry if so; but it is a genuine question about a phenomenon I’ve seen.) This part sounds like you’re saying, you’re asking someone what to do next. If so, what if it’s a mistake for you to be mentally executing the procedure 1. hear what someone says to do next, 2. do that thing? How would the person you’re asking communicate to you that it’s a mistake for you to execute that procedure, without you taking their attempted statements of fact as cryptic instructions for you to follow? (Assuming that’s a mistake you think you could plausibly be making, and assuming you’d want the mistake communicated to you if you were making it.)
>in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Someone’s role is a combination of their behavior and the behavior of others’ behavior and expectations. Suppose that Berry wants to invite Alec to a workshop with an offer to help, but *not* with any implied intent to play a role of advisor/expert. How could Berry communicate that to Alec?
>What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments.
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.
Hmm if CFAR organizes a workshop, then I would think it is reasonable to assume that the CFAR staff (assuming they are introduced as such) are there as experienced members of the AI risk community who are there to offer guidance to people who are new to the area.
Thus, if I ask them about career paths, I’m not asking for their opinions as individuals, I’m asking for their opinions as people with more expertise than I have.
Two possible motivations I can think of for consulting someone with expertise would be:
In school, I ask the teacher when I have a question so they tell me the answer. Likewise, if I’m not sure of the appropriate career move, I should ask an expert in AI risk and they can tell me what to do. If I follow their answer and it fails, then it’s their fault. I would feel betrayed and blame them for wasting my life.
Someone who has been working on AI risks would have more knowledge about the area. Thus, I should consult their opinions before deciding what to do. I would feel betrayed if I later find out that they gave me a misleading answer (e.g. telling me to study ML when that’s not what they actually think).
In both cases I’m trying to figure out what to do next, but in the first case I want to be told what to do next, and in the second case I just want more information so I can decide what to do next. (Expert doesn’t mean you bear responsibility for my life decisions, just responsibility for providing accurate information.)
If I’m not wrong, you’re asking about the first case? (My comment was trying to describe the second scenario.)
If it’s just a matter of ambiguity (e.g. I will accompany you just for moral support vs accompany you to provide advice), I would just state it explicitly (e.g. tell you “I won’t be helping you. If someone asks you something and you look to me for help, I’ll just look at you and smile.”) and then do precisely what I said I’ll do.
Otherwise, if it’s a mindset issue (thinking I should do what others tell me to), it’s a deeper problem. If it’s someone I’m close to, I would address the issue directly e.g. saying “If the GPS tells you to drive into a lake and you listen and drown, whose problem is it? I can tell you what to do, but if I’m wrong, you’re the one who will suffer, not me. Even if some very wise person tells you what to do, you still have to check if it makes sense and decide if you will do it, because you have the most to lose. If other people tell you the wrong thing, they can just go ‘Oops sorry! I didn’t realise.’ You, on the other hand, are stuck picking up the pieces. Make sure you think it is worth it before implementing anyone’s advice.” And if that fails, just… let them make their mistakes and eventually they’ll learn from their experiences?
Not really, saying “I don’t know” is very different from saying “after years of research, we still don’t know”.
“I’m not sure” sounds like the kind of answer a newbie would give, so I’m not really learning anything new from the conversation. Even worse, “don’t know” sounds like you don’t really want to help me—surely an expert knows more than me, if they say they don’t know, then that must mean they just don’t want to share their knowledge.
In contrast, if you said that no one knows because the field is still too new and rapidly changing, then you are giving me an informed opinion. I now know that this is a problem even the experts haven’t solved and can then make decisions based on this information.
Anyone can say “I’m an expert”. There has to be some other way that you’re distinguishing who’s a newbie who’s just clueless, from an expert who’s ignorance is evidence of a general difficulty. From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason. E.g., at a gloss, Yudkowsky says “no one has a plan that could possibly work”, and Christiano says “no, I have a plan which, with some optimistic but not crazy assumptions, looks like it’ll probably work”, and Yudkowsky says something that people don’t understand, and Christiano responds in a way that may or may not address Yudkowsky’s critique, and who knows who’s an expert? Just going by who says “I have a plan” is helpful if someone has a plan that will work (although it also opens up social niches for con artists).
Ah yes agreed. Alec doesn’t know that this is what’s happening though (judging from his response to answer 1). Personally I’d default to assuming that Berry will play the role of expert since he’s part of CFAR while I’m just a random computer scientist (from Berry’s perspective). I would switch to a more equal dynamic only if there’s a clear indicator that I should do so.
For example, if a student asks a professor a question, the professor may ask the student for their thoughts instead and then respond thoughtfully to their answer, like they would respond to a fellow professor. Or if a boss asks a new subordinate for their opinion but the subordinate thinks this is a fake question and tries to guess the cryptic instructions instead (because sometimes people ask you questions to hint that you should answer a certain way), the boss may ask someone else who’s been on the team longer. When the new member sees the senior member responding honestly and boss engaging thoughtfully with the response, then the new member would know that the question was genuine.
In the careers example, I can’t tell from the first answer that Anna is trying to engage me as a peer. (I’d imagine someone used to different norms might assume that as the default though.)