Gotcha, that makes sense. It sounds like we agree that the main question is of who the experts are and how much weight one should give to each of them. It also sounds like we disagree about what the answer to that question is. I give a lot of weight to Yudkowsky and adjacent people whereas it sounds like you don’t.
Consider two tennis pundits, one of whom has won prestigious awards for tennis punditry, and is currently employed by a large TV network to comment on matches. The other is a somewhat-popular tennis youtuber. If these two disagree about who’s going to win the match, with the famous TV commentator favouring Joe and the youtuber favouring Bill, a total tennis outsider would probably do best by going with the opinion of the famous guy. In order to figure out that you should instead go with the opinion of the youtuber, you’d need to know at least a little bit about tennis in order to determine that the youtuber is actually more knowledgeable.
Agreed. But I think the AIS situation is more analogous to if the YouTuber studied some specific aspect of tennis very extensively that traditional tennis players don’t really pay much attention to. Even so, it still might be difficult for an outsider to place much weight behind the YouTuber. It’s probably pretty important that they can judge for themself that the YouTuber is smart and qualified and stuff. It also helps that the YouTuber received support and funding and endorsements from various high-prestige people.
I give a lot of weight to Yudkowsky and adjacent people whereas it sounds like you don’t.
To be clear, I do give a lot of weight to Yudkowsky in the sense that I think his arguments make sense and I mostly believe them. Similarly, I don’t give much weight to Yann LeCun on this topic. But that’s because I can read what Yudkowsky has said and what LeCun has said and think about whether it made sense. If I didn’t speak English, so that their words appeared as meaningless noise to me, then I’d be much more uncertain about who to trust, and would probably defer to an average of the opinions of the top ML names, eg. Sutskever, Goodfellow, Hinton, LeCun, Karpathy, Benigo, etc. The thing about closely studying a specific aspect of AI (namely alignment) would probably get Yudkowsky and Christiano’s names onto that list, but it wouldn’t necessarily give Yudkowsky more weight than everyone else combined. (I’m guessing, for hypothetical non-English-speaking me, who somehow has translations for what everyone’s bottom line position is on the topic, but not what their arguments are. Basically the intuition here is that difficult technical achievements like Alexnet, GANs, etc. are some of the easiest things to verify from the outside. It’s hard to tell which philosopher is right, but easy to tell which scientist can build a thing for you that will automatically generate amusing new animal pictures.)
If I didn’t speak English, so that their words appeared as meaningless noise to me, then I’d be much more uncertain about who to trust, and would probably defer to an average of the opinions of the top ML names, eg. Sutskever, Goodfellow, Hinton, LeCun, Karpathy, Benigo, etc.
Do you think it’d make sense to give more weight to people in the field of AI safety than to people in the field of AI more broadly?
I would, and I think it’s something that generally makes sense. Ie. I don’t know much about food science but on a question involving dairy, I’d trust food scientists who specialize in dairy more than I would trust food scientists more generally. But I would give some weight to non-specialist food scientists, as well as chemists in general, as well as physical scientists in general, with the weight decreasing as the person gets less specialized.
Gotcha, that makes sense. It sounds like we agree that the main question is of who the experts are and how much weight one should give to each of them. It also sounds like we disagree about what the answer to that question is. I give a lot of weight to Yudkowsky and adjacent people whereas it sounds like you don’t.
Agreed. But I think the AIS situation is more analogous to if the YouTuber studied some specific aspect of tennis very extensively that traditional tennis players don’t really pay much attention to. Even so, it still might be difficult for an outsider to place much weight behind the YouTuber. It’s probably pretty important that they can judge for themself that the YouTuber is smart and qualified and stuff. It also helps that the YouTuber received support and funding and endorsements from various high-prestige people.
To be clear, I do give a lot of weight to Yudkowsky in the sense that I think his arguments make sense and I mostly believe them. Similarly, I don’t give much weight to Yann LeCun on this topic. But that’s because I can read what Yudkowsky has said and what LeCun has said and think about whether it made sense. If I didn’t speak English, so that their words appeared as meaningless noise to me, then I’d be much more uncertain about who to trust, and would probably defer to an average of the opinions of the top ML names, eg. Sutskever, Goodfellow, Hinton, LeCun, Karpathy, Benigo, etc. The thing about closely studying a specific aspect of AI (namely alignment) would probably get Yudkowsky and Christiano’s names onto that list, but it wouldn’t necessarily give Yudkowsky more weight than everyone else combined. (I’m guessing, for hypothetical non-English-speaking me, who somehow has translations for what everyone’s bottom line position is on the topic, but not what their arguments are. Basically the intuition here is that difficult technical achievements like Alexnet, GANs, etc. are some of the easiest things to verify from the outside. It’s hard to tell which philosopher is right, but easy to tell which scientist can build a thing for you that will automatically generate amusing new animal pictures.)
Do you think it’d make sense to give more weight to people in the field of AI safety than to people in the field of AI more broadly?
I would, and I think it’s something that generally makes sense. Ie. I don’t know much about food science but on a question involving dairy, I’d trust food scientists who specialize in dairy more than I would trust food scientists more generally. But I would give some weight to non-specialist food scientists, as well as chemists in general, as well as physical scientists in general, with the weight decreasing as the person gets less specialized.