Promoted to curated: I don’t think this post is perfect and both sides presented feel like they have a component of being strawmen to it, but I still found it a pretty good way of helping me understand the contrast between two perspectives, both of which I am often pretty sympathetic to (though again, neither of the sides here feel like they are getting close to perfect at representing them).
After skimming through this post, I thought the OP wouldn’t pass ITT for one of the perspectives he tried to represent in the post., so I was surprised to see it in my mailbox later. The contrast is between a more imaginary position and a less imaginary position. E.g., I would be happy to bet Yudkowsky or Soares, when asked, will say they wouldn’t be mentioning the fact that “capabilities generalize further than alignment” in the context it was mentioned in. We can discuss the odds if you’re potentially interested. I don’t think anyone even remotely close to understanding this fact would use it in that context.
I don’t think anyone even remotely close to understanding this fact would use it in that context.
I dispute this claim being called a “fact”, but I’m open to having my mind changed. Is there a clear explanation of this claim somewhere? I’ve read Lethalities and assorted bits of Nate’s writing.
I haven’t seen a clear explanation that I’d expect to change your mind. Some people understood a dynamic the claim refers to after reading the sharp left turn post (but most people I talked to didn’t, although I’m not sure whether they actually read it or just skimmed through it). If you haven’t read it, might be worth a try
Thanks for commenting. Is it any better if I delete that specific phrase, such that Doomimir’s line ends on “the results look like random garbage to you”? (Post edited.) I think there’s a legitimate point being made in that line (that the repetition behavior isn’t necessarily “wrong” “from the LLM’s perspective”, however wrong it looks to the user), even if it’s not the same point as “List of Lethalities” #21 (and the text I originally wrote was bad for suggesting that it was).
Yeah, I found this helpful. In general, I’d like to see more of these dialogues. I think that they do a good job of synthesing different arguments in an accessible way. I feel that’s increasingly important as more arguments emerge.
As an aside, I like the way that the content goes from relatively accessible high level discussion and analogy into more specific technical detail. I think this makes it much more accessible to novice and non-technical readers.
Promoted to curated: I don’t think this post is perfect and both sides presented feel like they have a component of being strawmen to it, but I still found it a pretty good way of helping me understand the contrast between two perspectives, both of which I am often pretty sympathetic to (though again, neither of the sides here feel like they are getting close to perfect at representing them).
After skimming through this post, I thought the OP wouldn’t pass ITT for one of the perspectives he tried to represent in the post., so I was surprised to see it in my mailbox later. The contrast is between a more imaginary position and a less imaginary position. E.g., I would be happy to bet Yudkowsky or Soares, when asked, will say they wouldn’t be mentioning the fact that “capabilities generalize further than alignment” in the context it was mentioned in. We can discuss the odds if you’re potentially interested. I don’t think anyone even remotely close to understanding this fact would use it in that context.
I dispute this claim being called a “fact”, but I’m open to having my mind changed. Is there a clear explanation of this claim somewhere? I’ve read Lethalities and assorted bits of Nate’s writing.
I haven’t seen a clear explanation that I’d expect to change your mind. Some people understood a dynamic the claim refers to after reading the sharp left turn post (but most people I talked to didn’t, although I’m not sure whether they actually read it or just skimmed through it). If you haven’t read it, might be worth a try
Thanks for commenting. Is it any better if I delete that specific phrase, such that Doomimir’s line ends on “the results look like random garbage to you”? (Post edited.) I think there’s a legitimate point being made in that line (that the repetition behavior isn’t necessarily “wrong” “from the LLM’s perspective”, however wrong it looks to the user), even if it’s not the same point as “List of Lethalities” #21 (and the text I originally wrote was bad for suggesting that it was).
Yeah, I found this helpful. In general, I’d like to see more of these dialogues. I think that they do a good job of synthesing different arguments in an accessible way. I feel that’s increasingly important as more arguments emerge.
As an aside, I like the way that the content goes from relatively accessible high level discussion and analogy into more specific technical detail. I think this makes it much more accessible to novice and non-technical readers.