But even MIRI says that alignment is “incredibly hard”, not “impossible”.
Yes. I’m not saying it is impossible, even though I’m deeply skeptical. That a character in my novel says it’s impossible doesn’t necessarily reflect my own opinion. I guess I’m as optimistic about it as Eliezer Yudkowsky. :( I could go into the details, but it probably doesn’t make sense to discuss this here in the comments. I’m not much of an expert anyway. Still, if someone claims to have solved alignment, I’d like to see a proof. In any case, I’m convinced that it is MUCH easier to prevent an AI-related catastrophe by not developing an uncontrollable AI than by solving alignment, at least in the short term. So what we need now, I think, is more caution, not more optimism. I’d be very, very happy if it turns out that I was overly pessimistic and everything goes well.
Yes. I’m not saying it is impossible, even though I’m deeply skeptical. That a character in my novel says it’s impossible doesn’t necessarily reflect my own opinion. I guess I’m as optimistic about it as Eliezer Yudkowsky. :( I could go into the details, but it probably doesn’t make sense to discuss this here in the comments. I’m not much of an expert anyway. Still, if someone claims to have solved alignment, I’d like to see a proof. In any case, I’m convinced that it is MUCH easier to prevent an AI-related catastrophe by not developing an uncontrollable AI than by solving alignment, at least in the short term. So what we need now, I think, is more caution, not more optimism. I’d be very, very happy if it turns out that I was overly pessimistic and everything goes well.