Andrew McKnight

An­drew McKnight’s Shortform

• Andrew McKnightDec 2, 2021, 10:12 PM

  5 points

  in reply to: Tee Canker’s comment on: Mo­ral­ity is Scary

  I think the main thing you’re missing here is that an AI is not generally going to share common learning facilities with humans. An AI growing up as a human will make it wildly different from a normal human because they aren’t built precisely to learn from those experiences the way a human does.

• Andrew McKnightNov 24, 2021, 10:11 PM

  2 points

  AF

  in reply to: Koen.Holtman’s comment on: Ngo and Yud­kowsky on al­ign­ment difficulty

  I haven’t read your papers but your proposal seems like it would scale up until the point when the AGI looks at itself. If it can’t learn at this point then I find it hard to believe it’s generally capable, and if it can, it will have incentive to simply remove the device or create a copy of itself that is correct about its own world model. Do you address this in the articles?

  On the other hand, this made me curious about what we could do with an advanced model that is instructed to not learn and also whether we can even define and ensure a model stops learning.

• Andrew McKnightNov 19, 2021, 9:55 PM

  15 points

  in reply to: Rafael Harth’s comment on: Ngo and Yud­kowsky on AI ca­pa­bil­ity gains

  I agree that this thread makes it clearer why takeoff speeds matter to people but I always want to ask why people think sufficient work is going to get done in that extended 4-10 years even with access to proto-AGI to directly study.

• Andrew McKnightNov 17, 2021, 9:11 PM

  7 points

  in reply to: Rob Bensinger’s comment on: At­tempted Gears Anal­y­sis of AGI In­ter­ven­tion Dis­cus­sion With Eliezer

  Thanks. This is great! I hadn’t thought of Embedded Agency as an attempt to understand optimization. I thought it was an attempt to ground optimizers in a formalism that wouldn’t behave wildly once they had to start interacting with themselves. But on second thought it makes sense to consider an optimizer that can’t handle interacting with itself to be a broken or limited optimizer.

• Andrew McKnightNov 16, 2021, 11:32 PM

  7 points

  in reply to: benjamincosman’s comment on: Dis­cus­sion with Eliezer Yud­kowsky on AGI interventions

  No one has yet solved “and then stop” for AGI even though this should be easier than a generic stop button which in turn should be easier than full corrigibility. (Also I don’t think we know how to refer to things in the world in a way that gets an AI to care about it rather than observations of it or its representation of it)

• Andrew McKnightNov 16, 2021, 10:31 PM

  4 points

  in reply to: Rob Bensinger’s comment on: At­tempted Gears Anal­y­sis of AGI In­ter­ven­tion Dis­cus­sion With Eliezer

  the ways in which solving AF would likely be useful

  Other than the rocket alignment analogy and the general case for deconfusion helping, has anyone ever tried to describe with more concrete (though speculative) detail how AF would help with alignment? I’m not saying it wouldn’t. I just literally want to know if anyone has tried explaining this concretely. I’ve been following for a decade but don’t think I ever saw an attempted explanation.