Yeah I guess I should be clear that I generally like the idea of building virtuous AI and maybe somehow this solves some of the problems we have with other designs, the trick is building something that actually implements whatever we think it means to be virtuous, which means getting precise enough about what it means to be virtuous that we can be sure we don’t simply collapse back into the default thing all negative feedback systems do: optimize for their targets as hard as they can (with “can” doing a lot of work here!).
Yeah I guess I should be clear that I generally like the idea of building virtuous AI and maybe somehow this solves some of the problems we have with other designs, the trick is building something that actually implements whatever we think it means to be virtuous, which means getting precise enough about what it means to be virtuous that we can be sure we don’t simply collapse back into the default thing all negative feedback systems do: optimize for their targets as hard as they can (with “can” doing a lot of work here!).