Conscience, as you’ve defined it, is value alignment. If the AI values the same things that we do, when offered a choice between two courses of action, it will choose the one that serves to enhance human values rather than degrade them. Designing an AI that does this, with no exceptions, is very hard.
In your opinion, what’s the biggest challenge about feeding a DNN with human values, and then adjusting the biases in such a manner that it’s not degrading them?
We’ve taught AI how to speak, and it appears that openAI has taught their AI how to produce as little offensive content as possible. So it seems to be feasible, or not?
We’ve taught AI how to speak, and it appears that openAI has taught their AI how to produce as little offensive content as possible.
The problem is that the AI can (and does) lie. Right now, ChatGPT and its ilk are a less than superhuman levels of intelligence, so we can catch their lies. But when a superhuman AI starts lying to you, how does one correct for that? If a superhuman AI starts veering off in a direction that is unexpected, how does one bring it back on track?
@gwern short story, Clippy highlights many of the issues with naively training a superintelligent algorithm on human-generated data and expecting that algorithm to pick up human values as a result. Another post to consider is The Waluigi Effect, which raises the possibility that the more you train an agent to say correct, inoffensive things, the more you’ve also trained a shadow-agent to say incorrect, offensive things.
Isn’t that exactly why we should develop an artificial conscience, to prevent an AI from lying or having a shadow side?
A built in conscience would let the AI know that lying is not something it should do. Also, using a conscience in the AI algorithm would make the AI combat it’s own potential shadow. It’ll have knowledge of right and wrong / good or bad, and it’s even got superhuman ability to orient itself towards that which is good & right, rather than to be “seduced” by the dark side.
Ah, but how do you make the artificial conscience value aligned with humanity? An “artificial conscience” that is capable of aligning a superhuman AI… would itself be an aligned superhuman AI.
Conscience, as you’ve defined it, is value alignment. If the AI values the same things that we do, when offered a choice between two courses of action, it will choose the one that serves to enhance human values rather than degrade them. Designing an AI that does this, with no exceptions, is very hard.
Thank you for your comment!
In your opinion, what’s the biggest challenge about feeding a DNN with human values, and then adjusting the biases in such a manner that it’s not degrading them?
We’ve taught AI how to speak, and it appears that openAI has taught their AI how to produce as little offensive content as possible. So it seems to be feasible, or not?
The problem is that the AI can (and does) lie. Right now, ChatGPT and its ilk are a less than superhuman levels of intelligence, so we can catch their lies. But when a superhuman AI starts lying to you, how does one correct for that? If a superhuman AI starts veering off in a direction that is unexpected, how does one bring it back on track?
@gwern short story, Clippy highlights many of the issues with naively training a superintelligent algorithm on human-generated data and expecting that algorithm to pick up human values as a result. Another post to consider is The Waluigi Effect, which raises the possibility that the more you train an agent to say correct, inoffensive things, the more you’ve also trained a shadow-agent to say incorrect, offensive things.
Makes perfect sense!
Isn’t that exactly why we should develop an artificial conscience, to prevent an AI from lying or having a shadow side?
A built in conscience would let the AI know that lying is not something it should do. Also, using a conscience in the AI algorithm would make the AI combat it’s own potential shadow. It’ll have knowledge of right and wrong / good or bad, and it’s even got superhuman ability to orient itself towards that which is good & right, rather than to be “seduced” by the dark side.
Ah, but how do you make the artificial conscience value aligned with humanity? An “artificial conscience” that is capable of aligning a superhuman AI… would itself be an aligned superhuman AI.
Correct! That’s my point with the main post. I don’t see anyone discussing conscience, I mostly hear them contemplate consciousness or computability.
As far as how to actually do this, I’ve dropped a few ideas on this site, they should be listed on my profile.