Bah! :D It’s sad to hear he’s updated away from ambitions value learning towards corrigiblity-like targets. Eliezer’s second-hand argument sounds circular to me; suppose that corrigibility as we’d recognize it isn’t a natural abstraction—then generic AIs wouldn’t use it to align child agents (instead doing something like value learning, or something even more direct), and so there wouldn’t be a bunch of human-independent examples, so it wouldn’t show up as a natural abstraction to those AIs.
I think the pointer “the thing I would do if I wanted to make a second AI that would be the best one I could make at my given intelligence” is what is being updated in favor of, since this does feel like a natural abstraction, given how many agents would think this (also seems very similar to the golden rule. “I will do what I would want a successor AI to do if the successor AI was actually the human’s successor AI”. or “treat others (the human) how I’d like to be treated (by a successor AI), (and abstracting one meta-level upwards)”). Whether this turns out to be value learning or something else 🤷. This seems a different question from whether or not it is indeed a natural abstraction.
Interesting. What is it that potentially makes “treat the human like I would like to be treated if I had their values” easier than “treat the human like they would like to be treated”?
John usually does not make his plans with an eye toward making things easier. His plan previously involved values because he thought they were strictly harder than corrigibility. If you solve values, you solve corrigibility. Similarly, if you solve abstraction, you solve interpretability, shard theory, value alignment, corrigibility, etc.
I don’t know all the details of John’s model here, but it may go something like this: If you solve corrigibility, and then find out corrigibility isn’t sufficient for alignment, you may expect your corrigible agent to help you build your value aligned agent.
Bah! :D It’s sad to hear he’s updated away from ambitions value learning towards corrigiblity-like targets. Eliezer’s second-hand argument sounds circular to me; suppose that corrigibility as we’d recognize it isn’t a natural abstraction—then generic AIs wouldn’t use it to align child agents (instead doing something like value learning, or something even more direct), and so there wouldn’t be a bunch of human-independent examples, so it wouldn’t show up as a natural abstraction to those AIs.
I think the pointer “the thing I would do if I wanted to make a second AI that would be the best one I could make at my given intelligence” is what is being updated in favor of, since this does feel like a natural abstraction, given how many agents would think this (also seems very similar to the golden rule. “I will do what I would want a successor AI to do if the successor AI was actually the human’s successor AI”. or “treat others (the human) how I’d like to be treated (by a successor AI), (and abstracting one meta-level upwards)”). Whether this turns out to be value learning or something else 🤷. This seems a different question from whether or not it is indeed a natural abstraction.
Interesting. What is it that potentially makes “treat the human like I would like to be treated if I had their values” easier than “treat the human like they would like to be treated”?
John usually does not make his plans with an eye toward making things easier. His plan previously involved values because he thought they were strictly harder than corrigibility. If you solve values, you solve corrigibility. Similarly, if you solve abstraction, you solve interpretability, shard theory, value alignment, corrigibility, etc.
I don’t know all the details of John’s model here, but it may go something like this: If you solve corrigibility, and then find out corrigibility isn’t sufficient for alignment, you may expect your corrigible agent to help you build your value aligned agent.
In what way do you think solving abstraction would solve shard theory?