I updated “diamond maximization problem” to “diamond alignment problem”.
I didn’t understand your proposal to involve surgically inserting the drive to value “diamonds are good”, but instead systematically rewarding the agent for acquiring diamonds so that a diamond shard forms organically. I also edited that sentence.
I am not sure I get your Nitpick: “Just as you can deny that Newtonian mechanics is true, without denying that heavy objects attract each other.” was supposed to be an example of “The specific theory is wrong, but the general phenomenon which it tries to describe exists”. In the same way that I think Natural Abstractions exist but (my flawed understanding) of Wentworths theory of natural abstractions is wrong. It was not supposed to be an example of a natural abstraction itself.
Thanks a lot for the comment and correction :)
I updated “diamond maximization problem” to “diamond alignment problem”.
I didn’t understand your proposal to involve surgically inserting the drive to value “diamonds are good”, but instead systematically rewarding the agent for acquiring diamonds so that a diamond shard forms organically. I also edited that sentence.
I am not sure I get your Nitpick: “Just as you can deny that Newtonian mechanics is true, without denying that heavy objects attract each other.” was supposed to be an example of “The specific theory is wrong, but the general phenomenon which it tries to describe exists”. In the same way that I think Natural Abstractions exist but (my flawed understanding) of Wentworths theory of natural abstractions is wrong. It was not supposed to be an example of a natural abstraction itself.