But a supersmart, uper-rational clipper has to be able to update.
has to be able to update
“update”
Please unpack this and describe precisely, in algorithmic terms that I could read and write as a computer program given unlimited time and effort, this “ability to update” which you are referring to.
I suspect that you are attributing Magical Powers From The Beyond to the word “update”, and forgetting to consider that the ability to self-modify does not imply active actions to self-modify in any one particular way that unrelated data bits say would be “better”, unless the action code explicitly looks for said data bits.
It’s uncontrovesial that rational agents need to update, and that AIs need to self-modify. The claim that values are in either case insulated from updates is the extraordinary one. The Cipper theory tells you that you could build something like that if you were crazy enough. Since Clippers are contrived, nothing can be inferred from them about
typical agents. People are messy, and can accidentally update their values when trying to do something else, For instance, LukeProg updated to “atheist” after studying Christian apologetics for the opposite reason.
Yes, value drift is the typical state for minds in our experience.
Building a committed Clipper that cannot accidentally update its values when trying to do something else is only possible after the problem of value drift has been solved. A system that experiences value drift isn’t a reliable Clipper, isn’t a reliable good-thing-doer, isn’t reliable at all.
It’s uncontrovesial that rational agents need to update, and that AIs need to self-modify. The claim that values are in either case insulated from updates is the extraordinary one.
I never claimed that it was controversial, nor that AIs didn’t need to self-modify, nor that values are exempt.
I’m claiming that updates and self modification do not imply a change of behavior towards behavior desired by humans.
I can build a small toy program to illustrate, if that would help.
I am not suggesting that human ethics is coincidentally universal ethics.
I am suggesting that if neither moral realism nor relativism is initially discarded, one can eventually arrive at a compromise position where rational agents in a particular context arrive at a non arbitrary ethics which is appropriate to that context.
Other key problem:
Please unpack this and describe precisely, in algorithmic terms that I could read and write as a computer program given unlimited time and effort, this “ability to update” which you are referring to.
I suspect that you are attributing Magical Powers From The Beyond to the word “update”, and forgetting to consider that the ability to self-modify does not imply active actions to self-modify in any one particular way that unrelated data bits say would be “better”, unless the action code explicitly looks for said data bits.
It’s uncontrovesial that rational agents need to update, and that AIs need to self-modify. The claim that values are in either case insulated from updates is the extraordinary one. The Cipper theory tells you that you could build something like that if you were crazy enough. Since Clippers are contrived, nothing can be inferred from them about typical agents. People are messy, and can accidentally update their values when trying to do something else, For instance, LukeProg updated to “atheist” after studying Christian apologetics for the opposite reason.
Yes, value drift is the typical state for minds in our experience.
Building a committed Clipper that cannot accidentally update its values when trying to do something else is only possible after the problem of value drift has been solved. A system that experiences value drift isn’t a reliable Clipper, isn’t a reliable good-thing-doer, isn’t reliable at all.
Next.
I never claimed that it was controversial, nor that AIs didn’t need to self-modify, nor that values are exempt.
I’m claiming that updates and self modification do not imply a change of behavior towards behavior desired by humans.
I can build a small toy program to illustrate, if that would help.
I am not suggesting that human ethics is coincidentally universal ethics. I am suggesting that if neither moral realism nor relativism is initially discarded, one can eventually arrive at a compromise position where rational agents in a particular context arrive at a non arbitrary ethics which is appropriate to that context.