How does a self-improving system improve itself, without discovering contradictions or gaps in its values?
By getting a faster brain, more memory, more stored resources and a better world model, perhaps.
Values don’t have to have “contradictions” or “gaps” in them. Say you value printing out big prime numbers. Where are the contradictions or gaps going to come from?
Does value freeze require knowledge freeze?
Usually values and knowledge are considered to be orthogoonal—so “no”.
How does a self-improving system improve itself, without discovering contradictions or gaps in its values? Does value freeze require knowledge freeze?
By getting a faster brain, more memory, more stored resources and a better world model, perhaps.
Values don’t have to have “contradictions” or “gaps” in them. Say you value printing out big prime numbers. Where are the contradictions or gaps going to come from?
Usually values and knowledge are considered to be orthogoonal—so “no”.