(Eliezer, why do you keep using “intelligence” to mean “optimization” even after agreeing with me that intelligence includes other things that we don’t yet understand?)
Morality does not compress
You can’t mean that morality literally does not compress (i.e. is truly random). Obviously there are plenty of compressible regularities in human morality. So perhaps what you mean is that it’s too hard or impossible to compress it into a small enough description that humans can understand. But, we also have no evidence that effective universal optimization in the presence of real-world computational constraints (as opposed to idealized optimization with unlimited computing power) can be compressed into a small enough description that humans can understand.
(Eliezer, why do you keep using “intelligence” to mean “optimization” even after agreeing with me that intelligence includes other things that we don’t yet understand?)
Morality does not compress
You can’t mean that morality literally does not compress (i.e. is truly random). Obviously there are plenty of compressible regularities in human morality. So perhaps what you mean is that it’s too hard or impossible to compress it into a small enough description that humans can understand. But, we also have no evidence that effective universal optimization in the presence of real-world computational constraints (as opposed to idealized optimization with unlimited computing power) can be compressed into a small enough description that humans can understand.