The way I see it, yes, Natural Selection defeats the Orthogonality Thesis. But Meta-Trolling—e.g. simulators choosing the fitness function—defeats Natural Selection; and then the Orthogonality Thesis defeats Meta-Trolling, because you can’t know the value systems of the simulators, so sufficiently intelligent agents have to steelman phenomena (treat appearances as real). #trollosophy
The way I see it, yes, Natural Selection defeats the Orthogonality Thesis. But Meta-Trolling—e.g. simulators choosing the fitness function—defeats Natural Selection; and then the Orthogonality Thesis defeats Meta-Trolling, because you can’t know the value systems of the simulators, so sufficiently intelligent agents have to steelman phenomena (treat appearances as real). #trollosophy