The argument in that post seems incoherent to me. In the conventional natural sciences, what is moral is the subject matter of biology. This employs game theory and evolutionary theory (i.e. logic), but also considers the laws of physics and the local state of the universe to explain existing moral systems.
For instance, consider the question of whether it is wrong to drive on the left-hand side of the road. That isn’t logic, it depends on the local state of the universe. Two advanced superintelligences which had evolved independently could easily find themselves in disagreement over such issues.
This is an example of spontaneous symmetry breaking. It is one of the factors which explains how arbitrarily-advanced agents can still disagree on what the right thing to do is.
The argument in that post seems incoherent to me. In the conventional natural sciences, what is moral is the subject matter of biology. This employs game theory and evolutionary theory (i.e. logic), but also considers the laws of physics and the local state of the universe to explain existing moral systems.
For instance, consider the question of whether it is wrong to drive on the left-hand side of the road. That isn’t logic, it depends on the local state of the universe. Two advanced superintelligences which had evolved independently could easily find themselves in disagreement over such issues.
This is an example of spontaneous symmetry breaking. It is one of the factors which explains how arbitrarily-advanced agents can still disagree on what the right thing to do is.