The problem with The orthogonality thesis is not so much that it’s wrong that it is misleading. It’s a special case of the idea that we will ultimately be able to create whatever we can imagine (because our brains are VR simulators—and because of Turing completeness). The problem with it is that what we can imagine and what evolution tends to produce are different things. Failing to account for that seems consistent with fear mongering about the future—a common marketing technique for these kinds of outfit. Sure enough, the paper goes on to talk about sinister dangers.
The problem with The orthogonality thesis is not so much that it’s wrong that it is misleading. It’s a special case of the idea that we will ultimately be able to create whatever we can imagine (because our brains are VR simulators—and because of Turing completeness). The problem with it is that what we can imagine and what evolution tends to produce are different things. Failing to account for that seems consistent with fear mongering about the future—a common marketing technique for these kinds of outfit. Sure enough, the paper goes on to talk about sinister dangers.