By far the largest criticism here is the requirements of set theory:
Yudkowsky is relying on a theorem when he says that second-order Peano arithmetic has a unique model. That theorem requires a substantial dose of set theory. So in order to avoid taking numbers as primitive objects, he’s effectively resorted to taking sets as primitive objects. But why is it any more satisfying to take set theory as “given” than to take numbers as “given”? Indeed, the formal study of numbers precedes the formal study of sets by millennia, which suggests that numbers are a more natural starting point than sets are. Whether or not you buy that argument, it’s important to recognize that Yudkowsky has “solved” the problem of accounting for numbers only by reducing it to the problem of accounting for sets — except that he hasn’t even done that, because his reduction relies on pretending that second order logic is logic.
This seems most damning in that it makes most of the philosophical issues about the nature of logic irrelevant.
By far the largest criticism here is the requirements of set theory:
This seems most damning in that it makes most of the philosophical issues about the nature of logic irrelevant.