Clearly, no one told them about the formal definition of Occam’s Razor, in whispered apprenticeship or otherwise.
“The”...?
Why use occams razor at all?
If we were only interested in empirical adequacy, the ability to make accurate predictions, simplicity only buys the ability to make predictions with fewer calculations. But SI, according to Yudkowsky (but not Solomonoff) doesn’t just make predictions, it tells you you true facts about the world .
If you are using a simplicity criterion to decide between theories that already known to be predictive , as in Solomonoff induction, then simplicity doesn’t buy you any extra predictiveness, so the extra factor it buys you is presumably truth.
There are multiple simplicity criteria, but not multiple truths. So you need the right simplicity criterion.
If you have a conceptually valid simplicity critetion, and you formalise it, then thats as good as it gets, you’ve ticked all the boxes.
If you formalise a simplicity criterion that has no known relationship to truth, then you haven’t achieved anything. So it is not enough to say that Solomojnoff is “the” formal standard of simplicity. There are any number of ways of conceptualising simplicity, and you need the right one.
Consider this exchange, from “A semi technical introduction to Solomonoff Induction”
.
“ASHLEY: Uh, but you didn’t actually use the notion of computational simplicity to get that conclusion; you just required that the supply of probability mass is finite and the supply of potential complications is infinite. Any way of counting discrete complications would imply that conclusion, even if it went by surface wheels and gears.
BLAINE: Well, maybe. But it so happens that Yudkowsky did invent or reinvent that argument after pondering Solomonoff induction, and if it predates him (or Solomonoff) then Yudkowsky doesn’t know the source. Concrete inspiration for simplified arguments is also a credit to a theory, especially if the simplified argument didn’t exist before that.
ASHLEY: Fair enough.”
I think Ashley deserves an answer to “the objection “[a]ny way of counting discrete complications would imply that conclusion, even if it went by surface wheels and gears”, not a claim about who invented what first!
Or you could write a theory in English, and count the number of letters...that’s formal. But what has it to do with truth and reality? But what, equally, does a count of machine code instructions have to do with truth or probability?
There is one interpretation of Occam’s razor, the epistemic interpretation of it, that has the required properties. If you consider a theory as a conjunction if propositions having a probability less than one, then all else being equal, a higher count of propositions will be less probable. We already know that propositions are truth-apt , that they are capable of expressing something about the world, and it is reasonable to treat them probabilistically.
So that is the right simplicity criterion...except that it had nothing to do with SI!
“The”...?
Why use occams razor at all? If we were only interested in empirical adequacy, the ability to make accurate predictions, simplicity only buys the ability to make predictions with fewer calculations. But SI, according to Yudkowsky (but not Solomonoff) doesn’t just make predictions, it tells you you true facts about the world .
If you are using a simplicity criterion to decide between theories that already known to be predictive , as in Solomonoff induction, then simplicity doesn’t buy you any extra predictiveness, so the extra factor it buys you is presumably truth.
There are multiple simplicity criteria, but not multiple truths. So you need the right simplicity criterion. If you have a conceptually valid simplicity critetion, and you formalise it, then thats as good as it gets, you’ve ticked all the boxes. If you formalise a simplicity criterion that has no known relationship to truth, then you haven’t achieved anything. So it is not enough to say that Solomojnoff is “the” formal standard of simplicity. There are any number of ways of conceptualising simplicity, and you need the right one.
Consider this exchange, from “A semi technical introduction to Solomonoff Induction” .
“ASHLEY: Uh, but you didn’t actually use the notion of computational simplicity to get that conclusion; you just required that the supply of probability mass is finite and the supply of potential complications is infinite. Any way of counting discrete complications would imply that conclusion, even if it went by surface wheels and gears.
BLAINE: Well, maybe. But it so happens that Yudkowsky did invent or reinvent that argument after pondering Solomonoff induction, and if it predates him (or Solomonoff) then Yudkowsky doesn’t know the source. Concrete inspiration for simplified arguments is also a credit to a theory, especially if the simplified argument didn’t exist before that.
ASHLEY: Fair enough.”
I think Ashley deserves an answer to “the objection “[a]ny way of counting discrete complications would imply that conclusion, even if it went by surface wheels and gears”, not a claim about who invented what first!
Or you could write a theory in English, and count the number of letters...that’s formal. But what has it to do with truth and reality? But what, equally, does a count of machine code instructions have to do with truth or probability?
There is one interpretation of Occam’s razor, the epistemic interpretation of it, that has the required properties. If you consider a theory as a conjunction if propositions having a probability less than one, then all else being equal, a higher count of propositions will be less probable. We already know that propositions are truth-apt , that they are capable of expressing something about the world, and it is reasonable to treat them probabilistically.
So that is the right simplicity criterion...except that it had nothing to do with SI!