Cox’s Theorem gives us the basic laws of probability, but there is nothing comparable telling us that algorithmic probability is the correct prior we should be using. A prior is an encoding of one’s prior knowledge / state of information before seeing the experimental data, and we have no a priori reason to expect simple explanations for everything.
I seem to recall Eliezer writing a post on this—and did not seem to disagree with the above passage.
I seem to recall Eliezer writing a post on this—and did not seem to disagree with the above passage.