1) Why “complexity penalty” should work in fiction, even in a rationalist fiction?
Because there will still be an infinite (countable) number of finite hypotheses which could be considered and only a finite amount of probability to divide among them, which necessarily implies that in the limit more complicated hypotheses will have individual probability approaching zero. This will be true in the limit even if you define ‘complexity’ differently than the person who constructed the distribution.
Is “A or B” more “complex” than “A”? It seems to me that it generally takes more bits to say “A or B”, but the prior for “A” should be smaller than for “A or B”. Is there something in the “assign prior according to complexity” heuristic that accounts for that?
Because there will still be an infinite (countable) number of finite hypotheses which could be considered and only a finite amount of probability to divide among them, which necessarily implies that in the limit more complicated hypotheses will have individual probability approaching zero. This will be true in the limit even if you define ‘complexity’ differently than the person who constructed the distribution.
Is “A or B” more “complex” than “A”? It seems to me that it generally takes more bits to say “A or B”, but the prior for “A” should be smaller than for “A or B”. Is there something in the “assign prior according to complexity” heuristic that accounts for that?