I think e−H(P) might also have a natural interpretation of something along the lines of “The probability that two consecutive samples from P are equal”. This holds exactly for the uniform distribution, but only holds approximately for the Bernoulli distribution, so this is not a perfect heuristic.
I think e−H(P) might also have a natural interpretation of something along the lines of “The probability that two consecutive samples from P are equal”. This holds exactly for the uniform distribution, but only holds approximately for the Bernoulli distribution, so this is not a perfect heuristic.