Shannon entropy is straightforwardly a special case of von Neumann entropy, so it applies to at least as many kinds of universes.
I still feel a bit confused about the “fundamentalness”, but in trying to formulate a response, I was convinced by Jaynes that von Neumann entropy has an adequate interpretation in terms of Shannon entropy.
Shannon entropy is straightforwardly a special case of von Neumann entropy, so it applies to at least as many kinds of universes.
Only if we’re already representing the universe in terms of quantum basis states. We can always take e.g. a deterministic universe and represent it in terms of pure states with zero entanglement, but that still involves a typecasting operation on our universe-state.
That’s the real issue here: von Neumann entropy comes with an assumption about the type signature of our universe-state, while Shannon entropy doesn’t—it’s just using plain old probability, and we can stick probability distributions on top of whatever we please.
This doesn’t make sense to me. It seems that if you’re being strict about types, then “plain old probabilities” also require the correct type signature, and by using Shannon entropy you are still making an implicit assumption about the type signature.
Things that you can cast as a finite set. You can stretch this a bit by using limits to cover things that can be cast as compact metric spaces (and probably somewhat more than this), but this requires care and grounding in the finite set case in order to be unambiguously meaningful.
Ok, I see what you’re picturing now. That’s the picture we get if we approach probability through the Kolmogorov axioms. We get a different picture if we approach it through Cox’ theorem or logical inductors: these assign probabilities to sentences in a logic. That makes the things-over-which-we-have-a-probability-distribution extremely general—basically, we can assign probabilities to any statements we care to make about the universe, regardless of the type signature of the universe state.
Isn’t this just begging the question, though, by picking up an implicit type signature via the method by which probabilities are assigned? Like, if we lived in a different universe that followed different physics and had different math I’m not convinced it would all work out the same.
If the physics were different, information theory would definitely still be the same—it’s math, not physics. As for “different math”, I’m not even sure what that would mean or if the concept is coherent at all.
Shannon entropy is straightforwardly a special case of von Neumann entropy, so it applies to at least as many kinds of universes.
I still feel a bit confused about the “fundamentalness”, but in trying to formulate a response, I was convinced by Jaynes that von Neumann entropy has an adequate interpretation in terms of Shannon entropy.
Only if we’re already representing the universe in terms of quantum basis states. We can always take e.g. a deterministic universe and represent it in terms of pure states with zero entanglement, but that still involves a typecasting operation on our universe-state.
That’s the real issue here: von Neumann entropy comes with an assumption about the type signature of our universe-state, while Shannon entropy doesn’t—it’s just using plain old probability, and we can stick probability distributions on top of whatever we please.
This doesn’t make sense to me. It seems that if you’re being strict about types, then “plain old probabilities” also require the correct type signature, and by using Shannon entropy you are still making an implicit assumption about the type signature.
What’s the type signature of the things over which we have a probability distribution?
Things that you can cast as a finite set. You can stretch this a bit by using limits to cover things that can be cast as compact metric spaces (and probably somewhat more than this), but this requires care and grounding in the finite set case in order to be unambiguously meaningful.
Ok, I see what you’re picturing now. That’s the picture we get if we approach probability through the Kolmogorov axioms. We get a different picture if we approach it through Cox’ theorem or logical inductors: these assign probabilities to sentences in a logic. That makes the things-over-which-we-have-a-probability-distribution extremely general—basically, we can assign probabilities to any statements we care to make about the universe, regardless of the type signature of the universe state.
Ah, that makes sense, thanks! I’d still say “sentences in a logic” is a specific type though.
Definitely, yes. The benefit is that it avoids directly specifying the type of the world-state.
Isn’t this just begging the question, though, by picking up an implicit type signature via the method by which probabilities are assigned? Like, if we lived in a different universe that followed different physics and had different math I’m not convinced it would all work out the same.
If the physics were different, information theory would definitely still be the same—it’s math, not physics. As for “different math”, I’m not even sure what that would mean or if the concept is coherent at all.