This is going to be a somewhat technical reply, but here goes anyway.
Boltzmann entropy, on the other hand, is a property of regions of phase space, not of ensembles or distributions. The famous Boltzmann formula equates entropy with the logarithm of the volume of a region in phase space. Now, it’s true that corresponding to every phase space region there is an ensemble/distribution whose Shannon entropy is identical to the Boltzmann entropy, namely the distribution that is uniform in that region and zero elsewhere.
You cannot calculate the Shannon entropy of a continuous distribution so this doesn’t make sense. However I see what you’re getting at here—if we assume that all parts of the phase space have equal probability of being visited, then the ‘size’ of the phase space can be taken as proportional to the ‘number’ of microstates (this is studied under ergodic theory). But to make this argument work for actual physical systems where we want to calculate real quantities from theoretical considerations, the phase space must be ‘discretized’ in some way. A very simple way of doing this is the Sackur-Tetrode formulation which discretizes a continuous space based on the Heisenberg uncertainty principle (‘discretize’ is the best word I can come up with here—what I mean is not listing the microstates but instead giving the volume of the phase space in terms of some definite elementary volume). But there’s a catch here. To be able to use the HUP, you have to formulate the phase space in terms of complementary parameters. For instance, momentum+position, or time+energy.
However, this wasn’t how Boltzmann himself envisioned the partitioning of phase space. In his original “counting argument” he partitioned phase space into regions based on the collective properties of the particles themselves, not the external constraints.
My previous point illustrates why this naive view is not physical—you can’t discretize any kind of system. With some systems—like a box full of particles that can have arbitrary position and momentum—you get infinite (non-physical) values for entropy. It’s easy to see why you can now get a fluctuation in entropy—infinity ‘minus’ some number is still infinity!
I tried re-wording this argument several times but I’m still not satisfied with my attempt at explaining it. Nevertheless, this is how it is. Looking at entropy based on models of collective properties of particles may be interesting theoretically but it may not always be a physically realistic way of calculating the entropy of the system. If you go through something like the Sackur-Tetrode way, though, you see that Boltzmann entropy is the same thing as Shannon entropy.
Boltzmann’s original combinatorial argument already presumed a discretization of phase space, derived from a discretization of single-molecule phase space, so we don’t need to incorporate quantum considerations to “fix” it. The combinatorics relies on dividing single-particle state space into tiny discrete boxes, then looking at the number of different ways in which particles could be distributed among those boxes, and observing that there are more ways for the particles to be spread out evenly among the boxes than for them to be clustered. Without discretization the entire argument collapses, since no more than one particle would be able to occupy any particular “box”, so clustering would be impossible.
So Boltzmann did successfully discretize a box full of particles with arbitrary position and momentum, and using his discretization he derived (discrete approximations of) the Maxwell-Boltzmann distribution and the Boltzmann formula for entropy. And he did all this without invoking (or, indeed, being aware of) quantum considerations. So the Sackur-Tetrode route is not a requirement for a discretized Boltzmann-esque argument. I guess you could argue that in the absence of quantum considerations there is no way to justify the discretization, but I don’t see why not. The discretization need not be interpreted as ontological, emerging from the Uncertainty Principle. It could be interpreted as merely epistemological, a reflection of limits to our abilities of observation and intervention.
Incidentally, none of these derivations require the assumption of ergodicity in the system. The result that the size of a macrostate in phase space is proportional to the number of microstates emerges purely from the combinatorics, with no assumptions about the system’s dynamics (other than that they are Hamiltonian). Ergodicity, or something like it, is only required to establish that the time spent by a system in a particular macrostate is proportional to the size of the macrostate, and that is used to justify probabilistic claims about the system, such as the claim that a closed system observed at an arbitrary time is overwhelmingly likely to be in the macrostate of maximum Boltzmann entropy.
So ultimately, I do think the point Carroll was making is valid. The Boltzmann entropy—as in, the actual original quantity defined by Boltzmann and refined by the Ehrenfests, not the modified interpretation proposed by people like Jaynes—is distinct from the Gibbs entropy. The former can increase (or decrease) in closed system, the latter cannot.
To put it slightly more technically, the Gibbs entropy, being a property of a distribution that evolves according to Hamiltonian laws, is bound to stay constant by Liouville’s theorem, unless there is a geometrical change in the accessible phase space or we apply some coarse-graining procedure. Boltzmann entropy, being a property of macrostates, not of distributions, is not bound by Liouville’s theorem. Even if you interpret the Boltzmann entropy as a property of a distribution, it is not a distribution that evolves in a Hamiltonian manner. It evolves discontinuously when the system moves from one Boltzmann macrostate to the next.
This is going to be a somewhat technical reply, but here goes anyway.
You cannot calculate the Shannon entropy of a continuous distribution so this doesn’t make sense. However I see what you’re getting at here—if we assume that all parts of the phase space have equal probability of being visited, then the ‘size’ of the phase space can be taken as proportional to the ‘number’ of microstates (this is studied under ergodic theory). But to make this argument work for actual physical systems where we want to calculate real quantities from theoretical considerations, the phase space must be ‘discretized’ in some way. A very simple way of doing this is the Sackur-Tetrode formulation which discretizes a continuous space based on the Heisenberg uncertainty principle (‘discretize’ is the best word I can come up with here—what I mean is not listing the microstates but instead giving the volume of the phase space in terms of some definite elementary volume). But there’s a catch here. To be able to use the HUP, you have to formulate the phase space in terms of complementary parameters. For instance, momentum+position, or time+energy.
My previous point illustrates why this naive view is not physical—you can’t discretize any kind of system. With some systems—like a box full of particles that can have arbitrary position and momentum—you get infinite (non-physical) values for entropy. It’s easy to see why you can now get a fluctuation in entropy—infinity ‘minus’ some number is still infinity!
I tried re-wording this argument several times but I’m still not satisfied with my attempt at explaining it. Nevertheless, this is how it is. Looking at entropy based on models of collective properties of particles may be interesting theoretically but it may not always be a physically realistic way of calculating the entropy of the system. If you go through something like the Sackur-Tetrode way, though, you see that Boltzmann entropy is the same thing as Shannon entropy.
Boltzmann’s original combinatorial argument already presumed a discretization of phase space, derived from a discretization of single-molecule phase space, so we don’t need to incorporate quantum considerations to “fix” it. The combinatorics relies on dividing single-particle state space into tiny discrete boxes, then looking at the number of different ways in which particles could be distributed among those boxes, and observing that there are more ways for the particles to be spread out evenly among the boxes than for them to be clustered. Without discretization the entire argument collapses, since no more than one particle would be able to occupy any particular “box”, so clustering would be impossible.
So Boltzmann did successfully discretize a box full of particles with arbitrary position and momentum, and using his discretization he derived (discrete approximations of) the Maxwell-Boltzmann distribution and the Boltzmann formula for entropy. And he did all this without invoking (or, indeed, being aware of) quantum considerations. So the Sackur-Tetrode route is not a requirement for a discretized Boltzmann-esque argument. I guess you could argue that in the absence of quantum considerations there is no way to justify the discretization, but I don’t see why not. The discretization need not be interpreted as ontological, emerging from the Uncertainty Principle. It could be interpreted as merely epistemological, a reflection of limits to our abilities of observation and intervention.
Incidentally, none of these derivations require the assumption of ergodicity in the system. The result that the size of a macrostate in phase space is proportional to the number of microstates emerges purely from the combinatorics, with no assumptions about the system’s dynamics (other than that they are Hamiltonian). Ergodicity, or something like it, is only required to establish that the time spent by a system in a particular macrostate is proportional to the size of the macrostate, and that is used to justify probabilistic claims about the system, such as the claim that a closed system observed at an arbitrary time is overwhelmingly likely to be in the macrostate of maximum Boltzmann entropy.
So ultimately, I do think the point Carroll was making is valid. The Boltzmann entropy—as in, the actual original quantity defined by Boltzmann and refined by the Ehrenfests, not the modified interpretation proposed by people like Jaynes—is distinct from the Gibbs entropy. The former can increase (or decrease) in closed system, the latter cannot.
To put it slightly more technically, the Gibbs entropy, being a property of a distribution that evolves according to Hamiltonian laws, is bound to stay constant by Liouville’s theorem, unless there is a geometrical change in the accessible phase space or we apply some coarse-graining procedure. Boltzmann entropy, being a property of macrostates, not of distributions, is not bound by Liouville’s theorem. Even if you interpret the Boltzmann entropy as a property of a distribution, it is not a distribution that evolves in a Hamiltonian manner. It evolves discontinuously when the system moves from one Boltzmann macrostate to the next.