I have struggled with entropy since I first studied it in high school. I was taught that entropy measures “disorder” or the “chaos” in a system. I was totally disconcerted by the units (Joules/Kelvin), because I couldn’t grasp it intuitively, the same way I could reason, for instance, about speed. I searched for a long time for an answer. Everything became much clearer (I remember this as a revelation) once I read “A farewell to Entropy”, by Arieh Ben-Naim. According to the author, the only reason entropy is measured in J/K (in thermodynamics) is historical: we have different units for temperature and energy because the temperature was defined at a time when we did not understand very well the kinetic theory (Temperature should be in fact, measured in Joules) By using Joules for temperature, the entropy of a thermodynamic system would become in fact unitless, and the later connection with the entropy from information theory would have been much easier to draw (apparently people have discussed much that they are different things). In fact, it seems that the reason why we use the term entropy in Information Theory is that Von Neumann suggested Shannon use the term since a similar concept had been applied in statistical mechanics “but nobody understood it really well”.
I don’t know if the above statements are controversial and some physicists would strongly disagree with what I wrote, but in any case, “A Farewell to Entropy” does a great job at explaining what entropy means with very simple math.
The historical baggage is something that tripped me up, too. In an upcoming post I have a section about classical thermodynamic entropy, including an explanation of the weird units!
Thank you very much for writing this.
I have struggled with entropy since I first studied it in high school. I was taught that entropy measures “disorder” or the “chaos” in a system. I was totally disconcerted by the units (Joules/Kelvin), because I couldn’t grasp it intuitively, the same way I could reason, for instance, about speed. I searched for a long time for an answer. Everything became much clearer (I remember this as a revelation) once I read “A farewell to Entropy”, by Arieh Ben-Naim. According to the author, the only reason entropy is measured in J/K (in thermodynamics) is historical: we have different units for temperature and energy because the temperature was defined at a time when we did not understand very well the kinetic theory (Temperature should be in fact, measured in Joules) By using Joules for temperature, the entropy of a thermodynamic system would become in fact unitless, and the later connection with the entropy from information theory would have been much easier to draw (apparently people have discussed much that they are different things). In fact, it seems that the reason why we use the term entropy in Information Theory is that Von Neumann suggested Shannon use the term since a similar concept had been applied in statistical mechanics “but nobody understood it really well”.
I don’t know if the above statements are controversial and some physicists would strongly disagree with what I wrote, but in any case, “A Farewell to Entropy” does a great job at explaining what entropy means with very simple math.
The historical baggage is something that tripped me up, too. In an upcoming post I have a section about classical thermodynamic entropy, including an explanation of the weird units!
That’s great, I subscribed and looking forward to it!