Some remarks and refinements in the definition of entropy

As we have given them, the definitions of the phase space volume $\Omega$$\Omega$ and entropy $S$$S$ are not correct and can give rise to annoying problems and paradoxes.

The dimension of the phase space volume

To begin with we can note that the definition of entropy given in the fundamental postulate of statistical mechanics, makes no sense since $\Omega$$\Omega$ is dimensional. In fact $\Omega$$\Omega$ is defined as an integral over $d\Gamma$$d\Gamma$ and:

$$d\Gamma =\prod _{i=1}^{N}d{\vec {q}}_{i}d{\vec {p}}_{i}\quad \Rightarrow \quad \left[d\Gamma \right]=\left({\text{m}}\cdot {\frac {\text{kg m}}{\text{s}}}\right)^{3N}=\left[\Omega \right]$$

$$d\Gamma =\prod _{i=1}^{N}d{\vec {q}}_{i}d{\vec {p}}_{i}\quad \Rightarrow \quad \left[d\Gamma \right]=\left({\text{m}}\cdot {\frac {\text{kg m}}{\text{s}}}\right)^{3N}=\left[\Omega \right]$$

Therefore, if we want to take the logarithm of $\Omega$$\Omega$ we must first adimensionalize it; to do so we must introduce a constant $h$$h$ with dimensions ${\text{m}}\cdot ({\text{kg m}}/{\text{s}})={\text{J}}\cdot {\text{s}}$${\text{m}}\cdot ({\text{kg m}}/{\text{s}})={\text{J}}\cdot {\text{s}}$, so that we can actually redefine the phase space volume as:

$$\Omega (E,V,N)=\int {\frac {d\Gamma }{h^{3N}}}\delta ({\mathcal {H}}-E)$$

$$\Omega (E,V,N)=\int {\frac {d\Gamma }{h^{3N}}}\delta ({\mathcal {H}}-E)$$

which is now really adimensional and now the fundamental postulate makes sense.

A legitimate question could now be: what is the value of $h$$h$? Which constant is it? Unfortunately, within classical statistical mechanics it is impossible to establish it; only in quantum statistical mechanics the constant $h$$h$ acquires a precise meaning, and in particular it turns out that $h$$h$ is Planck's constant:

$$h=6.62\cdot 10^{-34}{\text{ J s}}$$

$$h=6.62\cdot 10^{-34}{\text{ J s}}$$

Let us finally note that dividing $\Omega$$\Omega$ by $h^{3N}$$h^{3N}$ and integrating over the phase space can also be interpreted as dividing the whole phase space in cells of volume $h^{3}$$h^{3}$, thus "discretizing" the possible states of the system considering as equivalent all the points inside a cell, and summing over these cells: in this sense the entropy gives a measure of the quantity of such possible states.

Extensivity of the entropy and the Gibbs paradox

Even with the introduction of $h$$h$ we still have some problems. In particular, from the expression of $\Omega (E,V,N)$$\Omega (E,V,N)$ for an ideal gas we have:

$${\begin{aligned}\ln \Omega (E,V,N)=\ln \left({\frac {3N}{2E}}\right)-3N\ln h+N\ln V+{\frac {3}{2}}\ln \pi +\\+{\frac {3}{2}}N\ln(2mE)-{\frac {3}{2}}\ln \left({\frac {3}{2}}N\right)+{\frac {3}{2}}N\end{aligned}}$$

$${\begin{aligned}\ln \Omega (E,V,N)=\ln \left({\frac {3N}{2E}}\right)-3N\ln h+N\ln V+{\frac {3}{2}}\ln \pi +\\+{\frac {3}{2}}N\ln(2mE)-{\frac {3}{2}}\ln \left({\frac {3}{2}}N\right)+{\frac {3}{2}}N\end{aligned}}$$

Since $E$$E$ and $N$$N$ are both extensive, $E/N$$E/N$ is intensive and thus remains constant in the thermodynamic limit; this means that for large $N$$N$ we can neglect the first term so that in the end:

$$\ln \Omega (E,V,N)=N\ln \left[V\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]+{\frac {3}{2}}N$$

$$\ln \Omega (E,V,N)=N\ln \left[V\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]+{\frac {3}{2}}N$$

and therefore the entropy of the system will be:

$$S(E,V,N)=k_{B}N\ln \left[V\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]+{\frac {3}{2}}k_{B}N$$

$$S(E,V,N)=k_{B}N\ln \left[V\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]+{\frac {3}{2}}k_{B}N$$

This expression is rather problematic. In fact, first of all we can note that written in this way it is not extensive since it contains terms like $N\ln V$$N\ln V$ and $N\ln N$$N\ln N$. Furthermore, it gives rise to a catastrophic paradox known as the Gibbs paradox. In order to understand it let us consider a system divided by a wall into two subsystems each of $N_{i}$$N_i$ particles of mass $m_{i}$$m_{i}$, volume $V_{i}$$V_i$ and energy $E_{i}$$E_{i}$ (with $i=1,2$$i=1,2$). The entropy of the system in this initial state is:

$${\begin{aligned}S_{i}={\frac {3}{2}}k_{B}(N_{1}+N_{2})+\\+k_{B}\left\lbrace N_{1}\ln \left[V_{1}\left({\frac {4\pi m_{1}}{h^{2}}}{\frac {E_{1}}{N_{1}}}\right)^{3/2}\right]+N_{2}\ln \left[V_{2}\left({\frac {4\pi m_{2}}{h^{2}}}{\frac {E_{2}}{N_{2}}}\right)^{3/2}\right]\right\rbrace \end{aligned}}$$

$${\begin{aligned}S_{i}={\frac {3}{2}}k_{B}(N_{1}+N_{2})+\\+k_{B}\left\lbrace N_{1}\ln \left[V_{1}\left({\frac {4\pi m_{1}}{h^{2}}}{\frac {E_{1}}{N_{1}}}\right)^{3/2}\right]+N_{2}\ln \left[V_{2}\left({\frac {4\pi m_{2}}{h^{2}}}{\frac {E_{2}}{N_{2}}}\right)^{3/2}\right]\right\rbrace \end{aligned}}$$

If we now remove the wall the two gases will mix and the entropy of this final state of the system will be:

$${\begin{aligned}S_{f}={\frac {3}{2}}k_{B}(N_{1}+N_{2})+\\+k_{B}\ln \left[(V_{1}+V_{2})^{N_{1}+N_{2}}\left({\frac {4\pi }{3h^{2}}}\right)^{{\frac {3}{2}}(N_{1}+N_{2})}\left({\frac {m_{1}E_{1}}{N_{1}}}\right)^{{\frac {3}{2}}N_{1}}\left({\frac {m_{2}E_{2}}{N_{2}}}\right)^{{\frac {3}{2}}N_{2}}\right]\end{aligned}}$$

$${\begin{aligned}S_{f}={\frac {3}{2}}k_{B}(N_{1}+N_{2})+\\+k_{B}\ln \left[(V_{1}+V_{2})^{N_{1}+N_{2}}\left({\frac {4\pi }{3h^{2}}}\right)^{{\frac {3}{2}}(N_{1}+N_{2})}\left({\frac {m_{1}E_{1}}{N_{1}}}\right)^{{\frac {3}{2}}N_{1}}\left({\frac {m_{2}E_{2}}{N_{2}}}\right)^{{\frac {3}{2}}N_{2}}\right]\end{aligned}}$$

Therefore, the entropy of the system will have changed during this process by the amount:

$${\begin{aligned}\Delta S=S_{f}-S_{i}=k_{B}\ln \left[{\frac {(V_{1}+V_{2})^{N_{1}+N_{2}}}{V_{1}^{N_{1}}V_{2}^{N_{2}}}}\right]=\\=k_{B}\left[N_{1}\ln \left({\frac {V_{1}+V_{2}}{V_{1}}}\right)+N_{2}\ln \left({\frac {V_{1}+V_{2}}{V_{2}}}\right)\right]\end{aligned}}$$

$${\begin{aligned}\Delta S=S_{f}-S_{i}=k_{B}\ln \left[{\frac {(V_{1}+V_{2})^{N_{1}+N_{2}}}{V_{1}^{N_{1}}V_{2}^{N_{2}}}}\right]=\\=k_{B}\left[N_{1}\ln \left({\frac {V_{1}+V_{2}}{V_{1}}}\right)+N_{2}\ln \left({\frac {V_{1}+V_{2}}{V_{2}}}\right)\right]\end{aligned}}$$

called entropy of mixing, which is always positive. If $m_{1}\neq m_{2}$$m_{1}\neq m_{2}$ this is a correct result, since the mixing of the two gases is an irreversible process. However, since $\Delta S$$\Delta S$ doesn't depend on $m_{1}$$m_{1}$ and $m_{2}$$m_{2}$ this result holds also in the case $m_{1}=m_{2}$$m_{1}=m_{2}$. But this is a paradox: the mixing of two identical gases is a reversible process (we can recover the initial state reinserting the wall) so the entropy of the whole system shouldn't increase if we remove the wall. Furthermore, the fact that $\Delta S>0$$\Delta S>0$ also when $m_{1}=m_{2}$$m_{1}=m_{2}$ is catastrophic because it means that the entropy of a system depends on the history of the system itself, rather than on its state. But this ultimately means that entropy doesn't exist at all: consider a system of energy $E$$E$ made of $N$$N$ particles contained in a volume $V$$V$; then we can think that this system has been obtained from the separation of $M$$M$ pre-existing subsystems, with $M$$M$ arbitrarily large. This means that the entropy of the system has increased an arbitrarily large amount of times from its initial value, and therefore the entropy of the system in its final configuration (energy $E$$E$, volume $V$$V$, $N$$N$ particles) is greater than any arbitrary number: in other words, assuming ${\textstyle S(E,V,N)=k_{B}N\ln \left[V\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]+{\frac {3}{2}}k_{B}N}$${\textstyle S(E,V,N)=k_{B}N\ln \left[V\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]+{\frac {3}{2}}k_{B}N}$ as the entropy of an ideal gas we would conclude that the entropy of any ideal gas is infinite! There's clearly something wrong with the definition of entropy we have given. How can we solve this problem? We know that in classical mechanics identical particles are distinguishable; however in order to solve the Gibbs paradox we must treat them as undistinguishable, just like we would do in quantum mechanics: this way if we exchange two particles the representative point of the system in phase space won't change. Now, since $N$$N$ particles can be exchanged in $N!$$N!$ different ways there will be $N!$$N!$ different configurations of the system relative to the same representative point. This means that the we must redefine the phase space volume of the system reducing it by a factor $N!$$N!$ (which is sometimes called Boltzmann factor):

$$\Omega (E,V,N)={\frac {1}{N!}}\int {\frac {d\Gamma }{h^{3N}}}\delta ({\mathcal {H}}-E)$$

$$\Omega (E,V,N)={\frac {1}{N!}}\int {\frac {d\Gamma }{h^{3N}}}\delta ({\mathcal {H}}-E)$$

In this way we can solve all the problems we have described. In fact, using this definition of the phase space volume in the case of the ideal gas we have:

$$\Omega (E,V,N)={\frac {V^{N}}{N!h^{3N}}}{\frac {(2\pi mE)^{3N/2}}{\Gamma \left({\frac {3}{2}}N+1\right)}}$$

$$\Omega (E,V,N)={\frac {V^{N}}{N!h^{3N}}}{\frac {(2\pi mE)^{3N/2}}{\Gamma \left({\frac {3}{2}}N+1\right)}}$$

$$S(E,V,N)={\frac {5}{2}}Nk_{B}+Nk_{B}\ln \left[{\frac {V}{N}}\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]$$

$$S(E,V,N)={\frac {5}{2}}Nk_{B}+Nk_{B}\ln \left[{\frac {V}{N}}\left({\frac {4\pi m}{3h^{2}}}{\frac {E}{N}}\right)^{3/2}\right]$$

This expression for the entropy is clearly extensive since it is proportional to $N$$N$ and the logarithm depends only on $V/N$$V/N$ and $E/N$$E/N$. Furthermore, the computation of the entropy of mixing of two gases now gives:

$$\Delta S=k_{B}\left[(N_{1}+N_{2})\ln \left({\frac {V_{1}+V_{2}}{N_{1}+N_{2}}}\right)-N_{1}\ln {\frac {V_{1}}{N_{1}}}-N_{2}\ln {\frac {V_{2}}{N_{2}}}\right]$$

$$\Delta S=k_{B}\left[(N_{1}+N_{2})\ln \left({\frac {V_{1}+V_{2}}{N_{1}+N_{2}}}\right)-N_{1}\ln {\frac {V_{1}}{N_{1}}}-N_{2}\ln {\frac {V_{2}}{N_{2}}}\right]$$

If the two gases are different, their densities $N_{i}/V_{i}$$N_{i}/V_{i}$ will be different and so $\Delta S>0$$\Delta S>0$ as it must be. However, if the two gases are identical this time we have $V_{1}/N_{1}=V_{2}/N_{2}=V/N$$V_{1}/N_{1}=V_{2}/N_{2}=V/N$ and the entropy of mixing vanishes.

This solution to the Gibbs paradox, however, is rather an ad hoc one. Unfortunately there's no way to understand where does the Boltzmann factor really come from within the framework of classical statistical mechanics. This can be made clearer within quantum statistical mechanics: in that case the $N!$$N!$ comes from the fact that identical particles are intrinsically indistinguishable and does not "disappear" in the classical limit.

Conclusions

To conclude, the correct definition of the phase space volume that eliminates all the problems that we have mentioned, and the one we will always use in the future, is:

$$\Omega (E,V,N)={\frac {1}{N!}}\int {\frac {d\Gamma }{h^{3N}}}\delta ({\mathcal {H}}-E)$$

$$\Omega (E,V,N)={\frac {1}{N!}}\int {\frac {d\Gamma }{h^{3N}}}\delta ({\mathcal {H}}-E)$$

For the sake of simplicity we now redefine $d\Gamma$$d\Gamma$ as $d\Gamma _{\text{new}}=d\Gamma _{\text{old}}/N!h^{3N}$$d\Gamma _{\text{new}}=d\Gamma _{\text{old}}/N!h^{3N}$ (in other words we "incorporate" the $1/N!h^{3N}$$1/N!h^{3N}$ factor inside $d\Gamma$$d\Gamma$). This way, for the ideal gas we have:

$$\Omega (E,V,N)={\frac {V^{N}}{N!h^{3N}}}{\frac {(2\pi mE)^{3N/2}}{\Gamma \left({\frac {3}{2}}N+1\right)}}$$

$$\Omega (E,V,N)={\frac {V^{N}}{N!h^{3N}}}{\frac {(2\pi mE)^{3N/2}}{\Gamma \left({\frac {3}{2}}N+1\right)}}$$