Statistical Mechanics — Boltzmann, Entropy & Phase Transitions

1. The Microcanonical Ensemble

Statistical mechanics rests on a single foundational postulate: for an isolated system in equilibrium with fixed energy E, volume V, and particle number N, all accessible microstates are equally probable. This is the postulate of equal a priori probabilities, and it is the starting point for every ensemble in statistical mechanics.

A microstate is the most complete description of the system: for a classical gas of N particles, it is the set of all 3N positions and 3N momenta {q₁,…,q_N; p₁,…,p_N} — a point in 6N-dimensional phase space. A macrostate is a coarse-grained description specified by a small number of thermodynamic variables (E, V, N). The number of microstates compatible with a given macrostate is Ω(E, V, N).

The microcanonical ensemble describes an isolated system: the probability of each microstate with energy in [E, E + δE] is:

P_i = 1/Ω(E, V, N) for all accessible microstates i Ω(E, V, N) = number of microstates with energy in [E, E + δE]

From this, temperature is defined as:

1/T = ∂S/∂E |_{V,N} where S = k_B ln Ω This statistical definition of temperature agrees exactly with the thermodynamic definition at equilibrium.

2. Boltzmann Entropy: S = k_B · ln(Ω)

The cornerstone of statistical mechanics is Boltzmann's formula:

S = k_B \cdot ln Ω k_B = 1.380649 \times 10⁻²³ J/K (Boltzmann constant) Ω = number of accessible microstates (dimensionless integer)

The logarithm is chosen for a crucial reason: when two independent systems A and B are combined, their microstates multiply (Ω_{A+B} = Ω_A × Ω_B), but entropy should add (S_{A+B} = S_A + S_B). The logarithm converts multiplication to addition: ln(Ω_A × Ω_B) = ln Ω_A + ln Ω_B.

The Second Law — entropy never decreases in an isolated system — follows directly from the fact that systems evolve toward macrostates with more microstates (simply because randomly evolving systems are overwhelmingly likely to land in high-multiplicity regions of phase space). For a 1-mole gas expanding from V to 2V:

ΔS = N_A k_B ln(2V/V) = R ln 2 \approx 5.76 J/(mol\cdotK) The number of microstates increases by a factor of 2^N_A \approx 10^(1.8\times10²³) The probability of spontaneous recompression: (1/2)^N_A \approx 10^(-5.4\times10²²)

This astronomical improbability is why we never observe gases spontaneously concentrating in one half of a container — even though it is not forbidden by the laws of mechanics.

Gibbs entropy: The more general formulation due to Gibbs applies to arbitrary probability distributions over microstates: S = −k_B Σ_i p_i ln p_i. For the microcanonical ensemble (all p_i = 1/Ω), this reduces to S = k_B ln Ω. For quantum systems, it becomes the von Neumann entropy S = −k_B Tr(ρ ln ρ), where ρ is the density matrix.

3. The Canonical Ensemble and Partition Function

A system in contact with a heat reservoir at temperature T exchanges energy freely but maintains fixed N and V. This is the canonical ensemble. The probability of the system occupying microstate i with energy E_i is:

P_i = exp(-βE_i) / Z where β = 1/(k_B T) Z = Σ_i exp(-βE_i) (canonical partition function)

The Boltzmann factor exp(−βE_i) weights lower-energy states more heavily at low temperature (as T → 0, only the ground state is occupied) and distributes weight more uniformly at high temperature (as T → ∞, all states are equally probable).

The partition function Z is the central object of the canonical ensemble — a generating function from which all thermodynamic quantities follow by taking derivatives. It encodes the entire thermodynamic information of the system.

Free Energy

Helmholtz free energy: F = −k_B T ln Z = U − TS Minimising F (not U) determines equilibrium at fixed T, V, N. Gibbs free energy: G = F + PV = U − TS + PV (fixed T, P) Chemical potential: μ = ∂F/∂N |_{T,V}

The importance of the partition function cannot be overstated: once Z is computed as a function of T, V, and N, the entire thermodynamics of the system follows by differentiation. For an ideal gas of N identical classical particles:

Z_N = (1/N!) \cdot [V \cdot (2πmk_BT/h²)^(3/2)]^N F = -Nk_BT [ln(V/N) + (3/2)ln(2πmk_BT/h²) + 1] P = -\partialF/\partialV = Nk_BT/V \to Ideal gas law PV = Nk_BT ✓ U = -\partiallnZ/\partialβ = (3/2)Nk_BT \to Equipartition theorem ✓

4. Thermodynamic Averages from the Partition Function

The partition function generates thermodynamic averages by differentiation with respect to β = 1/(k_BT):

Average energy: <E> = -\partial ln Z/\partialβ = Σ_i E_i \cdot exp(-βE_i) / Z Energy fluctuations: <(ΔE)²> = <E²> - <E>² = \partial²ln Z/\partialβ² = k_B T² \cdot C_V where C_V = \partial<E>/\partialT is the heat capacity at constant volume. Entropy: S = k_B (ln Z - β \partialln Z/\partialβ) = k_B ln Z + <E>/T

The relative energy fluctuation scales as:

ΔE_rms / <E> = \sqrt(k_B T² C_V) / <E> \propto 1/\sqrtN For N ~ 10²³ particles: ΔE/<E> ~ 10⁻¹¹\cdot⁵ \approx 10⁻¹² \to Thermodynamic variables are extraordinarily well-defined for macroscopic systems.

This shows why thermodynamics is exact for macroscopic systems despite the microscopic randomness: fluctuations are suppressed by √N and are unobservably small for any laboratory-scale amount of matter.

🔥

Carnot Engine Simulator

Step through isothermal and adiabatic processes and measure entropy changes

5. The Ising Model

The Ising model (Lenz 1920, Ising 1925) is the canonical model of statistical mechanics — simple enough to solve exactly in 1D and 2D, yet rich enough to capture the essential physics of phase transitions, ferromagnetism, and order-disorder phenomena.

The model consists of N spins s_i = ±1 arranged on a lattice. Each spin interacts with its neighbours and with an external field h:

H = −J Σ_{<i,j>} s_i s_j − h Σ_i s_i J > 0: ferromagnetic coupling (aligned spins have lower energy) J < 0: antiferromagnetic coupling <i,j>: sum over nearest-neighbour pairs Partition function: Z = Σ_{all spin configs} exp(−βH) Magnetisation: m = <s_i> = (1/N) Σ_i <s_i>

Key results by dimension:

1D Ising (exact, Ising 1925): no phase transition at any finite T. Correlations decay exponentially with distance. The 1D transfer matrix gives Z = 2^N cosh^N(βJ) at h = 0.
2D Ising (exact, Onsager 1944): phase transition at k_BT_c/J = 2/ln(1+√2) ≈ 2.269. Near T_c, the specific heat diverges logarithmically. This was the first exact solution of a non-trivial phase transition.
3D Ising: no exact solution known. Numerical simulations give T_c ≈ 4.51 J/k_B with critical exponents determined to high precision by Monte Carlo and conformal bootstrap methods.

🧲

Ising Model Simulator

Watch magnetic domains form, dissolve, and fluctuate near the critical temperature

6. Phase Transitions and Critical Phenomena

A phase transition is a qualitative change in the thermodynamic state of a system as a control parameter (temperature, pressure, external field) crosses a critical value. Transitions are classified by the continuity of the free energy:

First-order transitions (discontinuous): the first derivative of free energy is discontinuous — volume, magnetisation, or entropy jumps discontinuously. Latent heat is absorbed or released. Examples: melting ice, boiling water, liquid-gas transition below the critical point.
Second-order (continuous) transitions: the first derivative of free energy is continuous, but the second derivative (heat capacity, susceptibility) diverges. The order parameter (e.g., magnetisation) approaches zero continuously. Examples: ferromagnetic transition at T_c, liquid-gas transition at the critical point, superconducting transition.

Near a second-order transition, physical quantities obey power laws in the reduced temperature t = (T − T_c)/T_c:

Order parameter: m ~ |t|^β (β ≈ 0.326 for 3D Ising) Susceptibility: χ ~ |t|^{−γ} (γ ≈ 1.237 for 3D Ising) Correlation length: ξ ~ |t|^{−ν} (ν ≈ 0.630 for 3D Ising) Specific heat: C ~ |t|^{−α} (α ≈ 0.110 for 3D Ising) Correlation fn: G(r) ~ r^{−(d−2+η)} at T = T_c Scaling relations (not all independent): α + 2β + γ = 2 (Rushbrooke) γ = ν(2−η) (Fisher) dν = 2 − α (Josephson hyperscaling)

At T_c, the correlation length ξ diverges — fluctuations occur on all length scales simultaneously. This is critical opalescence: a liquid near its critical point becomes milky white because density fluctuations of all sizes scatter light. The system becomes self-similar (fractal) at the critical point.

7. Spontaneous Symmetry Breaking

The Ising Hamiltonian at h = 0 is symmetric under flipping all spins (s_i → −s_i). Yet below T_c, the system develops a non-zero magnetisation m ≠ 0, choosing either the m > 0 or m < 0 phase. The symmetry of the Hamiltonian is "broken" by the equilibrium state — this is spontaneous symmetry breaking (SSB).

SSB requires the thermodynamic limit N → ∞. For finite N, the true ground state is a superposition of the two magnetised states, and thermal fluctuations will eventually flip the system between them. But for N ~ 10²³, the tunnelling time between degenerate ground states is astronomically long — the symmetry is effectively broken.

Goldstone's Theorem and Nambu-Goldstone Bosons

When a continuous symmetry (not a discrete ±1 symmetry) is spontaneously broken, Goldstone's theorem guarantees the existence of massless excitations — Nambu-Goldstone bosons. Examples abound:

Phonons in crystals: translational symmetry is broken by crystallisation; acoustic phonons are the Goldstone modes.
Magnons in ferromagnets: continuous rotational symmetry of spins is broken; spin waves are the Goldstone modes.
Pions in QCD: approximate chiral symmetry of quark masses is broken by the quark condensate; pions are the (pseudo-)Goldstone bosons, with small mass due to explicit breaking by quark masses.
The Higgs mechanism: in gauge theories (electroweak), the Goldstone bosons are "eaten" by gauge bosons (W and Z), giving them mass. The remaining physical scalar is the Higgs boson, discovered in 2012.

8. Universality Classes

One of the most profound discoveries in 20th-century physics is that systems as different as a magnet, a liquid near its boiling point, a polymer chain, and a superconductor can share identical critical exponents. This is universality: critical behaviour depends only on the spatial dimension d and the symmetry of the order parameter, not on microscopic details.

Major universality classes (3D systems): Ising (Z₂ symmetry, scalar order parameter): β \approx 0.3264, γ \approx 1.2372, ν \approx 0.6300 Examples: uniaxial magnets, liquid-gas transition, binary alloys XY (U(1) symmetry, 2D vector order parameter): β \approx 0.3470, γ \approx 1.3178, ν \approx 0.6717 Examples: superfluid He-4, easy-plane ferromagnets Heisenberg (O(3) symmetry, 3D vector order parameter): β \approx 0.3689, γ \approx 1.3960, ν \approx 0.7112 Examples: isotropic ferromagnets (iron, nickel above 150 K)

Universality is explained by the renormalisation group (RG), developed by Wilson (Nobel Prize 1982). The key idea: as we zoom out to longer length scales, the effective description of the system flows toward a fixed point in the space of all Hamiltonians. Different microscopic systems in the same universality class flow to the same fixed point — explaining why they share critical exponents.

The RG also explains why some microscopic details are irrelevant (they die away along the flow) while others are relevant (they determine which universality class). The dimension d and symmetry group determine the relevant perturbations — hence the universality class.

Mean-field theory and the upper critical dimension: Mean-field theory (Landau theory) ignores fluctuations and predicts universal exponents β = 1/2, γ = 1, ν = 1/2 for all systems. It becomes exact above the upper critical dimension d_c (d_c = 4 for the Ising class), where fluctuations are irrelevant. Below d_c, fluctuations are important and must be treated by the full RG — giving non-trivial, dimension-dependent exponents.

⚙️

Thermodynamics Simulator

Explore energy, entropy, and heat engine efficiency interactively