Learning #27 – Statistical Mechanics & Phase Transitions: Partition Functions, Ising Model and Renormalization

Thermodynamics describes macroscopic systems through state functions like temperature, pressure, and entropy, without asking what atoms are doing. Statistical mechanics provides the microscopic foundation: Boltzmann’s insight that entropy is S = k_B ln Ω — the logarithm of the number of accessible microscopic states — connected the abstract framework of thermodynamics to atomic reality. The resulting theory is among the most powerful in all of physics, predicting everything from the heat capacity of solids to the magnetic susceptibility of spin systems near critical points.

1. Partition Functions and the Canonical Ensemble

A system in thermal contact with a heat bath at temperature T is described by the canonical ensemble. Rather than tracking each microstate, we construct the partition function Z, which encodes all thermodynamic information through a single sum over all microstates weighted by their Boltzmann factors.

Canonical partition function:
  Z = ∑_i exp(−βE_i)    β = 1/(k_B T)
  (sum over all microstates i with energy E_i)

Boltzmann probability:
  P_i = exp(−βE_i) / Z

Helmholtz free energy:
  F = −k_B T ln Z

Thermodynamic observables from F:
  ⟨E⟩ = −∂ ln Z / ∂β = F + TS
  S = −(∂F / ∂T)_V = k_B [ln Z + β⟨E⟩]
  C_V = (∂⟨E⟩ / ∂T)_V = k_B β² [⟨E²⟩ − ⟨E⟩²]

Grand canonical ensemble (variable particle number N):
  Ω = ∑_{N,i} exp[−β(E_i − μN)]
  Ω relates to grand potential: J = −k_B T lnΩ
          

The elegance of this approach is that once Z is computed (analytically or numerically), all thermodynamic quantities follow by differentiation. The variance of energy gives the heat capacity; the variance of particle number gives the compressibility. This fluctuation-response connection is a deep principle that we will return to in section 5.

2. The Ising Model

The Ising model is the workhorse of statistical mechanics: N spins σ_i = ±1 arranged on a lattice, with nearest-neighbour ferromagnetic coupling J and external field h. Despite its simplicity, it captures the essential physics of phase transitions in magnets, binary alloys, and even neural networks.

Hamiltonian:
  H = −J ∑_{⟨ij⟩} σ_i σ_j − h ∑_i σ_i

1D Ising (Ising, 1925): exact solution by transfer matrix
  Z = λ&sub1;^N + λ&sub2;^N,   λ_{1,2} = e^(βJ) [cosh(βh) ± √(sinh²(βh) + e^(−4βJ))]
  No phase transition at finite T in 1D (Peierls argument)

2D Ising (Onsager, 1944): exact solution on square lattice
  Critical temperature: k_B T_c = 2J / ln(1 + √2) ≈ 2.269 J
  Spontaneous magnetisation (Onsager/Yang):
    m(T) = [1 − sinh^(−4)(2βJ)]^(1/8)   for T < T_c
  Critical exponents (exact):
    m ~ |T − T_c|^β,   β = 1/8
    χ ~ |T − T_c|^(−γ),   γ = 7/4
    C ~ |ln|T − T_c||   (α = 0, logarithmic divergence)
    ξ ~ |T − T_c|^(−ν),   ν = 1
          

Onsager’s 1944 solution of the 2D Ising model with h = 0 was a landmark in mathematical physics. It showed that phase transitions can be described exactly in statistical mechanics — previously only mean-field theories (which get the exponents wrong) were available. The critical exponents β = 1/8 and γ = 7/4 are universal: any 2D system in the Ising universality class, regardless of the details of its microscopic interactions, exhibits these same exponents near its critical point.

3. Landau Theory of Phase Transitions

Landau (1937) recognised that phase transitions are characterised by the appearance of an order parameter — a quantity that is zero in the disordered phase and non-zero in the ordered phase. Near the critical point, the free energy can be expanded as a power series in the order parameter m:

Landau free energy (m = order parameter, e.g. magnetisation):
  F(m) = F_0 + a(T)m² + (b/4)m&sup4; (no odd terms if Z&sub2; symmetry)
  a(T) = a_0(T − T_c)   (changes sign at T_c)
  b > 0  (second-order / continuous transition)

Equilibrium: ∂F/∂m = 0
  T > T_c: m = 0 (disordered)
  T < T_c: m = ±√[a_0(T_c − T)/b]  ~ (T_c − T)^(1/2)

Mean-field critical exponents:
  β = 1/2 (vs exact 2D: 1/8)
  γ = 1  (vs exact 2D: 7/4)
  δ = 3  (response to field at T_c)
  ν = 1/2  (vs exact 2D: 1)

Ginzburg criterion: mean-field fails when correlations dominate
  d > d_u (upper critical dimension) = 4 for Ising
  d = 3 Ising exponents: β = 0.326, γ = 1.237, ν = 0.630
          

First-Order Transitions

If b < 0 in the Landau expansion (or if a sixth-order term is needed), the transition is first-order: free energy has two degenerate minima at T_c, so the order parameter jumps discontinuously. Latent heat is released and phase coexistence occurs. Water freezing is first-order (ΔH = 334 J/g). Adding a cubic term (m³) breaks the Z₂ symmetry and also drives the transition first-order — relevant to the QCD quark-hadron transition in the early universe.

4. The Renormalization Group

Why do vastly different physical systems share identical critical exponents? Kenneth Wilson answered this question with the renormalization group (RG) in the early 1970s, earning the 1982 Nobel Prize. The RG is a procedure for coarse-graining a system by integrating out short-wavelength degrees of freedom and rescaling.

Block spin transformation (Kadanoff):
  Replace each b×b block of spins by a single effective spin
  Rescale lengths: x → x/b
  Rescale Hamiltonian: H(K) → H(K′)

RG flow in coupling constant space:
  K′ = R_b(K)  (recursion relation)

Fixed points K*:  K* = R_b(K*)
  Stable FP (b>0 phase): disordered
  Stable FP (b<0 phase / T<T_c): ordered
  Unstable FP at T=T_c: critical point

Critical exponents from eigenvalues of linearised RG:
  Eigenvalues: λ_i = b^{y_i}
  y_t > 0: relevant operator (T − T_c)  → ν = 1/y_t
  y_h > 0: relevant operator (magnetic field h) → δ = (d + y_h)/y_h
  y < 0: irrelevant operator (decays under RG)

Universality: systems with same dimension d and same symmetry
flow to the same fixed point → same critical exponents
          

Wilson’s ε-expansion works near the upper critical dimension d_u = 4: setting d = 4 − ε and expanding in powers of ε, the critical exponents can be computed perturbatively. At ε = 1 (d = 3), the expansion gives β ≈ 0.33, in good agreement with numerical values and experiment. The RG framework extends far beyond phase transitions: it underpins the running coupling constants in quantum field theory and the effective field theory approach to nuclear physics.

5. Fluctuation-Dissipation Theorem

The fluctuation-dissipation theorem (FDT) connects spontaneous equilibrium fluctuations to the linear response of a system to an external perturbation. It is one of the deepest results in non-equilibrium statistical mechanics.

Brownian motion (Einstein 1905):
  D = k_B T / (6πηr) = μ k_B T
  μ = mobility;  D = diffusion coefficient

Fluctuation-Dissipation Theorem (Callen & Welton 1951):
  S(ω) = (2k_B T / ω) Im[χ(ω)]
  S(ω) = power spectral density of fluctuations
  χ(ω) = complex susceptibility (linear response)

Nyquist noise (Johnson-Nyquist 1928):
  S_V(ω) = 4k_B T R
  (voltage noise in resistor R at temperature T)

Green-Kubo relations:
  η = (V/k_B T) ∫_0^∞ ⟨σ_{xy}(0)σ_{xy}(t)⟩ dt  (viscosity)
  D = (1/3) ∫_0^∞ ⟨v(0)·v(t)⟩ dt  (diffusion via VACF)
          

The FDT has profound practical consequences: thermal noise in electronic circuits (Johnson-Nyquist noise) is the fundamental limit on signal detection; the viscosity of a liquid can be computed from equilibrium velocity autocorrelation functions; the optical response of a material near a resonance embodies the same relationship between absorption and dispersion (Kramers-Kronig relations).

6. Monte Carlo Methods Near the Critical Point

Analytically solving statistical mechanics models is possible only in special cases. Monte Carlo (MC) methods provide a general numerical approach: by generating random samples from the Boltzmann distribution, one can estimate any thermodynamic observable.

Metropolis-Hastings algorithm:
  1. Propose a move: flip spin i, ΔE = E_new − E_old
  2. Accept with probability:
       a = min(1, exp(−βΔE))
  3. Detailed balance: P(A→B)·π(A) = P(B→A)·π(B)
     ensures convergence to Boltzmann distribution

Autocorrelation time τ:
  τ ~ ξ^z   (critical slowing down near T_c)
  z ≈ 2.17 for local Metropolis (2D Ising)

Wolff cluster algorithm:
  Flips entire correlated clusters in one step
  z ≈ 0 (essentially no critical slowing down)
  Key step: build cluster by adding neighbour j with P = 1 − e^(−2βJ δ_{σi,σj})

Finite-size scaling:
  m(T, L) = L^(−β/ν) f[(T−T_c)L^(1/ν)]
  → data collapse onto universal scaling curve
     allows T_c and ν extraction from finite systems
          

Try These Simulations