Statistical Mechanics and Thermodynamics

Introduction to Statistical Mechanics

Statistical mechanics provides the microscopic foundation of thermodynamics—explaining macroscopic properties (temperature, pressure, entropy, free energy) from the statistical behaviour of enormous numbers of particles (~10^23 atoms in a macroscopic system). Classical thermodynamics (laws of Gibbs, Clausius, Kelvin) describes energy conservation, entropy increase, and equilibrium conditions phenomenologically without reference to atomic structure. Statistical mechanics (Boltzmann, Gibbs, Maxwell, 1860s-1900s) derives these laws from probability theory applied to microscopic degrees of freedom, explaining why entropy increases (overwhelming statistical weight of disordered macrostates) and connecting macrostate properties to partition function sums over all microstates.

Statistical mechanics spans classical (Boltzmann statistics for distinguishable or dilute particles), quantum (Fermi-Dirac for fermions, Bose-Einstein for bosons—crucial below quantum degeneracy temperature), and non-equilibrium (Boltzmann transport equation, Fluctuation theorems, NESS—non-equilibrium steady states). Applications range from understanding stellar structure (degenerate electron pressure in white dwarfs) and semiconductor carrier densities (Fermi-Dirac distribution governing electron occupancy) to protein folding free energy landscapes, polymer physics, and information theory (Shannon entropy as statistical mechanical entropy—Jaynes maximum entropy principle).

Foundations: Boltzmann and Ensembles

Boltzmann Distribution and Partition Function

For a system in thermal equilibrium with a heat reservoir at temperature T, the probability of microstate i with energy E_i is the Boltzmann distribution: P_i = exp(-E_i/k_B*T) / Z, where Z = sum_i exp(-E_i/k_B*T) is the canonical partition function—the central object of statistical mechanics from which all thermodynamic quantities derive: F = -k_B*T*ln(Z) (Helmholtz free energy); S = -dF/dT|_V (entropy); U = -d(ln Z)/d(beta)|_V (internal energy, beta=1/k_B*T); P = -dF/dV|_T (pressure). Boltzmann's entropy S = k_B*ln(Omega) (Omega = number of accessible microstates)—the tombstone equation on Boltzmann's grave in Vienna—unifies the two-way arrow between microscopic counting and macroscopic entropy. Equipartition theorem: each quadratic degree of freedom (translational kinetic, rotational, harmonic oscillator potential) contributes k_B*T/2 to average energy—explaining heat capacities of ideal monatomic (3/2 Nk_B), diatomic (5/2 Nk_B at room temperature), and solid (3Nk_B, Dulong-Petit law) systems.

Quantum Statistical Mechanics

Quantum statistics arises from particle indistinguishability and the symmetry of many-body wavefunctions. Bosons (integer spin: photons, phonons, He-4, Cooper pairs): wavefunction symmetric under particle exchange—any number can occupy same state—Bose-Einstein distribution n(E) = 1/(exp((E-mu)/k_B*T) - 1)—diverges as E→mu, driving Bose-Einstein condensation below T_BEC. Fermions (half-integer spin: electrons, protons, He-3, neutrons): antisymmetric wavefunctions—Pauli exclusion; Fermi-Dirac distribution n(E) = 1/(exp((E-mu)/k_B*T) + 1)—at T=0 fills all states below Fermi energy E_F like water filling a bucket. Fermi energy for metals: E_F = (hbar^2/2m)*(3pi^2*n)^(2/3) (~3-11 eV for typical metals—room temperature thermal energy k_B*T = 0.025 eV << E_F, so electrons are degenerate). Electronic heat capacity of metals: C_e = pi^2/2 * (k_B*T/E_F) * N*k_B—linear in T at low temperature, much less than classical equipartition 3/2 Nk_B because only electrons near E_F can be thermally excited.

Phase Transitions and Critical Phenomena

Second-Order Phase Transitions and Universality

Phase transitions are qualitative changes in the thermodynamic state of matter as control parameters (T, P, field) cross critical values. First-order transitions (melting, boiling, solid-solid transitions) have discontinuous first derivatives of free energy (entropy, volume)—latent heat, metastability, nucleation. Second-order (continuous) transitions (ferromagnetic Curie point, superfluid lambda transition, superconducting T_c, liquid-gas critical point) have continuous free energy but divergent second derivatives (susceptibility chi, correlation length xi, heat capacity C). Landau theory: free energy expanded in order parameter phi (magnitude of symmetry breaking): F = a*phi^2 + b*phi^4 + grad phi^2 terms; minimisation gives phi=0 (disordered) above T_c, phi = sqrt(-a/2b) below T_c. Critical exponents (beta, gamma, nu, delta, alpha, eta) describe divergences near T_c—remarkably, systems with different microscopic physics share identical exponents (universality classes) determined only by dimensionality and symmetry: Ising universality class (n=1 order parameter, 3D: beta=0.326, nu=0.630); XY (n=2); Heisenberg (n=3). Renormalisation group (Wilson, Fischer, Nobel 1982): explains universality by showing RG flow in parameter space approaches universal fixed points regardless of microscopic details.

Non-Equilibrium Statistical Mechanics

Most physical systems of interest are not in thermal equilibrium—biological cells, turbulent fluids, driven condensed matter, glasses, and active matter are all non-equilibrium. Boltzmann transport equation: df/dt + v·grad_r f + F·grad_p f = (df/dt)_coll describes evolution of single-particle distribution function f(r,p,t) with collision integral; linearised gives Fermi liquid theory for metals, Drude model for electrical/thermal conductivity. Fluctuation-dissipation theorem: equilibrium fluctuations of a system are related to its linear response (dissipation) to external perturbations—connecting Brownian motion diffusion coefficient to viscosity (Einstein relation D = k_B*T / (6*pi*eta*R)) and Johnson-Nyquist thermal noise in resistors to resistance. Jarzynski equality (1997): exp(-W/k_B*T) = exp(-delta F/k_B*T) connects non-equilibrium work averages to equilibrium free energy differences—enabling free energy measurement from irreversible pulling experiments (complementing Crooks fluctuation theorem). Active matter (self-propelled particles: bacteria, motility assays, active colloids, bird flocks) breaks detailed balance generating non-equilibrium steady states with emergent flocking, pattern formation, and giant number fluctuations—a growing statistical mechanics frontier.

Examples and Applications

Example 1: Ising Model and Ferromagnetism

The Ising model—binary spins s_i = ±1 on a lattice with nearest-neighbour coupling H = -J*sum(s_i*s_j) - h*sum(s_i)—is the paradigmatic exactly solvable model of statistical mechanics. 1D Ising: exact solution (transfer matrix method, Kramers-Wannier 1941)—no phase transition at finite T (fluctuations destroy long-range order in 1D). 2D Ising: exact solution by Onsager (1944)—T_c = 2J/(k_B*ln(1+sqrt(2))), spontaneous magnetisation M = (1-sinh^-4(2J/k_B*T))^(1/8)—first exact calculation of a phase transition, critical exponents (beta=1/8, nu=1, alpha=0 (log divergence)) anchored the theory of critical phenomena. 3D Ising: no exact solution—best results from Monte Carlo (Wilson renormalisation group confirms universal exponents) and conformal bootstrap (precision determination of critical exponents). Monte Carlo simulations using Metropolis algorithm, Wolff cluster algorithm (avoiding critical slowing down), and parallel tempering reveal the phase diagram, domain coarsening dynamics, and finite-size scaling behaviour studied in undergrad and graduate courses worldwide as the entry point to phase transitions.

Example 2: Ideal Gas and Real Gas

The ideal gas—point particles with no interactions—has equation of state PV=nRT (directly derivable from Maxwell-Boltzmann kinetic theory) and internal energy U depending only on temperature: U = (f/2)nRT (f = degrees of freedom). Maxwell-Boltzmann speed distribution P(v) = 4pi*(m/2pi*k_B*T)^(3/2) * v^2 * exp(-mv^2/2k_B*T)—gives mean speed 8k_B*T/(pi*m))^1/2, rms speed (3k_B*T/m)^1/2, most probable speed (2k_B*T/m)^1/2—verified by rotating disc velocity selectors in molecular beam experiments. Real gases: van der Waals equation (P + an^2/V^2)(V-nb) = nRT (a = intermolecular attraction parameter; b = excluded volume)—predicts liquid-gas phase transition and critical point T_c = 8a/(27Rb), P_c = a/(27b^2), V_c = 3nb—qualitatively correct but quantitatively approximate. Virial expansion: PV/nRT = 1 + B_2(T)n/V + B_3(T)(n/V)^2 +... (virial coefficients relate to intermolecular potential integrals)—used in precision equation-of-state measurements of gases for thermometry and flow metering calibration.

Example 3: Entropy and Information Theory

Shannon entropy of a probability distribution: H = -sum_i p_i log2(p_i) bits—identical functional form to Boltzmann-Gibbs entropy S = -k_B sum_i p_i ln(p_i). Jaynes maximum entropy principle (1957): the least-biased probability distribution consistent with known constraints is the maximum entropy distribution—recovering Boltzmann distribution for constraint of mean energy; connecting statistical mechanics to Bayesian inference and information theory. Landauer's principle (1961): erasing one bit of information dissipates minimum heat Q_min = k_B*T*ln(2) = 0.018 eV at 300 K—connecting computation and thermodynamics. Landauer limit experimentally verified in single-bit erasure of colloidal particle in bistable optical trap (Bérut et al. 2012, Nature). Maxwell's demon—proposed thought experiment of a demon sorting fast/slow molecules reducing entropy without work—resolved by noting the demon's memory erasure restores entropy to be consistent with the second law (Bennett 1982). Black hole thermodynamics: Bekenstein-Hawking entropy S_BH = k_B*A/4*l_Planck^2 (A = event horizon area) is real statistical mechanical entropy—the holographic principle (information in a volume encoded on its boundary area) emerged from this connection between gravity and thermodynamics.

Example 4: Superfluidity and Bose-Einstein Condensation

Helium-4 undergoes a superfluid transition at T_lambda = 2.17 K (the lambda transition—named for the lambda-shaped heat capacity anomaly)—entering a state of zero viscosity, quantised vortex, and frictionless flow. BEC theory (Fritz London, 1938): He-4 superfluidity arises from macroscopic occupation of the zero-momentum ground state (BEC of bosonic He-4 atoms); superfluid fraction equals condensate fraction at T=0 but is not equal at finite T due to interactions. Landau two-fluid model: superfluid has two components—superfluid (no viscosity, no entropy) and normal (viscous, carries entropy)—explains thermal counterflow (second sound), Andronikashvili experiment measuring superfluid fraction as function of temperature. Quantised vortices (Feynman 1955): in rotating superfluid, angular momentum enters only through quantum vortices with circulation h/m—vortex arrays in rotating He-4 and He-3 are directly imaged as evenly spaced dots in Abrikosov-lattice-like patterns by ion trapping. He-3 superfluidity (Nobel 1996 Lee, Osheroff, Richardson): anisotropic p-wave (L=1, S=1) Cooper pairing at ~2 mK—more complex order parameter with topological defects and anisotropic properties illuminating unconventional superconductor physics.

Example 5: Polymer Physics and Soft Matter

Polymers—long chain molecules with N monomer units—exhibit rich statistical mechanical behaviour as flexible random walks in solution and melt. Freely jointed chain: end-to-end distance R = sqrt(N)*l (l=monomer size)—Gaussian chain statistics from random walk. Worm-like chain (WLC) with persistence length Lp: captures finite bending stiffness; force-extension F(z,L) = k_B*T/Lp * (z/L + 1/4(1-z/L)^-2 - 1/4)—WLC model fits single-molecule DNA stretching experiments to within experimental error. Polymer solutions: Flory theory for polymer-solvent interactions—theta temperature (second virial vanishes), good solvent swelling R~N^(3/5)*l, poor solvent collapse R~N^(1/3)*l. Reptation model (de Gennes Nobel 1991): polymer diffusion in entangled melts occurs by snake-like motion through effective tube formed by surrounding chains—predicts viscosity eta ~ M^3 (M=molecular mass, observed exponent ~3.4 empirically; slight deviation from reptation due to contour length fluctuations and constraint release). Polymer physics governs rubber elasticity, viscoelastic flow in processing, gel swelling/collapse in drug delivery, and DNA packing in chromosomes—underpinning materials science from tyre rubber to genomics.

Example 6: Stochastic Processes and Brownian Motion

Brownian motion—the random diffusive movement of small particles (~1 micrometer) visible in optical microscope—was explained by Einstein (1905) as arising from thermal fluctuations of molecular collisions: mean squared displacement MSD = 2*D*d*t (d=dimensionality, D=diffusion coefficient = k_B*T/(6*pi*eta*R)). Langevin equation: m*dv/dt = -gamma*v + xi(t) (gamma=friction, xi(t)=Gaussian white noise random force); leads to velocity autocorrelation function decaying exponentially and MSD linear in time for t >> m/gamma. Fokker-Planck equation: partial P/partial t = -partial/partial x (A(x)*P) + (1/2) partial^2/partial x^2 (B(x)*P)—deterministic PDE for probability density P(x,t) of stochastic process. Applications: stock price models (geometric Brownian motion); polymer translocation through nanopores; protein diffusion on chromosomal DNA; ion channel gating kinetics. Anomalous diffusion: MSD ~ t^alpha (alpha≠1)—subdiffusion (alpha<1, crowded biological media) and superdiffusion (alpha>1, active transport, Lévy flights) depart from normal Brownian diffusion in biologically relevant systems characterised by single-particle tracking experiments.

Example 7: Thermodynamics of Black Holes

Black hole thermodynamics—the analogy and suspected physical identity between black hole mechanics and thermodynamic laws—is one of the deepest results in theoretical physics. The four laws of black hole mechanics (Bardeen, Carter, Hawking 1973) precisely parallel thermodynamic laws: zeroth (surface gravity kappa constant over horizon ↔ temperature uniform at equilibrium); first (area-mass-charge changes satisfy dM = kappa/8pi * dA + Omega*dJ + Phi*dQ ↔ dU=T*dS-P*dV); second (area of event horizon never decreases, dA>=0 ↔ dS>=0); third (kappa cannot reach zero in finite steps ↔ absolute zero unattainable). Bekenstein-Hawking entropy S = k_B*c^3*A/(4*hbar*G)—enormous for macroscopic black holes (solar mass BH: S ~ 10^77 k_B, vs. ~10^58 k_B for Sun). The fact that entropy is proportional to area (not volume) spawned the holographic principle (t'Hooft, Susskind): all information in a 3D volume is encoded on its 2D boundary—a profound constraint on any quantum theory of gravity and the inspiration for AdS/CFT duality (Maldacena 1997) relating quantum gravity in anti-de Sitter space to conformal field theory on its boundary.

Example 8: Phase Diagrams and Materials Design

Phase diagrams—maps of thermodynamically stable phases as functions of temperature, pressure, and composition—are fundamental tools for materials processing and design. Fe-C phase diagram: dictates the phases (austenite, ferrite, cementite, martensite) and transformations (eutectoid at 723°C, 0.76 wt% C) controlling steel mechanical properties through heat treatment. Multi-component phase diagrams computed by CALPHAD (CALculation of PHAse Diagrams) using thermodynamic databases of Gibbs energies fit to experimental data—enable alloy design: Ni-based superalloy composition optimisation (gamma/gamma prime precipitate fraction), Ti alloy alpha/beta phase boundary control for aerospace structures, Li-ion battery cathode stability versus electrolyte composition range. CALPHAD coupled with DFT (first-principles phase diagram stability for new alloy systems without experimental data)—accelerates materials design from alloy development timescales of decades to years. High-entropy alloys (HEA): 5+ principal elements in equimolar or near-equimolar compositions forming single-phase solid solutions stabilised by configurational entropy—exhibiting outstanding strength-ductility trade-offs and radiation damage resistance—discovered 2003-2004 (Yeh, Cantor groups), theoretically explained by CALPHAD-DFT hybrid thermodynamic frameworks.

Try it live

Everything above runs in your browser — open SPH Fluid and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.