🧠 Thermodynamics Meets Information Theory: Maxwell's Demon and Landauer's Principle

Two Entropies, One Formula

Boltzmann's entropy is S = k_B · ln(W), where W is the number of microstates compatible with the observed macrostate and k_B = 1.38 × 10⁻²³ J/K is Boltzmann's constant. For a gas in a box, W counts the number of ways to arrange all the molecules while keeping the same total energy, volume, and particle count. High W (many microstates) means high entropy — the system is "spread out" in phase space.

Shannon's information entropy is H = −Σ p_i · log₂(p_i), where p_i are the probabilities of each possible message symbol. H is maximized when all symbols are equally likely (maximum uncertainty about the message) and zero when one symbol has probability 1 (perfect certainty). Shannon showed H is the only function satisfying a natural set of axioms for measuring uncertainty.

The two formulas are identical up to a constant: replace log₂ with ln and multiply by k_B to convert Shannon entropy to Boltzmann entropy. This is not merely formal similarity — they are measuring the same thing: the number of microstates compatible with our knowledge of the macrostate. Physical entropy IS information entropy about molecular positions and momenta.

Maxwell's Demon: A Violation of the Second Law?

In 1867, James Clerk Maxwell proposed a thought experiment to challenge the second law of thermodynamics. Imagine a box of gas divided by a partition with a small trapdoor. A tiny "demon" controls the trapdoor and can observe individual molecules approaching it. The demon allows fast molecules to pass left-to-right and slow molecules to pass right-to-left, systematically moving kinetic energy from the right compartment to the left. With enough time, the left side becomes hot and the right cold — a temperature gradient created without doing any work. This seems to violate the second law.

For decades, physicists offered various resolutions. Leo Szilard (1929) showed that the demon's act of measuring which molecules are fast or slow requires gaining information, and this information acquisition must cost entropy equivalent to the entropy decrease produced. But the argument was hand-wavy about where exactly the entropy cost appeared.

Landauer's Principle: Erasing a Bit Costs Energy

Rolf Landauer's 1961 paper provided the precise answer. Measurement itself can be thermodynamically reversible — a demon with unlimited memory can observe molecules without dissipating heat. The irreversible step is erasing the demon's memory. To cycle through the process repeatedly, the demon must eventually reset its memory registers to a known state (erase the old measurements). This erasure is logically irreversible: many input states map to one output state (the "erased" state). By Liouville's theorem (phase space volume is conserved in Hamiltonian mechanics), logically irreversible operations must increase entropy somewhere — and the heat dissipated is:

Q_erase ≥ k_B · T · ln(2)  per bit erased

At room temperature (T = 300 K), this is about 2.9 × 10⁻²¹ joules per bit — an incredibly small amount, but fundamental. The demon erasing its memory to repeat the process dissipates at least as much heat as the entropy decrease it produces. The second law is saved — but the cost is placed precisely on information erasure, not on measurement.

Landauer's Principle Verified Experimentally

In 2012, Antoine Bérut and colleagues at ENS Lyon experimentally verified Landauer's principle using a colloidal particle in a double-well optical trap. The particle's position encoded one bit (left well = 0, right well = 1). Erasing the bit (forcing the particle to the left well regardless of starting state) released heat of kT·ln(2) per erasure — within 5% of the Landauer bound, confirming the principle at room temperature with single-bit precision.

This places a fundamental thermodynamic limit on computation. A bit erasure costs at minimum kT·ln(2) ≈ 0.017 eV at room temperature. Modern transistors dissipate ~1000× this per switching operation — we are still far from the Landauer limit. But as chips approach nanoscale and approach reversible computing architectures, Landauer's principle will constrain chip design.

Reversible Computation and the Thermodynamics of Algorithms

If erasure costs energy but computation itself does not, then logically reversible computation — where every output uniquely determines the inputs — can in principle be thermodynamically free. Charles Bennett (1973) showed that any computation can be made logically reversible by keeping all intermediate results (so outputs uniquely determine inputs). The Toffoli gate, CNOT gate, and Fredkin gate are universal reversible logic gates with zero thermodynamic cost per operation (in principle).

The thermodynamic cost of an algorithm is proportional not to its running time but to the number of bit erasures it performs — the information it discards. Sorting algorithms that discard information (comparisons whose outcomes are not preserved) pay a thermodynamic price. Fully reversible sort algorithms exist but require O(n log n) ancilla bits to store all comparisons, which must be erased at the end — the costs reappear at cleanup.