The NPU die is modeled as a lumped thermal mass with a first-order RC thermal model, integrated every frame with a clamped timestep:
C · dT/dt = P(t) − k_cool · (T − T_ambient)
P(t) = P_idle + throughput(t) · E_token(quant)
C is the die's thermal capacitance, k_cool the heat-dissipation coefficient of the cooling solution, and E_token the energy spent per generated token — lower at INT4 than INT8 than FP16, since fewer bits move through the matrix-multiply units per weight.
A DVFS (dynamic voltage/frequency scaling) controller watches T and scales the clock — and therefore the achievable throughput — down once the die crosses a warning threshold, hitting a hard floor at the thermal limit:
clock_scale(T) = 1 T ≤ T_warm
clock_scale(T) = 1 − (T−T_warm)/(T_max−T_warm)·(1−s_min) T_warm < T < T_max
clock_scale(T) = s_min T ≥ T_max
throughput(t) = min(demand, max_tok_s · clock_scale(T))
- Requested generation rate — how fast the personal assistant is asking the model to produce tokens (a long streamed answer vs. a short reply).
- Weight precision — INT4/INT8 quantization cuts energy per token, which is exactly why on-device assistants quantize aggressively: it buys thermal headroom, not just smaller model files.
- Cooling solution — sets k_cool; a fanless phone throttles far sooner than a vapor-chamber flagship or an actively-cooled tablet dock.
- Start long generation burst — sustains the requested rate long enough to show the full heat-up → throttle → equilibrium cycle, then lets the die cool once the burst ends.
This is the real mechanism behind the throughput dips users see when a local LLM-based assistant (e.g. Phi-3, Gemma, or a quantized Llama 3 running fully on-device for privacy) is asked for a long response: the chip cannot sustain its peak tokens/sec indefinitely, and the DVFS controller trades speed for staying inside its thermal budget.
Honest simplifications. A single lumped (or 24×24 grid) thermal capacitance of 5.2 J/°C, linear cooling k·(T−Tamb), a piecewise-linear DVFS ramp between 45 and 62 °C and joules-per-token values are illustrative parameters for a phone-class NPU, not data for any real chip. Real SoCs have several thermal zones, skin-temperature limits, battery heat and memory-bound (not clock-bound) token rates. The steady state solves P(T) = k·(T−Tamb) with P = Pidle + throughput(T)·Etoken; the checks compare that fixed point with time integration, the exact RC step response and (2D) the full grid.