Software-based real-time inference of hard-to-measure bioprocess states from easy-to-measure indirect signals — the core of modern Process Analytical Technology
Viable cell density (VCD), specific productivity (qP), and intracellular metabolic flux govern the economics and quality of every biomanufacturing run — yet none of them can be read from a probe dipped into the bioreactor. They are measured, when they are measured at all, by slow offline assays: manual or automated cell counting, HPLC titer analysis, or LC-MS flux tracing, each taking anywhere from twenty minutes to several days and consuming operator time and sample volume the process can rarely spare.
A "soft sensor" (soft = software) is an algorithm that infers the value of a variable that cannot be measured directly — or can only be measured slowly and offline — from other variables that are measured continuously and cheaply. It is not a physical probe; it is a trained mathematical model sitting between the raw signal stream and the operator, converting correlated but indirect measurements into a continuous estimate of the state that actually matters.
This distinction matters because it changes where the engineering effort goes. A hardware sensor requires a new physical transducer, sterilizability, calibration hardware, and regulatory qualification of new equipment. A soft sensor requires historical data, a validated model, and a monitoring plan for drift — no new hardware touches the process stream at all.
Several classes of critical process states resist direct real-time measurement for fundamentally different reasons:
• Viable cell density: requires physical counting (trypan blue exclusion, image cytometry) or lysis-based assays — both destructive, both requiring sample extraction from a sterile, pressurized, agitated vessel • Specific productivity (qP): the rate of product formation per cell per unit time — a derived quantity requiring both a titer measurement and a simultaneous, precisely time-matched viable cell count • Intracellular metabolic flux: fluxes through central carbon metabolism are not measurable by any single probe at all; they must be inferred from isotope labeling experiments (¹³C-MFA) that take days • Product quality attributes (glycosylation, aggregation): require chromatography or mass spectrometry, fundamentally incompatible with continuous in-line measurement
Each of these states is exactly the kind of variable a plant would most like to control tightly — and exactly the kind that classical instrumentation cannot deliver in real time.
The soft sensor paradox: the variables with the greatest impact on batch outcome and product quality are, almost without exception, the variables that are slowest and most expensive to measure directly. This gap between "what we need to know" and "what we can measure" is precisely the gap soft sensors are built to close.
Without a continuous estimate, operators steer a bioreactor using yesterday's data point. A culture can shift from exponential growth into a nutrient-limited or lactate-inhibited decline phase in the hours between two offline samples, and by the time the assay result returns, the optimal window for a feed correction or harvest decision has already passed.
This is the operational motivation for soft sensing: not to replace the offline assay (which remains the ground-truth reference), but to fill the gaps between samples with a model-based estimate that is continuously available, cheap to compute, and good enough to act on in real time.
A modern bioreactor is already instrumented far more richly than it first appears. Dissolved oxygen probes, pH electrodes, off-gas mass spectrometers, capacitance/permittivity probes, and load cells tracking base addition all stream continuous data — none of which directly reports cell density or productivity, but all of which are physically coupled to the biology producing those states.
Radio-frequency permittivity probes measure the capacitance of the culture across a range of frequencies. Only intact, membrane-bound (i.e., viable) cells polarize under an applied field, so the resulting permittivity signal (often reported in pF/cm) correlates strongly and specifically with viable biovolume — largely insensitive to dead cells, debris, and bubbles that confound optical density measurements.
Because the physical relationship between permittivity and viable biovolume is comparatively direct, capacitance is often the single strongest input signal for a VCD soft sensor, though it still benefits from being combined with other signals to resolve morphology changes and probe drift.
Off-gas CO₂ and O₂ concentrations, measured continuously by mass spectrometry or infrared/paramagnetic analyzers on the exhaust stream, give the oxygen uptake rate (OUR) and carbon dioxide evolution rate (CER):
OUR = Q_gas × (O₂_in − O₂_out) / V_liquid CER = Q_gas × (CO₂_out − CO₂_in) / V_liquid
Both track cellular respiration and therefore active metabolism — OUR in particular tends to scale with viable cell density and specific growth rate, while the respiratory quotient (CER/OUR) carries information about substrate utilization and metabolic state shifts (e.g., the onset of lactate consumption).
Individually weaker predictors, these signals become valuable when combined with the above:
• Dissolved oxygen (DO): the control loop's own actuation (agitation, sparge rate needed to hold DO setpoint) is itself an indirect measure of oxygen demand, which scales with cell mass and activity • pH: drifts with metabolic byproduct accumulation (lactate, CO₂); the base addition needed to hold pH setpoint (mL of base per hour) tracks metabolic acid production, often correlating with growth phase and productivity • Agitation/feed rate trends: in fed-batch processes, the feed rate profile itself carries information the model can use as a covariate
No single one of these signals reliably tracks the hidden target state end-to-end — the value of a soft sensor comes precisely from fusing several imperfect, cheap signals into one better estimate than any single signal could provide alone.
A soft sensor is only as good as the historical dataset it is trained on: batches where both the continuous easy-to-measure signals and the offline reference target (VCD, titer, or another hard-to-measure state) were recorded, time-aligned, and paired. The modeling approach ranges from classical multivariate statistics to modern machine learning to hybrid structures that combine both with mechanistic process knowledge.
Partial Least Squares (PLS) regression remains the most widely deployed soft sensor technique in bioprocessing because it handles exactly the data pathology that process signals present: many correlated (collinear) input variables, few independent samples, and noisy offline reference values.
PLS finds latent variables (linear combinations of the input signals) that maximize covariance with the target variable, rather than simply maximizing variance explained in the inputs (as PCA would). This makes it robust to collinear, redundant sensor channels and interpretable — each latent variable's loading vector shows which raw signals are driving the prediction, which matters enormously for regulatory acceptance in a GMP environment.
Model form: ŷ = X · B, where B is derived from a small number of latent components fit by iterative deflation (NIPALS algorithm), typically cross-validated to select the optimal number of components and avoid overfitting on a training set of a few dozen historical batches.
Where the input-output relationship is strongly nonlinear or batch dynamics carry significant time dependence, machine learning models extend beyond PLS's linear assumption:
• Recurrent neural networks (LSTM/GRU) and 1D-CNNs: capture temporal dynamics across the full signal trajectory rather than a single time-point snapshot • Gaussian process regression: provides a predictive estimate together with a calibrated uncertainty band, valuable for confidence-aware control • Random forests / gradient boosting: robust to outliers and require less feature engineering, at some cost in extrapolation behavior
Hybrid mechanistic-data models combine a first-principles kinetic or mass-balance model (e.g., Monod-type growth kinetics, stoichiometric yield relationships) with a data-driven correction term that learns the residual between the mechanistic prediction and the observed reference values. This structure tends to extrapolate more gracefully outside the training envelope than a pure black-box model, because the mechanistic backbone enforces known physical constraints (mass conservation, non-negative concentrations) even where training data is sparse.
A hybrid model does not need to relearn conservation of mass from data — the mechanistic core already enforces it. The data-driven layer only needs to learn what the mechanistic model gets wrong, which is typically a much smaller, smoother function than the full input-output map, and requires proportionally less training data to fit reliably.
Training data quality depends on rigorous preparation:
1. Time alignment: offline sample timestamps must be matched precisely to the corresponding window of continuous signal data, accounting for sensor lag and sample processing delay 2. Signal preprocessing: smoothing/denoising raw signals, removing probe drift and calibration jumps, synchronizing sample rates across sensor types 3. Outlier handling: excluding batches with known process deviations, contamination events, or sensor faults that would corrupt the learned relationship 4. Batch-wise cross-validation: leaving entire historical batches out (not just individual time points) during validation, since consecutive time points within one batch are highly autocorrelated and would otherwise leak information and produce falsely optimistic accuracy estimates
Model accuracy scales directly with the number and diversity of training batches available — a model trained on runs spanning multiple clones, scales, and media lots generalizes far better than one trained on a narrow set of nominal runs.
Once validated, the soft sensor is deployed onto the live process data historian or control system, where it consumes the same continuous signals it was trained on — now streaming from a running bioreactor — and outputs a continuously updated estimate of the hidden state, typically refreshed every few seconds to a couple of minutes, with no offline sample required to produce each new value.
Deployment turns a fitted model object into a running service subscribed to the plant's real-time data infrastructure:
1. Signal ingestion: the model subscribes to the same tag names (DO, pH, capacitance, off-gas CO₂/O₂, base pump rate) via the plant historian or OPC-UA server that streamed the training data 2. Preprocessing parity: the exact same smoothing, scaling, and feature engineering pipeline used during training must be replicated online — a mismatch here ("training-serving skew") is one of the most common sources of deployed soft sensor failure 3. Inference: at each update interval, the current signal window is passed through the fitted model (PLS coefficients, network weights, or hybrid model equations) to produce a new state estimate 4. Publishing: the estimate is written back to the historian as a new virtual tag, visible to operators on the same trending screens as physical sensor readings, and available to downstream control logic
The right update frequency balances responsiveness against noise: too fast, and the estimate jitters with sensor noise; too slow, and it fails to capture fast transitions like a metabolic shift or a feed pump fault.
Most deployed soft sensors update on the order of once per minute, matched to the natural time constant of the biological process itself (cell growth and metabolic changes unfold over hours, not seconds) rather than to the fast sampling rate of the underlying instruments. Many implementations apply a light exponential or Kalman-style smoothing filter to the raw model output, trading a small amount of responsiveness for a cleaner, more trustworthy trend line for operators to act on.
A model deployed on live data will inevitably encounter operating conditions outside its training envelope: an unusually high feed rate, an atypical seed density, a new clone with different metabolism. Because the model has only ever learned the input-output relationship observed in historical batches, its behavior outside that envelope is unconstrained and potentially unreliable.
Production soft sensor implementations typically pair the point estimate with an applicability domain check — flagging when live signal values fall outside the range seen during training — so operators know to weight the estimate with appropriate caution rather than trusting it blindly during unusual process excursions.
A soft sensor extrapolating silently is more dangerous than no soft sensor at all — a confident but wrong estimate can drive a bad feedback control decision. Robust deployments always pair the numeric estimate with an explicit confidence or applicability-domain signal, not just the number itself.
A deployed soft sensor is never "done" — its estimate must be continually checked against sparse offline reference measurements to confirm it has not drifted from ground truth, and only once that trust is established does the estimate earn a role feeding real-time release testing (RTRT) and closed-loop feedback control strategies such as model predictive control (MPC).
Even after deployment, offline reference sampling does not stop — it changes role, from training data source to independent audit trail. Each offline VCD count or titer assay collected during a run is compared against the soft sensor estimate at the matching timestamp, generating a running residual:
residual(t) = |estimate(t) − reference(t)| / reference(t)
Tracking this residual across many production runs establishes the deployed model's real-world accuracy (commonly reported as RMSE or mean absolute percentage error) and — critically — reveals drift: a systematic degradation in accuracy over time caused by sensor recalibration, raw material lot changes, a new cell line, or seasonal drift in probe baselines.
A soft sensor that was accurate at qualification can silently become inaccurate as the process, equipment, or biology it was trained on evolves. Structured drift monitoring typically includes:
• Control charts on the estimate-vs-reference residual, with statistically derived alert and action limits • Periodic revalidation campaigns comparing the live model against a fresh batch of paired offline data • Scheduled or triggered retraining: when drift crosses a defined threshold, the model is refit incorporating the newest batches, expanding its training envelope and correcting for any systematic bias that has emerged
This lifecycle mirrors the calibration and requalification discipline applied to physical instruments — a soft sensor is treated as a measurement device subject to the same GMP change-control rigor, even though it exists only as code and coefficients.
Once a soft sensor has demonstrated sustained accuracy across enough validated runs, its estimate can graduate from a monitoring display to an active input for process decisions:
• Real-Time Release Testing (RTRT): a validated soft sensor estimate, together with other PAT measurements, can substitute for a subset of end-of-batch offline quality testing, accelerating batch disposition • Feedback control: the continuous VCD or productivity estimate becomes a controlled variable in a feed-rate or temperature control loop, allowing the process to react to the current estimated state rather than yesterday's offline sample • Model Predictive Control (MPC): the soft sensor estimate is fed as the current state into an MPC optimizer that forecasts the process trajectory forward and computes the optimal control action (e.g., feed profile) to hit target productivity or final titer, re-solving at every new soft sensor update
This is the payoff of the entire soft sensor pipeline: a state that could never be measured fast enough to control directly becomes, through modeling, a real-time controlled variable — closing a feedback loop that offline assays alone could never support.
The transition from open-loop monitoring to closed-loop control is the single highest-value milestone in a soft sensor program: it converts a passive dashboard number into an active decision variable that steers the process in real time, directly translating estimation accuracy into improved, more consistent batch outcomes.