Turning wearable-derived signals into regulatory-grade endpoints — the V3 framework, agreement statistics, and the FDA/EMA qualification pathway for decentralized trials
Every digital biomarker begins as an engineering question, not a clinical one: does a given sensor — accelerometer, photoplethysmography (PPG) LED, single-lead ECG, microphone, GPS — faithfully capture the physical quantity it claims to measure? The Digital Medicine Society (DiMe) formalized this first step as "Verification" within its V3 framework (Goldsack et al., npj Digital Medicine, 2020), the now-standard evidentiary structure referenced explicitly in FDA's December 2023 final guidance on Digital Health Technologies for Remote Data Acquisition in Clinical Investigations.
DiMe's V3 model decomposes "is this device fit for purpose?" into three separable, sequential questions, each requiring its own evidence package:
Verification: does the sensor accurately measure the underlying physical signal (e.g., linear acceleration in g, raw PPG waveform, ECG lead I voltage) under controlled bench conditions, against a metrology-traceable reference (optical motion capture, force plate, ECG simulator)? This is a hardware/firmware question with no clinical claim attached.
Analytical Validation: does the algorithm that turns the raw signal into a clinically interpretable measure (e.g., steps, stride velocity, heart rate, sleep stage) produce accurate output across the intended use population and use conditions? This is where device output is benchmarked against a criterion-standard instrument in humans.
Clinical Validation: does the resulting measure meaningfully relate to a clinical state, biological process, or response to intervention in the target population? This is where a Biometric Monitoring Technology (BioMeT) becomes a candidate endpoint, not merely an accurate sensor.
A device can pass Verification and Analytical Validation yet fail Clinical Validation entirely — an accelerometer can count steps with near-perfect fidelity (Verification+Analytical) while step count shows no meaningful relationship to disease severity in a given indication (Clinical Validation fails). Each layer must be established independently and is indication-specific: the same wearable validated for Parkinson's gait analysis requires an entirely new Clinical Validation package for a Duchenne muscular dystrophy program.
Before any bench testing, sponsors run a structured concept-of-interest (COI) exercise, typically informed by qualitative patient research and literature review, to map disease biology to measurable physical signals:
Mobility/gait disorders (Parkinson's, Duchenne, multiple sclerosis): tri-axial accelerometry + gyroscope at the ankle, wrist, or lumbar region; candidate measures include stride velocity, cadence, turning speed, arm swing asymmetry.
Cardiopulmonary (heart failure, COPD): single- or multi-lead ECG patches, PPG-based heart rate/HRV, respiratory inductance plethysmography, pulse oximetry; candidate measures include resting HR, HRV (SDNN, RMSSD), respiratory rate trends, activity-adjusted SpO2.
CNS/psychiatric (depression, schizophrenia): passive smartphone sensing (screen time, typing cadence, GPS-derived mobility radius, call/text metadata), actigraphy-derived sleep-wake cycles, voice biomarkers (prosody, pause structure, spectral features).
Oncology (functional status, cachexia): step count, sedentary time, sleep fragmentation as digital analogues of Karnofsky/ECOG performance status.
Of 8–15 candidate signal/algorithm combinations typically screened at this stage, only 2–4 usually survive bench verification with acceptable signal-to-noise ratio and are advanced into human analytical-validation studies — the rest fail on sensor drift, motion artifact susceptibility, or insufficient dynamic range.
Analytical validation asks a narrower, more tractable question than clinical meaningfulness: in humans, across the intended use conditions, does the algorithm's output agree with a criterion-standard measurement instrument? This is where Bland-Altman bias and limits of agreement, mean absolute percentage error (MAPE), and intraclass correlation coefficients (ICC) become the primary statistical currency, and where device performance across demographic strata (skin tone for PPG, BMI for accelerometry placement, age for gait variability) is formally stress-tested.
A near-universal pitfall in early digital biomarker programs is reporting Pearson's r as proof of validity — correlation measures linear association, not agreement, and two instruments can correlate at r=0.95 while being systematically and clinically unacceptably biased. The accepted toolkit instead includes:
Bland-Altman analysis: plots the difference between the two methods against their mean, yielding a bias (mean difference) and 95% limits of agreement (bias ± 1.96 SD of the differences). A device with near-zero bias but wide limits of agreement is precise on average but unreliable for any single patient — a critical distinction for individual-level clinical decisions versus population-level trial endpoints.
Intraclass correlation coefficient (ICC), specifically ICC(2,1) or ICC(3,1) per the Shrout-Fleiss convention: quantifies both consistency and absolute agreement between raters/instruments; COSMIN (COnsensus-based Standards for the selection of health Measurement Instruments) guidance treats ICC ≥0.75 as good and ≥0.90 as excellent reliability for a measurement instrument intended for individual-level use.
Mean absolute percentage error (MAPE): favored for continuous physiological signals (heart rate, respiratory rate) because it is scale-independent and clinically interpretable; <10% MAPE is a common — though not universal — regulatory expectation for continuous vital-sign monitors.
Equivalence testing (two one-sided tests, TOST): formally tests whether the digital measure falls within a pre-specified clinically acceptable margin of the reference, rather than merely testing for a statistically detectable difference — the latter becomes trivially significant at large sample sizes even for clinically irrelevant bias.
FDA and independent researchers (notably a 2020 New England Journal of Medicine correspondence on pulse oximetry) have documented that optical sensors systematically underperform in individuals with darker skin pigmentation, higher BMI, and certain wrist anatomies — motivating explicit stratified validation requirements now embedded in FDA's DHT guidance and in most sponsor Qualification Plans:
• Skin tone: validated across the full Fitzpatrix/Monk Skin Tone scale range for any PPG-based measure (HR, SpO2, HRV) • Body habitus: BMI strata spanning underweight to Class III obesity for accelerometry-based energy expenditure and step-count algorithms • Age range: pediatric, adult, and geriatric cohorts separately, since gait variability, skin elasticity (affecting PPG coupling), and cardiac conduction all shift with age • Use environment: treadmill/lab-controlled versus free-living conditions; device placement variability (wrist-worn accelerometers can shift 10–20% in step-count accuracy purely from strap tightness) • Comorbidities: tremor (Parkinson's), edema (heart failure), and arrhythmia (atrial fibrillation) each independently degrade signal quality for the algorithms most likely to be deployed in exactly those populations
A validation package that pools all subgroups into a single aggregate ICC without subgroup reporting is now a common deficiency cited in FDA pre-submission feedback and in EMA qualification advice letters.
Clinical validation is where a technically accurate digital measure becomes (or fails to become) a candidate trial endpoint. The central question shifts from "does the algorithm agree with a reference instrument" to "does a change in this digital score correspond to a change that matters to a patient, clinician, or regulator." This requires anchoring the digital measure against an established clinical outcome assessment and deriving a meaningful within-patient change (MWPC) threshold.
Establishing what a numeric change in a digital score means clinically follows methodology adapted from patient-reported outcome (PRO) science, now standard in FDA's PRO guidance and increasingly applied to digital measures:
Anchor-based methods: correlate change in the digital measure with change on an external "anchor" that has established clinical meaning — a Patient (or Clinician) Global Impression of Change (PGIC/CGIC) scale, a validated PRO instrument, or a categorical clinical milestone (e.g., loss of ambulation). The mean digital-score change among patients who report "minimally improved" on the anchor becomes the MWPC estimate.
Distribution-based methods: derive a threshold from the statistical properties of the measure itself — commonly 0.5 × baseline SD, or the standard error of measurement (SEM = SD × √(1−ICC)) multiplied by 1.96. These serve as sensitivity checks against anchor-based estimates rather than stand-alone justification, since a purely distribution-based threshold has no inherent clinical meaning.
Triangulation: regulators (and the 2009 FDA PRO guidance, now extended in practice to digital endpoints) expect multiple anchors and both method families to converge on a similar range before an MWPC is treated as defensible for use in a pivotal trial's responder analysis or as the basis for a minimal clinically important difference (MCID) claim.
Stride Velocity 95th Centile (SV95C), derived from the ActiMyo ankle-worn accelerometer (Sysnav), became the first digital mobility endpoint to receive both an EMA qualification opinion (2019) and FDA biomarker qualification (2022) for use as a secondary/exploratory endpoint in Duchenne muscular dystrophy trials — establishing continuous, free-living gait speed as a validated alternative to intermittent, effort-dependent clinic-based tests like the North Star Ambulatory Assessment.
The choice of clinical anchor determines what the eventual digital endpoint can claim to represent:
Mobility disorders: 6-Minute Walk Test (6MWT), Timed 25-Foot Walk (T25FW, multiple sclerosis), North Star Ambulatory Assessment (NSAA, Duchenne), UPDRS Part III motor subscore (Parkinson's)
Cardiopulmonary: NYHA functional class, KCCQ (Kansas City Cardiomyopathy Questionnaire) for heart failure, SGRQ (St George's Respiratory Questionnaire) for COPD, hospitalization/rehospitalization events
CNS: MADRS/HAM-D for depression, PANSS for schizophrenia, MDS-UPDRS for Parkinson's non-motor symptoms
Oncology: ECOG/Karnofsky performance status, PRO-CTCAE symptom burden
The strength of the anchor relationship (typically expressed as a correlation coefficient or effect size between digital-measure change and anchor-category membership) is itself a key clinical validation statistic — regulators generally expect at least a moderate correlation (r≈0.3–0.5 is common for functional-status anchors, given the anchors themselves are imperfect) alongside a plausible, mechanistically coherent explanation for the relationship.
A digital biomarker validated in a controlled clinic visit can behave very differently once deployed continuously, unsupervised, in patients' homes for weeks or months — the default deployment model in decentralized clinical trials (DCTs). This stage surveils measurement stability over time, handles the inevitability of missing data, and screens for algorithm or hardware drift across firmware updates and device replacements.
Continuous remote monitoring inevitably produces incomplete data: device removed for charging or bathing, connectivity dropouts, participant non-adherence, or battery failure. The statistical handling of this missingness must be pre-specified, not improvised post hoc:
Valid-day definitions: a common convention (adapted from actigraphy sleep research and used across many DCT protocols) requires ≥10 hours of valid wear-time within a 24-hour period for that day to count toward the analysis; a valid-week or valid-month requires a minimum fraction of valid days (commonly ≥70%, i.e., ≥5 of 7 days).
Missingness mechanism classification: sponsors must characterize whether data are Missing Completely At Random (MCAR), Missing At Random (MAR), or Missing Not At Random (MNAR) per ICH E9(R1) — critically, disease-driven non-adherence (a patient too fatigued or immobile to wear/charge a device) is often MNAR, since the missingness is directly informative about the outcome of interest, and naive imputation can bias effect estimates toward the null or introduce spurious treatment effects.
Handling approaches: multiple imputation under MAR assumptions, tipping-point sensitivity analyses under MNAR scenarios, and — increasingly preferred by regulators — estimands that explicitly define how device non-wear during worsening health is to be treated (e.g., a "treatment policy" estimand counting non-wear as part of the outcome versus a "hypothetical" estimand assuming full adherence).
Unlike a single clinic-based validation session, months of continuous monitoring expose the biomarker to sources of variability entirely absent from the lab:
Algorithm/firmware drift: device or cloud-side algorithm updates mid-trial can silently shift the measurement scale; sponsors increasingly freeze algorithm versions for the duration of a pivotal trial and require formal change-control documentation (analogous to CMC change control for a drug product) for any mid-study update.
Device-to-device variability: even within the same model, manufacturing tolerances and calibration drift over a device's service life can introduce measurement variance; device swaps (battery failure, loss, damage) require re-calibration verification before continued inclusion of that participant's data stream.
Environmental confounding: outdoor GPS-based gait measures are affected by terrain and weather; PPG-based HR measures are affected by ambient temperature and motion artifact from daily activities never seen in a treadmill validation; smartphone-passive-sensing measures are confounded by device-sharing, multiple owned devices, and behavior changes unrelated to disease (e.g., a new job changing mobility radius).
Seasonal and circadian effects: mood, activity, and sleep digital biomarkers exhibit strong seasonal and day-of-week periodicity that must be accounted for in the statistical model (e.g., via mixed-effects models with random participant intercepts and fixed seasonal terms) rather than misattributed to treatment effect.
The CTTI (Clinical Trials Transformation Initiative, a Duke-FDA partnership) has published consensus recommendations since 2018 specifically addressing DHT selection, deployment logistics, and data-quality monitoring for exactly these real-world deployment challenges.
Once a digital biomarker carries a complete V3 evidence package, sponsors can pursue formal regulatory qualification — a determination, independent of any single drug application, that the biomarker is fit for a specific, narrowly defined Context of Use (COU). A qualified biomarker can then be relied upon by any sponsor developing a product in that COU without re-litigating its validity in every individual submission.
FDA's Biomarker Qualification Program, operated jointly by CDER and CBER and given a defined statutory pathway under the 21st Century Cures Act (2016), proceeds through staged submissions:
1. Letter of Intent (LOI): a brief statement of the proposed biomarker, the disease area, and the intended Context of Use, reviewed by the Biomarker Qualification Review Team
2. Qualification Plan: a detailed protocol-like document specifying the evidence to be generated or already available — the V3 package, the statistical analysis plan for establishing analytical and clinical validity, and critically, the precise boundaries of the COU (which disease, which population, which stage of disease, used as which type of endpoint — primary, secondary, exploratory, or enrichment/stratification biomarker)
3. Full Qualification Package: the complete evidentiary submission, reviewed by FDA's internal qualification review team and, for higher-impact submissions, discussed at a public Biomarker Qualification Program workshop
4. Qualification decision: FDA issues a qualification memorandum (for older Drug Development Tool submissions) or, since Cures Act implementation, follows a more standardized dossier and decision process; the COU is published, and the biomarker becomes usable by any sponsor within those exact boundaries without needing to re-prove validity
The COU is the single most consequential document in the package: a biomarker qualified as "stride velocity as a secondary endpoint in ambulatory Duchenne muscular dystrophy patients aged 5–15" cannot be used, without a COU amendment, as a primary endpoint, in non-ambulatory patients, or in a different neuromuscular indication — even though the underlying sensor and algorithm are identical.
EMA operates a parallel, though procedurally distinct, qualification route via the Committee for Medicinal Products for Human Use (CHMP), typically routed through the Scientific Advice Working Party (SAWP) or the EMA Innovation Task Force for early, informal dialogue:
• Qualification advice: an early, non-binding scientific advice interaction, useful for sponsors still designing their validation program • Qualification opinion: a formal, published CHMP opinion on a completed evidence package, analogous to FDA's full qualification decision • Both routes increasingly encourage parallel FDA/EMA scientific advice for digital biomarker programs, given the high fixed cost of generating V3 evidence — the SV95C endpoint in Duchenne muscular dystrophy is a rare example of sequential success in both agencies (EMA qualification opinion 2019, FDA qualification 2022), illustrating both the feasibility and the multi-year timeline of dual qualification
International standards bodies increasingly shape the underlying technical requirements referenced by both agencies: ISO/IEEE 11073 personal health device communication standards, IEEE 2621 for pulse-oximeter equivalence testing, and CDISC's Digital Health Technology data standards (extending SDTM/CDASH domains to accommodate high-frequency sensor-derived variables) are now commonly cited within Qualification Plans as the technical backbone underlying the statistical validation claims.
As of 2024, the number of fully qualified digital biomarkers remains in the single digits worldwide — reflecting both the multi-year, multi-million-dollar cost of generating a complete V3 package and the field's youth. Most sponsors instead pursue a "fit-for-purpose" argument within a single drug's submission (a lower evidentiary bar, valid only for that program) rather than full, reusable biomarker qualification — a rational choice given qualification's cost is only recouped when multiple sponsors share the same COU.
The final stage closes the loop: a qualified (or robustly fit-for-purpose) digital biomarker is pre-specified as an efficacy endpoint in a registrational trial's Statistical Analysis Plan, its estimand fully defined per ICH E9(R1), and its data flow engineered end-to-end from device to eCRF under CDISC digital-health data standards — turning a research construct into evidence a regulator will use to approve or reject a therapy.
ICH E9(R1) (2019) requires sponsors to pre-specify an estimand — a precise description of the treatment effect being estimated — before analyzing any trial endpoint, and digital biomarkers make several intercurrent-event questions unavoidable that were previously implicit in sparse clinic-visit endpoints:
Device non-adherence as an intercurrent event: if a participant stops wearing the device (whether from device fatigue, adverse event, or disease progression), does the estimand treat subsequent missing digital data as part of the outcome (treatment policy strategy — often most clinically relevant, since non-adherence itself may reflect treatment failure), or does it estimate the effect as if adherence had been maintained (hypothetical strategy, requiring model-based imputation)?
Rescue medication / concomitant device use: for cardiopulmonary or metabolic digital endpoints, use of a rescue inhaler or insulin pump adjustment during the monitoring window must be handled as a defined intercurrent event, not silently averaged into the primary analysis.
Multiplicity and sensitivity analyses: because digital endpoints generate high-frequency, high-dimensional data (a single participant may contribute tens of thousands of accelerometer epochs), pre-specification must define exactly which derived summary statistic (e.g., median daily stride velocity, 95th-percentile stride velocity, weekly step count) constitutes the primary analysis variable — post hoc selection among many plausible summary statistics is a well-recognized source of false-positive inflation flagged in FDA statistical reviews.
A digital endpoint used in a registrational submission requires an auditable, validated data pipeline from sensor to statistical dataset, analogous to bioanalytical method validation for a PK/PD biomarker:
1. Edge capture: raw sensor data captured on-device with cryptographic timestamping and device ID binding, per FDA's expectations for data provenance in remote data acquisition 2. Secure transmission: encrypted transfer to a validated cloud environment, typically under 21 CFR Part 11 electronic records controls 3. Algorithmic derivation: the frozen, version-controlled algorithm converts raw signal to the derived clinical measure; any algorithm change during the pivotal trial requires formal change control and, often, a bridging analysis demonstrating equivalence pre/post change 4. CDISC mapping: derived measures are mapped into standardized domains under the CDISC Digital Health Technology Implementation Guide, extending traditional SDTM/CDASH conventions (originally built for sparse clinic-visit data) to accommodate high-frequency, continuously sampled variables 5. Statistical analysis: locked datasets analyzed per the pre-specified SAP, with the digital endpoint's summary statistics reconciled against the qualification COU boundaries to confirm the trial population and use conditions match what was qualified
ICH E6(R3), the revised Good Clinical Practice guideline finalized to explicitly address decentralized trial elements (remote data capture, direct-to-participant devices, telemedicine visits), formally recognizes this data chain as within GCP scope — meaning the same inspection-readiness standards applied to lab bioanalytical data now extend to a participant's smartwatch.
Roche/Genentech's Floodlight platform for multiple sclerosis and Duchenne's SV95C endpoint both illustrate the arc of this pipeline in practice — years of V3 evidence generation preceding a single pivotal-trial deployment where the digital measure, not a clinic-based scale, ultimately appears in the product label's efficacy claims or supports a regulatory decision alongside traditional endpoints.