Clinical endpoint design & validation for FDA-authorized software — modeled on reSET / reSET-O (OUD, SUD) and EndeavorRx (pediatric ADHD)
Substance use disorder (SUD), opioid use disorder (OUD), and pediatric ADHD share a common bottleneck: evidence-based behavioral treatment — cognitive behavioral therapy, contingency management, attentional training — is chronically undersupplied relative to demand. Prescription digital therapeutics (PDTs) were conceived to deliver a standardized, dose-controlled behavioral intervention through a smartphone or tablet, subject to the same clinical-trial and FDA-clearance rigor as a drug or device, rather than the unregulated wellness-app market.
Outpatient SUD treatment relies heavily on Therapeutic Education System-style cognitive behavioral therapy (CBT) and contingency management (CM) — structured curricula proven in decades of NIDA-funded trials to reduce substance use. But delivering CBT/CM face-to-face at scale requires trained counselors, fixed clinic hours, and consistent fidelity to the manualized protocol — resources that are unevenly distributed and frequently rationed by insurance authorization limits.
Pediatric ADHD faces an analogous bottleneck on the diagnostic and non-pharmacologic side: stimulant medication is effective for many children but is not tolerated or desired by all families, and access to computerized attention-training protocols historically required specialized clinics.
Pear Therapeutics, founded in 2013, was the first company to pursue full randomized-controlled-trial validation and FDA marketing authorization for a mobile app treating a behavioral health condition — establishing that software could be regulated, prescribed, and reimbursed as a distinct medical product category rather than a consumer wellness tool.
The Digital Therapeutics Alliance and FDA use several criteria to distinguish a prescription digital therapeutic from a general wellness app:
• Clinical evidence requirement: PDTs must demonstrate a treatment effect on a pre-specified, clinically validated endpoint in an adequately powered randomized trial — not just usability or engagement metrics • Regulatory clearance: PDTs are regulated as Software as a Medical Device (SaMD) and require FDA marketing authorization (De Novo, 510(k), or PMA) before commercial distribution with a treatment claim • Prescription requirement: most PDTs (reSET, reSET-O, EndeavorRx) are Rx-only, dispensed via a unique access code from a licensed prescriber, not downloaded freely from an app store • Active ingredient is behavioral: the mechanism of action is a structured, dosed behavioral or cognitive-training curriculum (CBT modules, contingency-management reinforcement, adaptive attention-training game levels) delivered algorithmically • Postmarket obligations: PDTs carry labeling constraints (adjunct-to-treatment claims, not standalone replacement) and are subject to FDA adverse-event reporting like any cleared device
The category's first clearance — reSET in September 2017 — was significant precisely because no predicate device existed: the FDA had never authorized a mobile application to treat a disease via a De Novo pathway before. That decision created the regulatory template every subsequent PDT would follow.
Before any trial begins, a PDT sponsor must select an indication narrow enough to be testable in a randomized trial yet broad enough to be commercially meaningful:
• reSET: adjunct to outpatient treatment for patients 18+ with alcohol, cocaine, cannabis, and stimulant use disorder (explicitly not opioid use disorder — carved out for a dedicated OUD product) • reSET-O: adjunct to buprenorphine and outpatient contingency management for OUD, positioned to increase retention in treatment rather than replace medication-assisted treatment (MAT) • EndeavorRx (AKL-T01): treatment of attention function deficit in children 8–12 with primarily inattentive or combined-type ADHD, as monotherapy or alongside stimulant medication
Each indication statement directly foreshadows the primary endpoint that must be selected in the next stage: abstinence for reSET, retention-in-treatment for reSET-O, and an objective attention-performance measure for EndeavorRx.
A PDT pivotal trial must solve a problem drug trials do not face as acutely: how do you blind a patient to whether they are using an "active" app versus a placebo app, when both are visibly interactive software? Sponsors design matched sham or treatment-as-usual (TAU) comparator arms, and pre-register a single primary endpoint capable of surviving FDA statistical review — typically a biologically or behaviorally objective measure rather than a self-report scale alone.
Pivotal PDT trials use computer-generated randomization sequences, typically stratified by site and sometimes by baseline severity, with allocation concealed from enrolling clinicians until after consent and baseline assessment are complete:
• 1:1 allocation is standard for PDT pivotal trials (reSET, EndeavorRx); reSET-O used a similar balanced design • Stratified randomization by clinical site controls for between-site variance in usual-care quality and population characteristics • Block randomization within strata prevents late-trial imbalance in arm sizes as enrollment proceeds • Central, computerized randomization (rather than site-level envelopes) prevents selection bias by investigators who might otherwise preferentially enroll healthier patients into the arm they favor
Software interventions are difficult to blind because patients directly experience the intervention's content. Two design strategies dominate:
TAU-only control (reSET, reSET-O): the control arm receives standard outpatient counseling with no companion app at all. This is not blinded to the patient — participants know whether they received the app — but the primary endpoint (urine-confirmed abstinence, treatment retention) is an objective, hard-to-game measure that is far less susceptible to placebo/expectancy effects than a subjective symptom scale would be.
Sham/attention-matched control (EndeavorRx): the STARS-ADHD trial gave control-arm children a different tablet game, styled similarly and played on the same schedule, but lacking the adaptive algorithmic difficulty targeting sensorimotor and interference-processing circuits that constitutes EndeavorRx's purported active mechanism. This preserves participant and rater blinding to arm assignment far better than a no-treatment control, isolating the effect of the adaptive algorithm itself rather than generic screen-time engagement.
Outcome assessors (raters administering the TOVA, laboratory personnel reading urine toxicology) are kept blinded to arm assignment in both designs — assessor blinding is often more achievable, and statistically more important, than participant blinding for objective endpoints.
Because the mechanism of action is behavioral rather than pharmacologic, the choice of comparator is itself a scientific hypothesis: EndeavorRx's sham game was specifically engineered to control for "gamification" placebo effects, isolating whether the adaptive algorithm — not merely engaging screen time — drives the attention-performance signal.
FDA statistical review requires a single, pre-registered primary endpoint with a pre-specified analysis method, defined before any outcome data are unblinded:
• reSET: proportion of patients abstinent from their substance(s) of abuse during weeks 9–12 of the 12-week trial, with abstinence defined by a composite of self-report (timeline follow-back) confirmed by urine drug screening — a stricter bar than self-report alone, since urine toxicology cannot be socially desirability-biased • reSET-O: retention in treatment through week 12, a pragmatic endpoint reflecting that OUD treatment discontinuation is itself a major driver of relapse and overdose risk • EndeavorRx: change from baseline to end-of-treatment (4 weeks) in TOVA API (Test of Variables of Attention, Attention Performance Index) — a computerized, objective, criterion-referenced measure of sustained attention and impulsivity, chosen specifically because it does not rely on parent or clinician subjective rating
Secondary endpoints (ADHD-RS-5 parent/clinician ratings for EndeavorRx; treatment satisfaction and craving scores for reSET) are analyzed but explicitly not permitted to substitute for the primary endpoint in the regulatory decision — a hierarchy that prevents "endpoint shopping" after the data are seen.
Enrollment is where trial design meets operational reality: patients must be screened against eligibility criteria, randomized, and then retained long enough to generate an interpretable primary-endpoint measurement. For PDTs, "dose" is defined by app engagement — number of lessons completed, minutes of active gameplay — so trial conduct must monitor and report adherence alongside conventional retention metrics.
Pivotal PDT trials screen substantially more candidates than they enroll. Typical exclusion drivers include severe psychiatric comorbidity requiring inpatient care, inability to use a smartphone/tablet reliably, concurrent enrollment in another investigational treatment, and — for OUD/SUD trials — insufficiently severe or already-abstinent baseline status that would make an abstinence endpoint uninformative.
Once enrolled, participants proceed through baseline assessment, randomization, and an active-treatment window (12 weeks for reSET/reSET-O, 4 weeks of active game sessions plus follow-up assessment for EndeavorRx). Site-level attrition — patients lost to follow-up before the endpoint window — is tracked and reported per CONSORT guidelines, since differential dropout between arms can bias intention-to-treat estimates even when randomization was properly executed.
Unlike a pill with a fixed daily dose, PDT "dose" is behavioral engagement, captured passively by the app itself: lessons opened, exercises completed, in-game levels reached, minutes of active use per session. Trial protocols pre-specify a minimum engagement threshold for the per-protocol (completer) analysis population, distinct from the intention-to-treat (ITT) population that includes everyone regardless of how much they actually used the app:
• reSET: per-protocol analyses typically required completion of a majority of the 61 CBT/community-reinforcement lessons across the 12-week course • EndeavorRx: STARS-ADHD prescribed five 25-minute sessions per week for four weeks (≈20 sessions total); adherence was monitored via the device's built-in usage logging, and completion rate became a key operational metric reported alongside efficacy
Because engagement is optional and self-directed, PDT trials — like behavioral trials generally — face a structural risk that low real-world adherence outside the closely-monitored trial setting will erode the effect size seen in the pivotal study, a concern regulators explicitly weigh when reviewing postmarket study commitments.
The adherence-threshold slider on this simulation controls what fraction of enrolled participants are treated as having reached per-protocol engagement — mirroring how real PDT trials must report both ITT (conservative, includes non-adherers) and per-protocol (optimistic, only real users) efficacy estimates side by side.
| Product | Indication | Trial Design | Key Result |
|---|---|---|---|
| reSET (Pear Therapeutics) | Non-opioid SUD, adults 18+ | 12-week RCT, TAU vs. TAU+app, N=399, 10 outpatient sites | Abstinence wks 9–12: ~40.3% vs ~17.6% |
| reSET-O (Pear Therapeutics) | Opioid use disorder, adjunct to buprenorphine | 12-week RCT, N≈170, retention-in-treatment primary endpoint | Improved treatment retention vs. TAU |
| EndeavorRx / AKL-T01 (Akili) | Pediatric ADHD, ages 8–12 | STARS-ADHD RCT, N≈348, 4-week adaptive game vs. sham game | TOVA API improvement, p≈0.006 |
Once the trial completes and the database is locked, the pre-specified statistical analysis plan (SAP) is executed exactly as registered — no post-hoc endpoint switching. Both intention-to-treat (all randomized patients, conservative) and per-protocol (adherent completers, optimistic) analyses are reported, and continuous endpoints are converted into clinically interpretable "responder" categories for labeling and prescriber communication.
ITT analysis includes every randomized participant in the arm to which they were assigned, regardless of how much of the app they actually used or whether they dropped out — it answers "what effect does prescribing this PDT have, accounting for real-world non-adherence?" Missing endpoint data in ITT analyses is typically handled by treating missing/non-confirmed samples as non-abstinent (a conservative "missing = failure" imputation common in addiction trials), which biases the estimated effect size toward the null rather than inflating it.
Per-protocol analysis restricts to participants who met the pre-specified engagement threshold — it answers "what effect does the therapeutic content have when actually used as directed?" Per-protocol effect sizes are typically larger than ITT, and the gap between the two is itself informative: a large ITT-to-per-protocol gap signals that real-world effectiveness will depend heavily on engagement-support strategies (reminders, coaching calls, prescriber follow-up) layered around the software itself.
FDA reviewers weight the ITT result most heavily for the marketing-authorization decision, since it reflects expected real-world performance, while per-protocol and secondary analyses support the plausibility and mechanism of the observed effect.
A statistically significant between-arm difference in a continuous or proportion endpoint is necessary but not sufficient for a clinically meaningful labeling claim. Sponsors additionally define a "responder" — a binary clinical-improvement category — to communicate benefit in terms a prescriber and patient can act on:
• reSET: a responder is a participant who achieved confirmed abstinence during the full weeks 9–12 evaluation window, not merely a reduction in use days • EndeavorRx: responder analyses examined the proportion of children who showed clinically meaningful improvement in at least one objective measure of impairment (attention/inhibitory control), reported alongside — but not substituting for — the continuous TOVA API change score
Effect-size reporting (Cohen's d, odds ratio, or risk difference with 95% CI) is required alongside the p-value: reSET's ~22.7 percentage-point absolute risk difference in abstinence, for example, conveys a magnitude of benefit that a p-value alone cannot — critical for FDA benefit-risk determinations and for downstream payer value assessments.
A p-value answers "is this difference likely due to chance?" A responder rate and effect size answer "how many patients benefit, and by how much?" FDA De Novo review — and every subsequent payer coverage decision — depends on both being reported together, not the p-value alone.
Secondary and exploratory endpoints round out the clinical picture but cannot rescue a failed primary endpoint under FDA statistical conventions:
• reSET secondary measures included retention in treatment and reduction (not just cessation) in days of use • EndeavorRx's secondary endpoint, the ADHD-RS-5 parent/clinician-rated symptom scale, did not separate as cleanly from the sham-game control as the objective TOVA API primary endpoint did — an important nuance illustrating that an objective performance measure and a subjective symptom-rating scale can diverge, and that FDA authorization rested specifically on the pre-specified objective primary endpoint rather than on symptom-scale improvement • Multiplicity correction (adjusting significance thresholds when multiple secondary endpoints are tested) is applied to avoid false-positive secondary claims riding on a single true primary-endpoint signal
With no legally marketed predicate device to which a PDT could claim "substantial equivalence," the FDA authorized reSET, reSET-O, and EndeavorRx through the De Novo classification pathway — a route for novel, low-to-moderate-risk devices that simultaneously grants marketing authorization and creates a new regulatory classification (with special controls) that future, similar PDTs can then follow via the faster 510(k) route.
FDA device pathways branch on novelty and risk:
• 510(k) clearance requires a predicate device already on the market with the same intended use and technological characteristics — impossible for the first-in-category PDT, since nothing like it had been cleared before • PMA (premarket approval) is reserved for Class III, highest-risk devices requiring the most extensive clinical evidence — disproportionate to a low-risk behavioral software intervention with a well-characterized safety profile (essentially no physical risk, since the "dose" is app content, not a drug or invasive procedure) • De Novo classification threads the needle: the sponsor demonstrates the device is low-to-moderate risk despite lacking a predicate, FDA reviews the clinical evidence package, and if authorized, the device is granted its own new regulatory classification with special controls — after which it becomes the predicate that subsequent, similar devices can reference via 510(k)
This is precisely why reSET (2017) mattered beyond its own indication: it created the classification "computerized behavioral therapy device for substance use disorder," and reSET-O (2018) and later products in the space could leverage that precedent. EndeavorRx followed the same logic in 2020, establishing the classification for a game-based digital therapeutic for pediatric ADHD attention deficit.
De Novo authorization is not merely "the app passed a trial" — it is FDA creating an entirely new regulatory category, complete with special controls (labeling requirements, cybersecurity controls, human-factors validation, clinical performance requirements) that legally binds every future device in that classification.
Class II special controls attached to each PDT classification typically specify: required clinical performance testing (replicating the pivotal-trial endpoint methodology for any future 510(k) follow-on), cybersecurity and software validation documentation, human-factors/usability testing to ensure patients can operate the device as intended without misuse risk, and precise labeling boundaries.
Labeling is deliberately conservative: reSET and reSET-O are authorized as an adjunct to outpatient treatment — not a replacement for counseling, contingency management, or (for OUD) medication-assisted treatment with buprenorphine. EndeavorRx is authorized to improve attention function as measured by computer-based testing, explicitly not marketed as a cure or a substitute for stimulant medication where clinically indicated. These boundaries mirror exactly the population and comparator design decisions made back in Stage 2 — the label cannot claim more than the pivotal trial's design was capable of demonstrating.
Regulatory clearance is necessary but not sufficient for a PDT to reach patients. Post-clearance, sponsors typically pursue: real-world evidence (RWE) studies tracking engagement and outcomes outside the closely monitored trial setting; payer coverage and reimbursement pathways (CPT codes, formulary-like PDT benefit categories), which lagged clearance by years and remained inconsistent across insurers; and continued adverse-event and complaint monitoring required of any Class II device.
The commercial history of this category is a cautionary complement to its scientific success: Pear Therapeutics, despite two landmark De Novo clearances, filed for bankruptcy in 2023, undone less by clinical evidence than by reimbursement uncertainty, high patient-acquisition costs, and a payer landscape unprepared to price software as a therapeutic. EndeavorRx, by contrast, has continued to accumulate post-market and real-world data under Akili, illustrating that FDA clearance opens the door to the market but does not guarantee the durable reimbursement and prescriber-adoption pathway that ultimately determines whether validated clinical evidence translates into patients actually receiving — and staying engaged with — the therapy.