The privacy/functionality tradeoff behind every health app: process where the data is born, or ship it to a server
Every health wearable and phone app is, underneath the marketing, a continuous sensor-data pipeline. A modern smartwatch samples heart rate, PPG waveform, SpO2, accelerometer motion and sometimes ECG dozens of times per second. The instant that raw signal exists, the software architecture makes a decision with enormous privacy consequences: run inference where the data was born, or ship the raw stream to a server first.
HIPAA only binds "covered entities" — healthcare providers, insurers, clearinghouses — and their contracted "business associates." A period-tracking app, a sleep app, or a direct-to-consumer wearable that sells straight to you is almost never a covered entity, so the health-specific data it collects is not protected by HIPAA at all.
This gap is exactly what the FTC's Health Breach Notification Rule exists to close. Originally issued in 2009 and rarely enforced, the rule was substantially expanded in April 2024 to explicitly cover health apps, wearables, and any technology that "draws information from multiple sources" — requiring breach notification to users and the FTC even for companies with zero HIPAA obligations.
In March 2023 the FTC fined BetterHelp $7.8 million after it shared visitors' health questionnaire answers, email addresses and IP addresses with Facebook, Snapchat, Pinterest and Criteo for ad targeting — despite its privacy policy promising that health information would stay private. It was the FTC's first-ever monetary settlement over misuse of health data.
At capture time, raw sensor data can take one of two paths. Path A: a small on-device model (TensorFlow Lite, Core ML, or a custom NPU kernel) consumes the raw waveform and outputs only a derived label — "irregular rhythm detected," a step count, a mood score — and the raw biosignal never leaves RAM. Path B: the raw stream is compressed, timestamped, and uploaded to a cloud endpoint, where a larger server-side model performs the same inference remotely.
Both paths can produce an identical user-facing result. The difference is entirely about what exists, in raw form, outside the device — and therefore what a breach, subpoena, or misconfigured server can ever expose.
The EU treats this as settled law: GDPR Article 9 classifies health data as a "special category" requiring explicit consent or a narrow statutory exemption, with fines up to 4% of global revenue or €20M. The US has no equivalent federal statute — instead a patchwork of HIPAA, the 2008 Genetic Information Nondiscrimination Act (GINA, which bars genetic discrimination in employment and health insurance but not life or disability insurance, and does not cover direct-to-consumer testing companies at all), and newer state laws like Washington's My Health My Data Act (2023), the first US law to broadly regulate consumer health data with a private right of action.
Every additional hop that raw data takes — device → app server → cloud storage → analytics vendor → ad partner — multiplies the number of jurisdictions, contracts, and promises that all have to hold simultaneously for the data to stay private.
Federated learning, formalized by McMahan et al. at Google in 2016-17 as "Federated Averaging" (FedAvg), inverts the traditional ML pipeline: instead of moving data to a central model, it moves a copy of the model to the data. Millions of phones each train briefly on their own local examples, and only the resulting mathematical updates — never the underlying keystrokes, waveforms or health entries — are sent anywhere.
A coordinating server broadcasts the current global model to a cohort of eligible devices — typically only phones that are idle, charging, and on unmetered Wi-Fi, exactly Google's original Gboard constraint. Each device trains a handful of local epochs purely on its own on-device data, which never leaves storage. Devices then upload only the resulting weight deltas (gradients), which the server averages — weighted by each device's local sample count — into an improved global model. The round repeats, often thousands of times, and the global model steadily improves without any raw example ever crossing the network.
Gboard's next-word prediction, one of the largest production deployments of this technique, uploads model updates on the order of ~100 KB — versus the raw text corpus that a centralized approach would otherwise need to collect from every keystroke.
Federated learning alone hides raw examples but not necessarily every statistical trace of them, so production systems layer differential privacy (DP) on top. Before any gradient or aggregate leaves the device, calibrated noise is injected — the magnitude controlled by a privacy budget, epsilon (ε). A smaller ε adds more noise and yields a stronger, more provable privacy guarantee at the cost of statistical accuracy; a larger ε leaks a closer approximation of the true value.
Apple's public differential-privacy deployment assigns per-feature daily epsilon budgets — roughly 2 for QuickType suggestions, around 4 for emoji suggestions, capped near 16 across all features combined per day — so that even Apple's own servers only ever see a noisy population statistic, never an individual user's true value. At vastly larger scale, the US Census Bureau adopted a similar formal-DP framework for the 2020 Census with a total cumulative privacy-loss budget of ε ≈ 19.61 across all published tables — a controversial tradeoff among demographers who argued it degraded small-area accuracy.
Apple's Health app and Screen Time run virtually all analysis on-device via the Secure Enclave and Neural Engine. Even when Apple aggregates trends across users — like typical headphone listening volume — it does so through local differential privacy, so its servers never receive any single user's real reading, only a noisy aggregate statistic.
Federated learning is not automatically private. Gradients are a lossy but non-zero function of the training data, and "Deep Leakage from Gradients" (Zhu et al., 2019) demonstrated that raw gradients can, under some conditions, be partially inverted to reconstruct approximations of the original training examples. Membership-inference attacks can also determine whether a specific record was used in training at all.
This is why production systems pair two independent mechanisms rather than relying on either alone: secure aggregation (a cryptographic protocol where the server can only ever decrypt the *sum* of many users' updates, never any individual contribution) together with differential-privacy noise applied before upload. Neither mechanism alone is sufficient; together they make both gradient inversion and membership inference computationally and statistically impractical.
Choosing cloud processing means raw biosignal is compressed and pushed over the network to a server. TLS 1.3 encrypts that pipe against passive eavesdropping — but encryption protects the wire, not the endpoints, and the moment data is decrypted server-side it becomes visible to every process, employee, and integrated third-party SDK that touches it.
TLS prevents a passive network observer from reading payload contents, but two exposures survive it. First, traffic analysis: packet size, timing, and destination remain visible even when content is encrypted, and can often reveal what type of event just occurred (a symptom check vs. a routine sync) without decrypting anything. Second, and larger: the instant data is decrypted at the server, it is exposed to every internal process and every third-party integration with access to that pipeline.
A 2022 audit of mobile health apps found the average app embeds roughly 14 third-party SDKs — analytics, crash reporting, ad networks — many of which receive a copy of transmitted events regardless of what the app developer's own privacy policy promises the end user.
Compromising one phone exposes one person's data. Compromising one cloud database exposes everyone's. In October 2023, attackers used credential stuffing — reusing passwords leaked from unrelated breaches — to log directly into roughly 14,000 23andMe accounts. Because 23andMe's "DNA Relatives" feature links accounts to genetic relatives inside one centralized, cross-referenceable database, the attackers pivoted from those 14,000 logins to expose ancestry, some health-predisposition, and family-tree data belonging to approximately 6.9 million connected profiles.
The 23andMe breach turned 14,000 compromised passwords into 6.9 million exposed genetic profiles — roughly a 500× amplification, made possible entirely because every relative's data lived in one centralized, queryable cloud database rather than distributed across individual devices.
Centralizing raw records doesn't just create a bigger breach target — it also makes bulk legal access trivial at scale. GEDmatch, a genetic-genealogy database built from consumer DNA-test uploads, opened its roughly one-million-profile database to law enforcement in 2018 during the Golden State Killer investigation, without most users having anticipated that use when they uploaded their raw genetic data (the platform later moved to opt-in consent after public backlash).
Once data sits in one queryable cloud store, a single warrant, subpoena, or breach can compel or expose the entire corpus at once — an outcome that is structurally impossible under an on-device-only architecture, where a comparable legal request would have to target individual devices one at a time.
Placed side by side, the two architectures have structurally different blast radii. An on-device breach requires physically compromising one phone and exposes one user. A cloud breach requires compromising one perimeter and can expose an entire population at once — and three of the highest-profile consumer health-privacy failures of the last five years all trace back to exactly that asymmetry.
The math of centralization is unforgiving. If a health platform stores raw data for 10 million users in one cloud database, a single successful intrusion — via credential stuffing, an unpatched server, or a leaked API key — can expose all 10 million records in one event. An architecture that keeps raw data on-device by design caps the maximum single-incident exposure at whatever a targeted, individualized attack on one device can obtain: one person's data, requiring one attack per victim rather than one attack per population.
Three cases spanning three different categories of sensitive health data all show the same centralization-driven pattern:
• Reproductive health — Flo Health, one of the most-downloaded period and ovulation trackers (reporting several hundred million cumulative downloads worldwide), was found by the FTC in 2021 to have shared users' health information — including whether a user indicated she was pregnant — with Facebook and Google's analytics platforms, despite promising in its privacy policy to keep such data private.
• Mental health — BetterHelp's $7.8 million FTC settlement (2023) covered years of sharing intake-questionnaire answers, email addresses and IP addresses with Facebook, Snapchat, Pinterest and Criteo for ad targeting.
• Genetic data — 23andMe's 2023 breach exposed ancestry and health-predisposition data for 6.9 million people, disclosed to the public in December 2023, sparking dozens of lawsuits and a since-finalized $30 million class-action settlement.
| Product | Indication | Trial Design | Key Result |
|---|---|---|---|
| On-device local store | Single device only | Encrypted at rest via Secure Enclave / Android Keystore; requires physical device compromise | Blast radius = 1 user |
| Data-in-transit (TLS) | Network path device → server | Payload encrypted, but packet size/timing/destination remain observable (traffic analysis) | Confidentiality strong; metadata leak possible |
| Cloud server / database | Entire user population | Centralized store; one credential-stuffing or injection attack exposes millions of records at once | 23andMe: 14k logins → 6.9M profiles exposed |
| Third-party SDK / ad partner | Whoever the SDK forwards data to | Analytics/ad SDKs embedded in the app receive event copies, sometimes health-labeled | Flo & BetterHelp: shared despite privacy promises |
Given everything above, the privacy-superior architecture is obvious on paper — yet the majority of consumer health apps still route raw data to the cloud by default. That is not an accident or an oversight; it is the rational outcome of three converging forces, and a regulatory landscape that, until very recently, made the cheaper choice also the legally safer one.
Compute and battery limits: running a large diagnostic model continuously drains battery and many low/mid-range phones still lack a capable NPU, so cloud inference is often simply faster and cheaper to ship.
Population-level model improvement: a centralized model can retrain on aggregated data from millions of users within days, catching rare conditions and edge cases that any single device's local data never contains; federated learning narrows this gap but, as shown earlier, typically still trails centralized training by 1-5 percentage points, and offers nothing for a brand-new user with zero local history.
Business model: raw data is a monetizable asset — ad targeting, "aggregate insight" products sold to insurers, employers and pharma, and outright resale through data brokers. An architecture that keeps raw data on-device forecloses that revenue stream entirely, which is exactly why free, ad-supported health apps skew heavily cloud-based.
Because most consumer wellness apps fall outside HIPAA entirely, the legal cost of choosing cloud-plus-sharing has historically been far lower than the engineering cost of building genuine on-device or federated pipelines. That calculus is only now shifting: Washington's My Health My Data Act (2023) — passed specifically in response to Flo- and BetterHelp-style cases — grants a private right of action against any company, not just HIPAA-covered ones, that mishandles "consumer health data." The FTC's 2024-expanded Health Breach Notification Rule and GDPR's Article 9 penalties are pushing in the same direction: making the cloud-default choice measurably more expensive.
The realistic architecture emerging across serious health-privacy products is hybrid, not absolutist: local inference for anything time-critical or highly sensitive (arrhythmia detection, mental-health text analysis), federated learning plus differential-privacy aggregation for anything that genuinely needs population-level learning, and raw cloud storage reserved only for data with explicit, narrowly-scoped consent backed by a real HIPAA/GDPR compliance program — not a growth-team default.
Apple's on-device Neural Engine performs on-watch ECG analysis in under a second with zero raw waveform ever leaving the device — a feature cleared by the FDA through the De Novo pathway in 2018. A typical cloud-based symptom-checker, by contrast, spends 100–500 ms on network latency alone before inference even begins, and every one of those raw payloads becomes a permanent, subpoenable, breachable record.