1. Large-sample experiment: simulate about 200,000 patients, compare measured recall / precision with the closed form.
2. Imbalance sweep: how recall, precision and PR-AUC change with 1 : N, plain versus SMOTE.
The science
Synthetic cohort. Each patient has two biomarkers x = (A, B). Healthy patients: x ~ N(μ0, Σ); diseased: x ~ N(μ1, Σ), common covariance Σ = [[1, ρ],[ρ, 1]]. The shift Δ = μ1 − μ0 has Mahalanobis length D (the separation slider): D² = Δᵀ Σ⁻¹ Δ. The prevalence is π1 = 1/(1+N) for an imbalance 1 : N.
SMOTE (Chawla et al., 2002): for a random diseased patient xi pick one of its k nearest diseased neighbours xnn (Euclidean) and a number u ~ U(0,1):
x_new = x_i + u (x_nn - x_i)
Repeat until diseased : healthy reaches the target. Every synthetic point lies exactly on the segment between two real patients (drawn as thin lines). It adds no new information about the disease; it only fills the space between the points you already have.
Logistic regression. p(y=1|x) = σ(w·x + b), fitted by Newton / IRLS on the mean log-loss plus (λ/2)|w|²:
theta ← theta - H^-1 g, g = (1/N) X^T (p - y) + λ (w,0), H = (1/N) X^T diag(p(1-p)) X + λ diag(1,1,0)
Why imbalance hurts. For this Gaussian model the ideal classifier is logistic with w = Σ⁻¹Δ and b = −½(μ0+μ1)·w + ln(π1/π0) (linear discriminant analysis). With prior 1 : N the intercept contains −ln N, so at p = 0.5 a patient is flagged only when the likelihood ratio exceeds N. That is optimal for accuracy but misses many diseased patients. Oversampling to a 1 : 1 training set (or class weights) moves the intercept by about ln N: recall rises, precision falls, because the test set still has the true prevalence. The ranking of patients (ROC-AUC, PR-AUC) hardly changes: SMOTE mostly acts like a different threshold.
Metrics. precision = TP/(TP+FP), recall = TP/(TP+FN), F1 = 2PR/(P+R). Precision from rates: precision = TPRπ / (TPRπ + FPR(1−π)). PR-AUC is the average precision AP = Σ(Rn − Rn−1) Pn (a random model scores the prevalence π). ROC-AUC = P(score of a diseased patient > score of a healthy one).
Closed form used in the experiment. With the LDA score s, s|healthy ~ N(−D²/2 + ln(π1/π0), D²) and s|diseased ~ N(+D²/2 + ln(π1/π0), D²), so recall = Φ((D²/2 + ln(π1/π0) − τ)/D) with τ = logit of the threshold.
Honest simplifications. Synthetic Gaussian patients, two biomarkers, common covariance: real data are messier and SMOTE behaves differently with categorical or high-dimensional features. The test set comes from the same generator at the true prevalence. The cohort is small, so numbers move with the seed (use «New cohort» and the sweep). SMOTE is applied once to the whole training set; in real work it must be applied inside each cross-validation fold, on training data only, otherwise information leaks into validation. The best-F1 threshold shown for the plain model is tuned on the test set and therefore optimistic. Literature values (Chawla 2002; Saito & Rehmsmeier 2015 on precision-recall versus ROC) are cited from memory.
Educational only. Not medical advice. No real patients, no real disease, and no clinical conclusion can be drawn from this model.