Bootstrap Resampling: Statistical Inference Without Assumptions

Sampling with replacement

Classical statistics rests on assumptions: a normal distribution, a large-enough sample for the central limit theorem, a formula for the standard error of your specific statistic. Bootstrap resampling sidesteps all of that. Given n observations, draw a bootstrap sample of n values uniformly at random, with replacement, from the original data — any observation can appear zero, one, or many times. Compute the statistic T* on this resample, repeat B times (typically 1,000-10,000), and the spread of the resulting T* values directly approximates the sampling variability of T.

1. Compute T_obs from the original n observations
2. Draw x*1 .. x*n by sampling n values with replacement
3. Compute T* from the resample
4. Repeat steps 2-3 B times → T*1 .. T*B
5. Summarise the T* distribution → CI, standard error

The empirical distribution — probability 1/n on each observed value — is the best nonparametric estimate of the true population distribution, so resampling from it mimics resampling from the truth. On average, each resample contains only about 1 − e⁻¹ ≈ 63.2% of the unique original observations; the unselected ~36.8% form the basis of the out-of-bag error estimate in random forests.

Building a confidence interval

The simplest approach is the percentile method: sort the B bootstrap statistics and take the 2.5th and 97.5th percentiles as a 95% interval. The bias-corrected and accelerated (BCa) interval applies two corrections — a bias-correction factor for skew and an acceleration factor estimated via jackknife — giving better coverage for skewed statistics. The bootstrap-t (studentised) method standardises each resample by its own bootstrap standard error and is the most accurate but most computationally intensive, requiring a nested bootstrap.

When to use it, and its limits

Bootstrap shines where classical theory is absent or unreliable — a correlation coefficient's confidence interval, a trimmed mean, Spearman's rank correlation, an ROC-curve area. Classical methods still win when their assumptions genuinely hold, giving exact intervals with less computation. Bootstrap cannot fix a biased estimator or a non-representative sample — it estimates sampling variability, nothing more. The standard bootstrap also assumes independent, identically distributed data; for time series or clustered data (pupils within schools, patients within hospitals), a block bootstrap or cluster bootstrap resamples whole blocks or clusters to preserve the correlation structure.

Real-world applications

Random forests use the out-of-bag bootstrap estimate for generalisation error without a separate test set. Clinical trials use bootstrap intervals for skewed survival-time comparisons. Value-at-Risk and Expected Shortfall in finance are computed via historical simulation, itself a form of bootstrap. Phylogenetic tree reliability is reported as bootstrap support values, and national statistical agencies use closely related replicate-weight methods to handle complex stratified survey designs.

Frequently asked questions

Try it live

Everything above runs in your browser — open Bootstrap Resampling and watch the histogram of bootstrap statistics build up as you resample, extracting confidence intervals with the percentile method live.