Statistics — Probability, Inference & Data Analysis

Descriptive Statistics

Measures of central tendency : mean (μ = Σxᵢ/n), median (middle value, robust to outliers), mode (most frequent). Measures of spread : range, variance (σ² = Σ(xᵢ – μ)²/n), standard deviation (σ), interquartile range (IQR = Q3 – Q1). Shape : skewness (asymmetry), kurtosis (tail heaviness). Visualization : histograms, box plots, scatter plots, Q-Q plots. Empirical rule for normal distributions: 68% within ±1σ, 95% within ±2σ, 99.7% within ±3σ.

Probability Theory

Probability P(A) ∈ [0,1] measures the likelihood of event A. Axioms (Kolmogorov): P(Ω) = 1, P(A) ≥ 0, P(A ∪ B) = P(A) + P(B) for disjoint events. Conditional probability : P(A|B) = P(A∩B)/P(B). Bayes’ theorem : P(A|B) = P(B|A)P(A)/P(B). Independence : P(A∩B) = P(A)P(B). Key distributions : Bernoulli (single trial), Binomial (n trials, P(X=k) = C(n,k)pᵏ(1–p)ⁿ⁻ᵏ), Poisson (λ events/interval), Normal (N(μ,σ²), bell curve, CLT), Exponential (time between events), t-distribution (small samples, heavier tails than normal), chi-squared (χ², goodness of fit).

Hypothesis Testing

Framework: (1) State null hypothesis H₀ and alternative H₁. (2) Choose significance level α (typically 0.05). (3) Compute test statistic. (4) Compare p-value to α. If p < α, reject H₀. Types of error : Type I (false positive, probability = α), Type II (false negative, probability = β). Power = 1 – β. Common tests : z-test (known σ, large n), t-test (unknown σ: one-sample, two-sample, paired), χ² test (categorical data, independence, goodness of fit), ANOVA (compare 3+ group means, F-statistic), Mann-Whitney U (non-parametric). Multiple comparisons : Bonferroni correction, FDR (Benjamini-Hochberg). Effect size (Cohen’s d, r²) supplements p-values.

Regression Analysis

Simple linear regression : y = β₀ + β₁x + ε. Least squares: minimize Σ(yᵢ – ŷᵢ)². R² (coefficient of determination): proportion of variance explained. Multiple regression : y = β₀ + β₁x₁ + ... + βₙxₙ + ε. Assumptions: linearity, independence, homoscedasticity, normality of residuals, no multicollinearity (check VIF). Logistic regression : for binary classification, models log-odds: ln(p/(1–p)) = β₀ + βᵀx. Regularized regression : Ridge (L2), Lasso (L1, feature selection), Elastic Net (both). Generalized linear models (GLM): link function + exponential family distribution (Poisson regression for counts).

Bayesian Statistics

Bayesian inference uses Bayes’ theorem: P(θ|data) ∝ P(data|θ) · P(θ). Prior P(θ): beliefs before data. Likelihood P(data|θ): data given parameters. Posterior P(θ|data): updated beliefs. Conjugate priors (Beta-Binomial, Normal-Normal) give closed-form posteriors. For complex models: MCMC (Markov Chain Monte Carlo — Metropolis-Hastings, Gibbs sampling, NUTS/HMC) samples from the posterior. Tools: Stan, PyMC, Turing.jl. Bayesian advantages : principled uncertainty quantification (credible intervals), natural regularization (priors), sequential updating, handling small data. Bayesian vs. frequentist : parameters are random variables (Bayesian) vs. fixed unknowns (frequentist).

📚 Data science

All Data science articles

Home

Physics

Chemistry

Biology

Mathematics

Computer Science

🌐 Languages

🇺🇦 Українська

🇬🇧 English (current)

❓ Frequently Asked Questions

What is a p-value?

A p-value is the probability of observing results at least as extreme as the data, assuming the null hypothesis H₀ is true. If p < α (typically 0.05), we reject H₀. A p-value does NOT indicate the probability that H₀ is true, the magnitude of an effect, or practical significance. Always report effect sizes alongside p-values.

What is the Central Limit Theorem?

The CLT states that the sampling distribution of the sample mean approaches a normal distribution as sample size n increases, regardless of the population’s distribution (given finite variance). For n ≥ 30, X̄ ~ N(μ, σ²/n) approximately. This justifies using z-tests and confidence intervals even when the population isn’t normally distributed.

What is the difference between correlation and causation?

Correlation (r) measures the linear association between two variables (-1 to +1). Causation means one variable directly influences another. Correlation does NOT imply causation: confounding variables, reverse causation, and coincidence can produce misleading correlations. Establishing causation requires controlled experiments (RCTs) or rigorous causal inference methods (instrumental variables, regression discontinuity).

When should I use Bayesian vs frequentist statistics?

Bayesian: when you have prior information, want full posterior distributions, handle small samples, or need sequential updating. Frequentist: when you need widely accepted methods (peer review), have large samples, want simpler computation, or regulatory requirements demand it (FDA clinical trials). In practice, many data scientists use both approaches complementarily.

What is regression to the mean?

Extreme observations (very high or low) tend to be followed by less extreme observations, simply due to random variation. This is not a causal effect but a statistical artifact. Example: students scoring very high on Test 1 tend to score lower on Test 2 (not because they got worse, but because extreme scores include more luck). Ignoring this leads to wrong causal conclusions.

Try it live

Everything above runs in your browser — open Dimensionality Reduction: PCA, t-SNE & UMAP and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.