English:Mathematical Statistics

Mathematical Statistics
Introduction
Mathematical statistics develops the mathematical foundations of statistical reasoning. It uses probability theory, analysis, linear algebra, optimization, and computation to turn data into statements about unknown features of a population or data-generating process. In this aiMOOC, you will move from probability models to estimators, sampling distributions, confidence intervals, hypothesis tests, asymptotic theory, resampling, and Bayesian ideas.
The central question is not merely "What number did the sample produce?" but "What can you justify about an unknown quantity when your evidence is random?" Mathematical statistics makes that question precise by specifying a model, identifying a target parameter or estimand, choosing a statistical procedure, and quantifying the procedure's uncertainty and long-run performance.

The normal density is a useful reference model throughout statistical theory, but mathematical statistics is not limited to normal data. Discrete models, heavy-tailed models, exponential families, nonparametric methods, and simulation-based procedures all belong to the subject.
The MIT OpenCourseWare lecture above introduces the role of mathematical theory in understanding statistical procedures.
Intended Learners and Prerequisites
This course is designed for university students who have completed introductory calculus and probability. Familiarity with derivatives, integrals, sums, limits, basic matrix algebra, and common probability distributions will help you work through the derivations. Some sections also use the language of convergence of random variables.
By the end of the course, you should be able to:
- Statistical model: Translate a data problem into a probabilistic model with clearly identified assumptions.
- Point estimation: Construct estimators and compare bias, variance, mean squared error, consistency, sufficiency, and efficiency.
- Likelihood function: Derive and analyze likelihoods, score functions, Fisher information, and maximum likelihood estimators.
- Sampling distribution: Derive or approximate the distribution of a statistic under repeated sampling.
- Confidence interval: Construct and interpret interval estimators using pivots, asymptotics, or resampling.
- Statistical hypothesis testing: Formulate null and alternative hypotheses and analyze Type I error, Type II error, power, and p-values.
- Asymptotic theory: Apply the law of large numbers, central limit theorem, Slutsky's theorem, and the delta method.
- Bayesian inference: Contrast frequentist procedures with posterior inference and decision-theoretic reasoning.
Probability Models and Statistical Inference
A statistical model is a family of probability distributions indexed by an unknown parameter. Write the model as , where is the parameter space. If the observations have density or probability mass function , then the unknown determines which member of the model generated the data.
A parameter is a fixed but unknown feature of the model, such as a population mean , variance , event probability , or regression coefficient. A statistic is any function of the observed sample that does not depend on unknown parameters. An estimator is a statistic used to estimate a parameter, while an estimate is the numerical value obtained after the data have been observed.
Random Samples and Independence
A standard mathematical idealization is an independent and identically distributed sample,
Independence makes the joint density factorize, while identical distribution means that the same model applies to each observation. These assumptions are powerful, but you should not adopt them automatically. Time series, clustered samples, repeated measurements, spatial data, and adaptive experiments often require dependence structures.
For an iid sample, the joint density is
This simple product is the starting point for likelihood-based inference.
Distribution Functions, Expectation, and Variability
A distribution can be summarized by its cumulative distribution function . When densities exist, . Expectations such as and variances such as describe model properties, while sample means and sample variances are random statistics whose distributions depend on the sampling process.
For iid data with finite variance,
The factor explains why averages become more stable as sample size grows.
Sampling Distributions and Large-Sample Theory
A sampling distribution is the probability distribution of a statistic over hypothetical repetitions of the sampling process. It is the bridge between probability theory and statistical inference. When you report a standard error, confidence interval, or p-value, you are using information about a sampling distribution, either exactly or approximately.
Law of Large Numbers
Under standard conditions, the law of large numbers states that the sample mean approaches the population mean:
This is a statement of consistency for the sample mean. It says that large samples increasingly concentrate around the target, not that any finite sample must be close to the truth.
Central Limit Theorem
If are iid with mean and finite positive variance , then the classical central limit theorem gives
This result explains why normal approximations occur throughout statistics even when the individual observations are not normally distributed.

The image illustrates how sums of independent dice become increasingly bell-shaped after centering and rescaling.
The central limit theorem is an asymptotic result. Its practical accuracy depends on the underlying distribution, the statistic, and the sample size; highly skewed or heavy-tailed data can require much larger samples than nearly symmetric light-tailed data.
Slutsky's Theorem and the Delta Method
Slutsky's theorem allows consistent estimators to replace unknown constants in many limiting distributions. For example, if , then
The delta method transfers asymptotic normality through a differentiable function. If
and is differentiable at , then
This is useful for transformed parameters such as log odds, risk ratios, or nonlinear functions of estimated coefficients.
Point Estimation
A point estimator compresses the sample into a proposed value for an unknown parameter. Good estimators are not judged only by one observed value; they are judged as random procedures.
Bias, Variance, and Mean Squared Error
For an estimator ,
Its mean squared error is
A small bias can sometimes be acceptable if it yields a much larger reduction in variance. This bias-variance tradeoff is a central idea in modern statistics.
Consistency and Efficiency
An estimator is consistent if . Consistency is asymptotic: it concerns what happens as the sample size tends to infinity. An estimator can be unbiased but noisy, biased but consistent, or both unbiased and consistent.
Within a class of estimators, an efficient estimator has relatively small variance or asymptotic variance. Under regularity conditions, the Cramér–Rao bound gives
for unbiased estimators, where is the Fisher information in the sample.
Sufficiency
A statistic is sufficient for if, once is known, the remaining sample contains no additional information about within the model. The Fisher-Neyman factorization criterion states that is sufficient when the joint density can be written as
Sufficiency is especially important in exponential families, where low-dimensional sufficient statistics often summarize all likelihood information about the parameter.
Likelihood and Maximum Likelihood Estimation
For observed data , the likelihood function views the joint density as a function of the parameter:
The data are fixed in this expression; varies. The log-likelihood
turns products into sums and is usually easier to differentiate and optimize.
The score function is . The Fisher information can be written, under regularity conditions, as
Maximum Likelihood Estimator
A maximum likelihood estimator is any parameter value that maximizes the likelihood:
When the maximum is in the interior of the parameter space and the likelihood is smooth, solving the score equation often identifies candidates, but boundary solutions and multiple local maxima must also be checked.

This Poisson example shows a likelihood curve whose maximum identifies the maximum likelihood estimate.
Maximum likelihood has attractive large-sample properties under regularity conditions: consistency, asymptotic normality, and asymptotic efficiency. Those claims are not automatic; non-identifiability, parameters on boundaries, model misspecification, or irregular likelihoods can invalidate standard results.
Method of Moments
The method of moments equates sample moments to theoretical moments. If , a one-parameter moment estimator may solve
For multi-parameter models, several moments can be matched. Method-of-moments estimators are often easy to compute and can provide useful starting values for likelihood optimization, although they need not be efficient.
Worked Example: Bernoulli Model
Suppose , where . Let . Then
The log-likelihood is
Differentiating gives
For , setting the score to zero gives
The estimator is unbiased, because , and
The Fisher information is
so attains the Cramér-Rao bound in this model. The central limit theorem also gives the approximation
for large samples.
This example demonstrates how likelihood, unbiasedness, information, efficiency, and asymptotic normality can describe the same estimator from different viewpoints.
Confidence Intervals
A confidence interval is a random interval constructed by a procedure with a specified long-run coverage probability. A 95% frequentist confidence procedure is designed so that, under repeated sampling from the assumed model, approximately 95% of intervals produced by the procedure contain the fixed true parameter.
It is important to distinguish this repeated-sampling claim from a probability statement about a fixed parameter after a particular frequentist interval has been observed.

This image uses 50% intervals to make repeated-sampling coverage visually easy to see: some random intervals cover the true mean and some do not.
Pivotal Quantities
A pivot is a function of the data and parameter whose sampling distribution does not depend on unknown parameters. Pivots can often be inverted to produce exact confidence intervals.
For normal data with unknown mean and variance,
Therefore an exact confidence interval for is

The Student t family has heavier tails than the standard normal distribution for small degrees of freedom and converges toward the standard normal as the degrees of freedom increase.
Confidence Interval for a Normal Variance
If the data are iid normal, then
This pivot yields an exact interval for :
The result depends strongly on the normality assumption.
Hypothesis Testing
A statistical test converts data into a decision rule about competing claims. A null hypothesis specifies the default model or parameter set, while an alternative describes departures of interest.
A Type I error occurs when you reject even though it is true. The significance level controls the probability of this error under the null. A Type II error occurs when you fail to reject for a parameter value in the alternative. The power of a test at a particular alternative is the probability that the test rejects when that alternative is true.
p-Values
A p-value is computed under the null hypothesis. It is the probability, assuming and the specified model are true, of obtaining a test statistic at least as incompatible with as the observed statistic, according to the chosen test. It is not the probability that is true and it is not, by itself, a measure of practical importance.
Statistical significance should be interpreted together with effect sizes, uncertainty intervals, study design, model assumptions, and the costs of false positive and false negative decisions.
Neyman-Pearson Lemma
For testing a simple null against a simple alternative , the Neyman–Pearson lemma states that a likelihood-ratio rejection region gives the most powerful test of a specified size. This result motivates many likelihood-based tests and clarifies why likelihood ratios compare how well two parameter values explain the observed data.
Likelihood-Ratio Tests
For a null parameter set , define
Small values of indicate that restricting the model to the null greatly reduces the best attainable likelihood. Under regularity conditions, Wilks' theorem gives an asymptotic chi-square distribution for

The image illustrates a right-tailed chi-square test in which the observed test statistic lies beyond the critical value.
Worked Example: Normal Mean and Variance
Suppose , with both parameters unknown. The maximum likelihood estimator of the mean is
The maximum likelihood estimator of the variance is
This variance estimator is biased downward. The unbiased sample variance uses denominator :
The pair is sufficient for . Normal theory also provides exact pivots:
These identities explain why Student t and chi-square distributions are fundamental to classical inference for normal samples.
Resampling and the Bootstrap
When exact sampling distributions are unavailable, you can approximate uncertainty computationally. The nonparametric bootstrap treats the empirical distribution of the observed sample as an estimate of the population distribution.
A basic bootstrap workflow is:
- Draw a sample of size with replacement from the observed data.
- Compute the statistic of interest on that resample.
- Repeat the resampling many times to approximate the statistic's sampling distribution.
- Use the resulting bootstrap distribution to estimate standard errors, bias, or confidence intervals.
Bootstrap methods are powerful but not automatic. They can fail or require modification for very small samples, extreme-value statistics, strong dependence, parameters on boundaries, or estimators with irregular asymptotic behavior. Time series and clustered data require resampling schemes that respect dependence.
Bayesian Inference and Decision Theory
In Bayesian inference, the parameter is assigned a prior distribution . The likelihood updates the prior through Bayes' theorem:
The resulting posterior distribution quantifies uncertainty about the parameter conditional on the model, prior, and observed data.

A conjugate example arises for Bernoulli data. If and the sample contains successes and failures, then

Bayesian credible intervals and frequentist confidence intervals answer different probability questions. A posterior credible interval assigns posterior probability to parameter values under the chosen prior and likelihood. A frequentist confidence interval is calibrated by repeated sampling under a statistical model.
Decision theory adds a loss function for action . A Bayesian decision minimizes posterior expected loss. Frequentist decision theory instead evaluates procedures through risk functions over repeated samples. These frameworks help make the consequences of statistical decisions explicit.
Model Assumptions, Diagnostics, and Responsible Inference
A mathematically elegant procedure can still give misleading conclusions if its assumptions do not match the data-generating process. Before interpreting inferential results, ask whether the sampling mechanism, independence assumptions, distributional form, parameter identifiability, and measurement process are plausible.
You should also separate statistical significance from scientific or practical importance. A tiny effect can be statistically detectable in a very large sample, while an important effect can remain uncertain in a small sample. Report effect sizes and uncertainty, inspect sensitivity to modeling choices, and document exclusions or transformations.
When data come from observational studies, statistical association alone does not establish causation. Causal claims require additional design or identification assumptions, such as randomization, valid instruments, or carefully justified causal models.
A Strategy for Solving Mathematical Statistics Problems
When you face a new inference problem, use the following reasoning sequence:
- Model specification: State the sample space, probability model, parameter space, and assumptions.
- Target parameter: Identify exactly what quantity you want to learn.
- Statistic construction: Choose an estimator, test statistic, pivot, or posterior quantity.
- Sampling behavior: Derive an exact distribution or justify an approximation.
- Uncertainty quantification: Compute a standard error, confidence interval, credible interval, or power function.
- Assumption check: Identify conditions under which the result is valid and discuss sensitivity.
- Interpretation: Translate the mathematical result back to the substantive question without overstating what the procedure proves.
Interactive Tasks
Quiz: Test Your Knowledge
Which statement best defines a statistic? (A function of the observed sample that contains no unknown parameter) (!A fixed unknown feature of the population) (!A probability assigned to a hypothesis after observing data) (!A rule that must always be unbiased)
What does consistency of an estimator mean? (The estimator converges in probability to the target parameter as sample size grows) (!The estimator is exactly correct for every sample) (!The estimator has zero variance for every sample size) (!The estimator has a normal distribution for every model)
What does maximum likelihood estimation choose? (A parameter value that maximizes the likelihood of the observed data) (!A sample that minimizes the population variance) (!A prior distribution with the largest mean) (!A test with zero Type I error)
Which quantity combines variance and squared bias? (Mean squared error) (!Fisher information) (!Statistical power) (!Likelihood ratio)
What does the central limit theorem typically approximate? (The standardized distribution of a sample mean for large samples) (!The exact distribution of every statistic for every sample size) (!The posterior distribution of any parameter) (!The probability that a null hypothesis is true)
What is the usual frequentist meaning of a confidence level? (The long run coverage rate of the interval procedure under the model) (!The probability that the fixed parameter changes between samples) (!The proportion of observations that lie inside every interval) (!The probability that the statistical model is correct)
What is a Type I error? (Rejecting a true null hypothesis) (!Failing to reject a false null hypothesis) (!Estimating a parameter with positive bias) (!Using a small sample in a normal model)
What does statistical power measure? (The probability of rejecting the null at a specified alternative) (!The probability that the null hypothesis is true) (!The numerical size of an estimated effect) (!The number of parameters in a model)
What does a sufficient statistic preserve within a model? (All sample information about the parameter represented in the likelihood) (!Every raw observation in its original order) (!Only the sample size and nothing else) (!The prior probability of each parameter value)
What is the defining operation in a basic nonparametric bootstrap? (Resampling observed data with replacement) (!Resampling observed data without replacement only once) (!Replacing the likelihood with a normal density) (!Discarding observations farther than two standard deviations away)
Memory Game
| Parameter | Fixed but unknown feature of a statistical model |
| Statistic | Function of observed data containing no unknown parameter |
| Bias | Difference between an estimator's expectation and its target |
| Consistency | Convergence of an estimator toward its target as sample size grows |
| Sufficiency | Property of retaining all model information about a parameter |
| Power | Probability that a test rejects the null at a specified alternative |
Drag and Drop
| Match the correct terms. | Topic |
|---|---|
| Maximum likelihood estimator | Parameter value that maximizes the observed-data likelihood |
| Central limit theorem | Large-sample normal approximation for standardized sums or means |
| Confidence interval | Random interval procedure calibrated by repeated-sampling coverage |
| Likelihood-ratio test | Test based on comparing restricted and unrestricted maximum likelihoods |
| Bootstrap distribution | Empirical distribution of a statistic across resampled datasets |
Crossword Puzzle
| Likelihood | What function treats the observed data as fixed and varies the model parameter? |
| Estimator | What statistic is designed to approximate an unknown parameter? |
| Variance | What quantity measures squared dispersion around an expectation? |
| Sufficiency | What property means a statistic retains all likelihood information about a parameter? |
| Bootstrap | What resampling method repeatedly samples observed data with replacement? |
| Power | What is the probability that a test rejects the null at a specified alternative? |
LearningApps
Cloze Text
Open-Ended Tasks
Easy
- Sampling Distribution Simulation: Simulate repeated samples from a skewed distribution, plot the sample means for several sample sizes, and explain how the shape and spread change.
- Estimator Comparison Poster: Create a one-page visual explanation of bias, variance, and mean squared error using two hypothetical estimators.
- Likelihood Sketch: Choose a simple Bernoulli or Poisson sample, compute its likelihood on a grid of parameter values, and mark the maximizing value.
- Confidence Interval Explanation: Write a 300-word explanation for a first-year university student that distinguishes a confidence interval from a probability statement about a fixed parameter.
Standard
- Bernoulli Inference Notebook: Build a small computational notebook that derives the Bernoulli maximum likelihood estimator, verifies its bias and variance by simulation, and compares theoretical and simulated standard errors.
- Central Limit Theorem Video: Produce a short video that demonstrates the central limit theorem for at least two non-normal parent distributions and discusses when convergence appears slow.
- Hypothesis Test Investigation: Design a simulation that estimates Type I error and power for a one-sample test across several effect sizes and sample sizes, then interpret the resulting power curves.
- Bootstrap Confidence Interval Study: Compare an asymptotic confidence interval with a bootstrap interval for a statistic such as a median, and describe conditions under which the two methods disagree.
Advanced
- Sufficiency Proof Project: Choose an exponential-family model, identify a sufficient statistic using the factorization criterion, and explain what information reduction the statistic achieves.
- Fisher Information Investigation: Derive the Fisher information and Cramér-Rao bound for a one-parameter model, then compare the bound with the variance of a natural estimator.
- Likelihood-Ratio Research Note: Derive a likelihood-ratio test for a simple parametric problem, justify its reference distribution, and evaluate its finite-sample behavior by simulation.
- Statistical Interview and Case Analysis: Interview a researcher who uses inferential statistics, reconstruct one of their analyses as a model-estimator-uncertainty pipeline, and critically assess assumptions, interpretation, and possible sources of error.
Learning Assessment
- Model-to-Inference Assessment: Given a new dataset description, specify a plausible statistical model, identify the parameter of interest, choose an estimator, and justify how you would quantify its uncertainty.
- Estimator Risk Assessment: Compare two estimators with different bias and variance patterns and determine which you would prefer under squared-error loss for several parameter values.
- Asymptotic Reasoning Assessment: Starting from a central limit theorem result, use Slutsky's theorem or the delta method to derive an approximate distribution for a transformed estimator and explain every assumption.
- Testing and Power Assessment: Design a level-controlled hypothesis test for a stated scientific question, derive or approximate its power, and explain how sample size and effect size influence the result.
- Likelihood Assessment: Derive the likelihood, score, maximum likelihood estimator, and observed or expected information for a one-parameter model, then discuss regularity conditions needed for asymptotic normality.
- Interval Interpretation Assessment: Critique several statements about confidence intervals, p-values, and credible intervals, correcting any statement that confuses repeated-sampling and posterior probability.
- Resampling Assessment: Decide whether an ordinary bootstrap is appropriate for a given dataset with dependence or extreme statistics, and propose a more suitable resampling strategy when necessary.
Evidence of Learning
Important evidence of learning includes:
- Knowledge: You can define models, parameters, statistics, estimators, likelihood, sufficiency, Fisher information, confidence procedures, hypothesis tests, power, asymptotic distributions, and posterior distributions.
- Mathematical skills: You can derive likelihoods, differentiate log-likelihoods, compute expectations and variances, manipulate pivots, and justify limiting distributions.
- Computational skills: You can simulate sampling distributions, verify theoretical approximations, implement bootstrap procedures, and inspect the numerical behavior of estimators and tests.
- Products: You can produce derivations, simulation reports, visualizations, reproducible notebooks, short explanatory videos, and research-style statistical analyses.
- Transfer achievements: You can take an unfamiliar applied problem, identify the inferential target, select a defensible method, check assumptions, quantify uncertainty, and communicate conclusions without confusing statistical evidence with certainty or causation.
OERs on the Topic
The English Wikipedia article provides a broad overview of the field:
For deeper university study, you can also use the openly available MIT OpenCourseWare materials:
MIT OpenCourseWare: Mathematical Statistics
MIT OpenCourseWare: Statistics for Applications
NIST/SEMATECH Engineering Statistics Handbook
Linked Learning Areas
aiMOOC Projects
MOOCwiki · Deutsch
Nach dem Lernen ist vor dem Lernen
Entdecke direkt den nächsten Lernkurs. Weitere Inhalte erscheinen, wenn Du weiter nach unten scrollst.
Zur MOOCwiki-HauptseiteMediathek
Mediathek
Mediathek wird aus dem Wiki geladen ...
Keine passenden Inhalte gefunden. Bitte ändere Suche oder Filter.
NEWSLernweltNOAH fragen