Die Druckversion wird nicht mehr unterstützt und kann Darstellungsfehler aufweisen. Bitte aktualisiere deine Browser-Lesezeichen und verwende stattdessen die Standard-Druckfunktion des Browsers.

Hypothesis Testing

Introduction

Hypothesis testing is a core method of inferential statistics. It helps you use information from a sample to evaluate a claim about a larger population. Instead of asking whether a sample result is merely different from a claimed value, you ask a more precise question: How surprising would data at least this extreme be if the null hypothesis and the statistical model were true?

This course is designed for Grades 11–13. You should already be comfortable with averages, proportions, basic probability, standard deviation, and the idea of a normal distribution. By the end, you should be able to formulate hypotheses, choose a suitable test, calculate and interpret a test statistic and p-value, distinguish statistical significance from practical importance, explain Type I and Type II errors, and critique conclusions from real studies.

The bell-shaped normal curve is important because many test statistics have distributions that are normal or approximately normal under suitable conditions. However, hypothesis testing is not limited to normally distributed raw data.

As you work through the course, keep one principle in mind: a hypothesis test does not prove a claim. It quantifies how compatible the observed data are with a particular null model, under stated assumptions.

From Research Question to Statistical Hypotheses

A good hypothesis test begins with a clear research question. Suppose a manufacturer claims that a rechargeable battery lasts 10 hours on average. You collect a random sample of batteries and observe a sample mean above 10 hours. Is the difference convincing evidence that the true population mean has changed, or could ordinary sampling variation explain it?

Null and Alternative Hypotheses

The null hypothesis, written $H_{0}$ , is the claim tested as the baseline. It usually states no difference, no effect, or a specific population value. The alternative hypothesis, written $H_{A}$ or $H_{1}$ , describes the competing claim.

For the battery example, a two-sided test could be written as:

$H_{0} : μ = 10$

$H_{A} : μ \neq 10$

Here, $μ$ is the true population mean battery life. Notice that the hypotheses concern a population parameter, not the sample mean. The sample provides evidence; it is not itself the claim being tested.

If the research question is specifically whether battery life has increased, a one-sided alternative could be $H_{A} : μ > 10$ . If the question is whether it has decreased, the alternative could be $H_{A} : μ < 10$ .

One-Sided and Two-Sided Alternatives

A two-sided test looks for evidence of a departure in either direction. A one-sided test looks only in a pre-specified direction. The direction must be chosen from the research question and study design before inspecting the data. Choosing a one-sided test after seeing which direction the sample moved can exaggerate evidence.

A one-sided test places the rejection region in one tail of the null distribution.

A two-sided test divides the chosen significance level between both tails.

The Logic of a Hypothesis Test

A hypothesis test compares observed data with what would be expected if $H_{0}$ were true. The logic can be summarized as follows:

Null hypothesis: State a precise baseline claim about a population parameter.
Alternative hypothesis: State the competing claim that matches the research question.
Significance level: Choose a threshold $α$ before examining the test result.
Test statistic: Standardize the observed difference so it can be compared with a reference distribution.
P-value: Measure how extreme the observed statistic is under the null model.
Statistical decision: Reject $H_{0}$ when the p-value is at most $α$ ; otherwise fail to reject $H_{0}$ .
Statistical interpretation: Translate the result back into the context of the research question.

The phrase fail to reject is important. A non-significant result does not prove that $H_{0}$ is true. It may reflect limited data, high variability, a small true effect, or a model that lacks power to detect the effect.

Significance Level and Critical Regions

The significance level $α$ is the long-run probability of rejecting $H_{0}$ when $H_{0}$ is true, assuming the test conditions hold. Common choices are 0.05 and 0.01, but no value is automatically correct for every situation. The consequences of false positives should influence the choice.

For a two-sided z-test with $α = 0.05$ , the standard normal critical values are approximately -1.96 and +1.96. Values beyond these cutoffs lie in the rejection regions.

Test Statistics

A test statistic measures how far the sample result lies from the null-hypothesis value relative to its standard error. A general pattern is:

$test statistic = \frac{estimate - null value}{standard error}$

A large absolute test statistic means the estimate is many standard errors away from the null value. Whether it counts as unusually large depends on the reference distribution and the chosen alternative.

Common test statistics include:

z statistics for some means and proportions when the relevant standard-error conditions are met.
t statistics for means when the population standard deviation is unknown.
chi-square statistics for categorical-data questions such as goodness of fit and association.
F statistics in comparisons of several means and in some regression models.

P-Values

The p-value is the probability, calculated under the null hypothesis and the statistical model, of obtaining a test statistic at least as extreme as the one observed in the direction specified by the alternative hypothesis.

A small p-value means that the observed result would be unusual if the null model were correct. It therefore counts as evidence against $H_{0}$ . It does not tell you the probability that $H_{0}$ is true, and it does not measure the size or importance of an effect.

Reading a P-Value Correctly

Suppose a study gives $p = 0.03$ . A careful interpretation is: if the null hypothesis and model assumptions were true, the probability of a test statistic at least as extreme as the observed one would be 3 percent.

Do not interpret $p = 0.03$ as any of the following:

The null hypothesis has a 3 percent chance of being true.
There is a 97 percent chance that the alternative hypothesis is true.
The result has a 97 percent chance of replicating.
The effect is large or practically important.
Randomness caused the result with probability 3 percent.

The p-value is conditional on a model and a null hypothesis. It is one piece of evidence, not a complete scientific conclusion.

P-Value and Alpha

A common decision rule is:

If $p \leq α$ , reject $H_{0}$ .

If $p > α$ , fail to reject $H_{0}$ .

The threshold should not become a substitute for judgment. Results just below and just above 0.05 can represent very similar evidence. Good reporting includes the actual p-value when possible, an effect estimate, uncertainty such as a confidence interval, the study design, and the assumptions.

Sampling Distributions and Reference Models

A hypothesis test relies on the sampling distribution of its test statistic under $H_{0}$ . This distribution describes how the statistic would vary across repeated samples if the null model generated the data.

The standard normal distribution is the reference model for many z procedures. Areas in its tails correspond to probabilities of extreme standardized values.

Student's t Distribution

When testing a population mean and the population standard deviation is unknown, the sample standard deviation introduces extra uncertainty. The Student's t-distribution accounts for this through heavier tails, especially for small degrees of freedom.

As the degrees of freedom increase, the t distribution approaches the standard normal distribution.

A t distribution can be used to locate critical values or tail areas for a test statistic. Modern software usually reports the p-value directly, but understanding the reference distribution is essential for interpreting the result.

Worked Example: One-Sample t-Test

A battery company claims that its new model lasts 10 hours on average. A random sample of 25 batteries has mean lifetime $\bar{x} = 10.75$ hours and sample standard deviation $s = 1.5$ hours. You want to test whether the mean lifetime differs from 10 hours.

Step 1: State hypotheses.

$H_{0} : μ = 10$

$H_{A} : μ \neq 10$

Step 2: Choose a significance level.

Let $α = 0.05$ .

Step 3: Check the design and assumptions.

The batteries should form an independent random or otherwise defensibly representative sample. For a one-sample t procedure, the sample distribution should not show severe outliers or extreme skewness when the sample is modest.

Step 4: Calculate the standard error.

$S E = \frac{s}{\sqrt{n}} = \frac{1.5}{\sqrt{25}} = 0.30$

Step 5: Calculate the t statistic.

$t = \frac{10.75 - 10}{0.30} = 2.50$

The degrees of freedom are $d f = n - 1 = 24$ .

Step 6: Find the two-sided p-value.

For $t = 2.50$ with 24 degrees of freedom, the two-sided p-value is about 0.020.

Step 7: Make the statistical decision.

Because $0.020 < 0.05$ , reject $H_{0}$ .

Step 8: Interpret in context.

The sample provides statistically significant evidence at the 5 percent level that the population mean battery lifetime differs from 10 hours. The estimated difference is 0.75 hours, so you should also consider whether that difference is practically meaningful and report a confidence interval for the mean or the difference.

Worked Example: One-Proportion z-Test

Suppose a school survey asks whether students support a proposed timetable change. In a random sample of 120 students, 72 support it. You want to test whether the true support proportion differs from 0.50.

$H_{0} : p = 0.50$

$H_{A} : p \neq 0.50$

The sample proportion is $\hat{p} = 72 / 120 = 0.60$ . Under the null hypothesis, the standard error is:

$S E_{0} = \sqrt{\frac{0.50 (1 - 0.50)}{120}} \approx 0.0456$

The z statistic is:

$z = \frac{0.60 - 0.50}{0.0456} \approx 2.19$

The two-sided p-value is about 0.028. At $α = 0.05$ , you would reject $H_{0}$ and conclude that the sample provides evidence that the population support proportion differs from 0.50.

This conclusion depends on the quality of the sampling process. A large sample cannot repair systematic bias caused by a badly chosen sample or a leading survey question.

Errors and Statistical Power

A hypothesis-testing decision can be wrong because it is based on a sample rather than complete knowledge of the population.

A Type I error occurs when you reject a true null hypothesis. Its long-run probability is controlled by $α$ , provided the test conditions are met.

A Type II error occurs when you fail to reject a false null hypothesis. Its probability is denoted by $β$ for a specified true alternative.

Statistical power is $1 - β$ . It is the probability that a test rejects $H_{0}$ when a specified alternative is true.

Power generally increases when the true effect is larger, the sample size is larger, measurement variability is smaller, or the significance level is less strict. Increasing $α$ can increase power, but it also raises the risk of Type I errors. Study design therefore involves trade-offs.

A Decision Table

Reality	Fail to reject the null hypothesis	Reject the null hypothesis
Null hypothesis is true	Correct decision	Type I error
Null hypothesis is false	Type II error	Correct detection

The labels describe long-run decision behavior. In a single study, you usually do not know with certainty which state of reality applies.

Statistical Significance, Effect Size, and Practical Importance

Statistical significance concerns whether the observed data are sufficiently incompatible with the null model under a chosen threshold. Effect size concerns how large the estimated difference or relationship is. Practical importance concerns whether that magnitude matters in the real context.

With a very large sample, a tiny and unimportant effect can produce a small p-value. With a small sample, an important effect can fail to reach statistical significance because the estimate is too imprecise.

For a one-sample mean, a standardized effect size can be written as:

$d = \frac{\bar{x} - μ_{0}}{s}$

For the battery example, $d = (10.75 - 10) / 1.5 = 0.50$ . Whether this is meaningful depends on the application, costs, risks, and comparison standards.

Confidence Intervals and Hypothesis Tests

A two-sided hypothesis test and a matching confidence interval often express the same underlying information. For many standard procedures, a two-sided test at $α = 0.05$ rejects a null value exactly when that null value lies outside the corresponding 95 percent confidence interval.

Confidence intervals add information because they show a range of parameter values reasonably compatible with the data under the model. They help you judge both direction and plausible magnitude, not just whether a threshold was crossed.

Assumptions and Conditions

A calculation is only as trustworthy as the data-generating process and assumptions behind it. Before interpreting a hypothesis test, ask the following questions.

Sampling: Was the sample randomly selected or otherwise representative of the target population?
Independence: Are observations independent, or does the method account for clustering, repeated measures, or pairing?
Measurement: Are the variables measured reliably and without systematic bias?
Distribution: Does the chosen test require a normal model, approximate normality, or sufficient expected counts?
Sample size: Is the sample large enough for the approximation used by the test?
Study design: Is the study observational or experimental, and does the design justify causal conclusions?
Missing data: Could missing observations distort the estimate?
Multiple comparisons: Were many hypotheses tested, increasing the chance of false positives?

Do not mechanically apply a test because the software offers it. First identify the parameter, data type, design, and assumptions.

Choosing a Suitable Test

The correct test depends on the research question, data type, number of groups, and study design.

Research goal	Typical parameter	Common procedure
Compare one numerical mean with a claimed value	Population mean	One-sample t-test
Compare two independent numerical means	Difference in population means	Independent-samples t-test
Compare paired numerical measurements	Mean of paired differences	Paired t-test
Compare one proportion with a claimed value	Population proportion	One-proportion z-test or exact method
Test association between categorical variables	Pattern of cell probabilities	Chi-square test of independence
Test a linear relationship	Regression slope or correlation	Regression or correlation test

The table gives common choices, not automatic rules. Exact methods, nonparametric procedures, permutation tests, or model-based methods may be better when standard assumptions are unsuitable.

Multiple Testing and Reproducibility

If you test many independent null hypotheses at $α = 0.05$ , the probability of at least one false positive across the set can become much larger than 5 percent. This is the multiple testing problem.

Methods such as the Bonferroni correction can control a family-wise error rate by using a stricter threshold for each test. Other approaches control the false discovery rate. Which method is appropriate depends on the goals of the analysis.

Reproducible statistical practice also includes pre-specifying hypotheses when possible, documenting exclusions and transformations, reporting all relevant analyses rather than only significant ones, sharing methods clearly, and distinguishing exploratory from confirmatory work.

Communicating a Hypothesis Test

A strong written conclusion should include the parameter and comparison, the direction and approximate magnitude of the observed effect, the test statistic and degrees of freedom when relevant, the p-value, the significance level if a decision rule is used, and a confidence interval or other uncertainty measure when possible.

For the battery example, a concise report might state that the sample mean was 10.75 hours, 0.75 hours above the claimed 10 hours, and that a two-sided one-sample t-test gave $t (24) = 2.50$ with $p \approx 0.020$ . The data therefore provide evidence against the 10-hour population mean at the 5 percent level, subject to the sampling and model assumptions.

Avoid the phrase the test proves. Statistics supports measured uncertainty; it does not turn sample evidence into certainty.

Interactive Tasks

Quiz: Test Your Knowledge

What does the null hypothesis usually represent in a hypothesis test? (A baseline claim about a population parameter) (!The observed sample result) (!A guarantee that there is no effect) (!The probability that the data are correct)

What is a p-value? (The probability of data at least as extreme under the null model) (!The probability that the null hypothesis is true) (!The size of the effect in the population) (!The probability that the study will replicate)

When should the direction of a one-sided test be chosen? (Before examining the test result) (!After seeing which sample mean is larger) (!Only after calculating the p-value) (!Whenever a two-sided test is not significant)

What decision is made when the p-value is at most the chosen alpha level? (Reject the null hypothesis) (!Accept the null hypothesis as proven) (!Increase the sample mean) (!Set the p-value equal to zero)

What is a Type I error? (Rejecting a true null hypothesis) (!Failing to reject a false null hypothesis) (!Using a sample that is too large) (!Reporting an effect size)

What is statistical power? (The probability of rejecting a false null hypothesis) (!The probability that the null hypothesis is true) (!The chosen significance level) (!The observed standard deviation)

Why is a t distribution used in many tests of a mean? (The population standard deviation is unknown and estimated) (!The sample contains only categories) (!The null hypothesis must always be zero) (!The p-value must be larger than alpha)

What does a non-significant result justify? (Failing to reject the null hypothesis) (!Proving the null hypothesis is true) (!Proving there is no meaningful effect) (!Ignoring the study assumptions)

Why should effect size be reported with statistical significance? (It describes the magnitude of the estimated effect) (!It makes every result statistically significant) (!It replaces the need for data) (!It guarantees a causal conclusion)

What can happen when many hypotheses are tested without adjustment? (The chance of false positive findings can increase) (!Every p-value becomes exactly zero) (!The sample size automatically decreases) (!Type II errors become impossible)

Memory Game

Null hypothesis	Baseline population claim evaluated by the test
P-value	Tail probability for results at least as extreme under the baseline model
Significance level	Preselected threshold used for a rejection decision
Type I error	Rejection of a baseline claim that is actually true
Statistical power	Probability of detecting a specified real effect
Effect size	Numerical measure of the magnitude of a difference or relationship

Drag and Drop

Match the correct terms.	Topic
Null hypothesis	Baseline population claim
Alternative hypothesis	Competing research claim
Test statistic	Standardized distance from the null value
P-value	Tail probability under the null model
Statistical power	Probability of detecting a specified true effect

Crossword Puzzle

Pvalue	Which quantity measures tail evidence against the null model?
Alpha	Which word names the chosen significance threshold?
Power	Which quantity equals one minus the Type II error probability?
Statistic	What standardized quantity is compared with a reference distribution?
Sampling	What process produces sample-to-sample variation?
Inference	What branch of statistical reasoning draws conclusions about populations from samples?

LearningApps

Cloze Text

A hypothesis test begins with a baseline claim called the

. The competing claim is the

. A standardized measure of departure from the baseline is the

. The probability of results at least as extreme under the baseline model is the

. The preselected decision threshold is the

. Rejecting a true baseline claim is a

. Failing to reject a false baseline claim is a

. The probability of detecting a specified real effect is statistical

. A numerical description of the magnitude of a difference is an

. A non-significant result means you should

the baseline claim rather than declare it proven.

Open-Ended Tasks

Easy

Hypothesis Statements: Choose a school-related question and write a null hypothesis and a matching two-sided alternative hypothesis using a clearly defined population parameter.
P-Value Explanation: Create a 90-second audio or video explanation of a p-value for a learner who knows basic probability but has not studied inferential statistics.
Error Scenarios: Draw a two-by-two diagram that explains Type I and Type II errors through a real decision such as a medical screening, quality check, or school policy trial.
Media Critique: Find a news or social-media claim that uses the word significant and write a short paragraph explaining whether it appears to mean statistical significance, practical importance, or ordinary importance.

Standard

Coin Experiment: Toss a coin at least 60 times, test the claim that the probability of heads is one half using an appropriate method, and report the result with a graph and interpretation.
Survey Design: Design a short anonymous survey with one proportion you could test, explain how you would sample participants, and identify at least two possible sources of bias.
Confidence Interval Connection: Use a dataset or simulation to calculate a two-sided hypothesis test and a matching confidence interval, then explain how the two results tell a consistent story.
Power Simulation: Use a spreadsheet, calculator, or statistical program to simulate repeated tests under two sample sizes and compare how often each design detects a chosen effect.

Advanced

Reproducible Analysis: Analyze an open dataset with a suitable hypothesis test, document every decision, include effect size and uncertainty, and provide enough detail for another learner to reproduce the analysis.
Multiple Testing Investigation: Simulate many null hypotheses and estimate how often at least one false positive appears when no correction is used, then compare the result with a correction method.
Experimental Study Proposal: Design a controlled experiment, identify the response and explanatory variables, pre-specify hypotheses, choose alpha, discuss power, and describe how randomization would support causal inference.
Research Interview: Interview a researcher, data analyst, or laboratory professional about how hypothesis tests are used in practice, then produce a written or video report comparing professional practice with the methods in this course.

Learning Assessment

Choosing a Test: Given three research scenarios with different data types and study designs, choose a suitable hypothesis test for each and justify your choice using the parameter, assumptions, and sampling structure.
Interpreting Evidence: Compare results with p-values 0.049 and 0.051 and explain why they should not be treated as fundamentally different levels of scientific evidence.
Error Trade-Offs: For a safety-critical quality-control problem, explain the consequences of Type I and Type II errors and recommend a significance level with a reasoned justification.
Study Critique: Evaluate a short study report for sampling bias, model assumptions, multiple testing, effect size reporting, and whether the causal language is justified.
Power and Sample Size: Explain how increasing sample size can change power and p-values while leaving the true effect size unchanged, and describe why a larger study is not automatically less biased.
Transfer to New Context: Construct a complete hypothesis-testing plan for a new real-world question, including hypotheses, test choice, assumptions, test statistic, decision rule, effect-size measure, and communication strategy.

Evidence of Learning

Knowledge: You can define null and alternative hypotheses, significance level, test statistic, p-value, Type I and Type II errors, statistical power, effect size, sampling distribution, and confidence interval, and you can explain how these concepts connect.

Skills: You can translate a research question into statistical hypotheses, choose and carry out an appropriate test, check assumptions, calculate or obtain a p-value, interpret results in context, and distinguish statistical from practical importance.

Products: Strong evidence may include a reproducible data analysis, a simulation of sampling or power, a well-designed survey or experiment proposal, a visual explanation of errors, and a clear written statistical report.

Reasoning: You can explain why a small p-value counts as evidence against a null model without treating it as the probability that the null hypothesis is true. You can also explain why failing to reject is not the same as proving no effect.

Transfer: You can apply hypothesis-testing ideas to unfamiliar contexts, critique statistical claims in media and research, and choose methods according to data type, design, assumptions, and the real consequences of decision errors.

OERs on the Topic

The linked Wikipedia article provides a broad reference on the logic, history, methods, and interpretation of statistical hypothesis testing. Use it to extend this course and compare terminology across different statistical traditions.

Linked Learning Areas

Hypothesis testing connects probability models with decisions based on data. The essential links are population parameters, sampling distributions, standard errors, null and alternative hypotheses, test statistics, p-values, significance levels, confidence intervals, error rates, power, effect sizes, study design, and reproducible reporting.

Hypothesis Testing