Correlation and Regression

Introduction

Correlation and Regression are central tools in statistics for studying relationships between quantitative variables. At Grades 11–13, you should learn not only how to calculate a correlation coefficient or a regression line, but also how to decide whether those calculations are meaningful. A convincing statistical analysis combines a graph, numerical summaries, a model, checks of the model, and careful interpretation in context.

Imagine that you collect paired data such as hours of practice and performance score, temperature and electricity use, or advertising expenditure and sales. A scatterplot lets you see the form, direction, strength, clusters, gaps, and unusual observations. Pearson's correlation coefficient summarizes the direction and strength of a linear association. Simple linear regression goes one step further by fitting a line that predicts one variable from another.

Fehler beim Erstellen des Vorschaubildes:

The image above shows an important lesson: the same or similar correlation value can hide very different shapes. You should therefore inspect the scatterplot before you rely on a single summary number.

Learning Goals

By the end of this aiMOOC, you should be able to:

Construct and interpret scatterplots by describing form, direction, strength, and unusual observations.
Calculate and interpret Pearson's correlation coefficient for a linear relationship.
Find and interpret a least-squares regression line in context.
Calculate residuals and use residual plots to judge whether a linear model is appropriate.
Interpret the coefficient of determination without confusing it with causation.
Recognize outliers and influential observations and explain how they can affect correlation and regression.
Distinguish interpolation from extrapolation and judge prediction risk.
Separate association from causation and identify possible confounding variables.

Prerequisites and Notation

You should already be comfortable with means, standard deviations, coordinates, straight-line equations, and basic algebra. In this course, $x$ usually denotes the explanatory or predictor variable and $y$ the response variable. A measured response is written $y_{i}$ ; a value predicted by a model is written ${\hat{y}}_{i}$ . A bar such as $\bar{x}$ denotes a sample mean.

The words association and relationship describe how variables vary together. The word cause is stronger: a causal claim says that changing one variable produces a change in another. Observational data can reveal associations, but a correlation by itself cannot establish causation.

Exploring Bivariate Data

Start with a Scatterplot

A scatterplot displays each paired observation as one point. Before calculating anything, ask five questions. What is the overall form? Is the direction positive, negative, or unclear? How closely do the points follow the form? Are there clusters or gaps? Are there unusual points that might strongly affect a numerical summary?

A positive association means that larger values of one variable tend to occur with larger values of the other. A negative association means that larger values of one variable tend to occur with smaller values of the other. An association can also be curved, grouped, or otherwise non-linear.

Datei:Correlation coefficient.svg

The scatterplots above illustrate different values of Pearson correlation. Notice that correlation describes linear direction and strength; it does not tell you the slope of a fitted line, and it is not a complete description of the dataset.

Form, Direction, Strength, and Unusual Features

A useful written description follows a consistent order. First state the form: linear, curved, clustered, or irregular. Then state the direction: positive, negative, or no clear direction. Next describe the strength: how tightly the points follow the stated form. Finally identify unusual features: outliers, gaps, clusters, or points with extreme predictor values.

Avoid treating labels such as “weak,” “moderate,” and “strong” as universal scientific boundaries. A correlation that is useful in one field may be inadequate in another. Always interpret the numerical value together with the context, the scatterplot, measurement quality, and the purpose of the analysis.

A Real-World Scatterplot

Datei:2021 Carbon dioxide (CO2) emissions per person versus GDP per person - scatter plot.svg

This example compares national economic output per person with carbon dioxide emissions per person. It is useful for practicing statistical reading: you can identify a broad positive association, but the graph alone cannot prove that one variable directly causes the other. Countries differ in industrial structure, energy sources, climate, policy, technology, and many other factors. In addition, the chart uses logarithmic scaling, which affects the visual appearance of the relationship.

When you analyze a real scatterplot, describe exactly what the variables represent and what each point represents. A statistical statement without that context can easily become misleading.

Correlation

Pearson's Correlation Coefficient

The Pearson correlation coefficient for sample data is usually written $r$ . It measures the direction and strength of a linear association between two quantitative variables. Its value is always between $- 1$ and $1$ when both variables have non-zero variation.

A common computational form is:

$r = \frac{\sum (x_{i} - \bar{x}) (y_{i} - \bar{y})}{\sqrt{\sum (x_{i} - \bar{x})^{2} \sum (y_{i} - \bar{y})^{2}}}$

If $r$ is close to $1$ , the data show a strong positive linear association. If $r$ is close to $- 1$ , the data show a strong negative linear association. If $r$ is near zero, there is little linear association, but there may still be a strong non-linear relationship.

Correlation is unit-free. Changing metres to centimetres or euros to cents does not change the magnitude of $r$ when the change is a positive linear rescaling. Correlation is also symmetric: the correlation of $x$ with $y$ is the same as the correlation of $y$ with $x$ .

Why Standardization Matters

Pearson's $r$ can be understood through standardized values. Convert an observation to a z-score by subtracting the mean and dividing by the standard deviation. A point with both standardized values positive or both negative contributes positively to correlation. A point with one positive and one negative standardized value contributes negatively.

This view explains why $r$ measures co-movement rather than slope. Two datasets can have very different measurement scales and regression slopes but the same correlation.

Correlation Is Not Causation

A correlation can arise for several reasons. One variable may influence the other, the causal direction may run the opposite way, a third variable may influence both, the relationship may be created partly by selection or measurement, or the observed association may be a chance pattern in a sample.

A confounding variable is related to both variables of interest and can help create or distort an observed association. For example, ice-cream sales and swimming incidents may both rise during warm weather. The association does not mean that buying ice cream causes swimming incidents; temperature and seasonal behavior are plausible common influences.

To support a causal claim, you need a suitable research design, subject-matter reasoning, and evidence that addresses alternative explanations. Randomized experiments can strengthen causal inference when they are ethical and feasible, but many questions require careful observational methods instead.

Non-Linear Relationships and Zero Correlation

A value of $r$ near zero does not mean “no relationship.” Pearson correlation is designed for linear association. If points follow a U-shaped curve, for example, positive and negative linear tendencies can cancel.

This is why the scatterplot comes first. For a monotonic relationship that is not well described by a straight line, Spearman's rank correlation may be a useful alternative. For more complex relationships, you may need a different model entirely.

Outliers, Leverage, and Influence

An outlier is an observation that departs noticeably from the main pattern. A point with an extreme $x$ -value has high leverage because it can strongly affect the fitted regression line. A point is influential when removing it would substantially change an important result such as the slope, intercept, or correlation.

Do not delete an unusual point simply because it is inconvenient. First check whether it is a recording or measurement error. If it is a genuine observation, analyze why it differs and report how sensitive your conclusions are to it.

Datei:Anscombe's quartet 3.svg

Anscombe's quartet contains four datasets with almost identical common numerical summaries, including similar regression results, but very different scatterplots. It demonstrates why graphical inspection is essential.

Linear Regression

From Association to Prediction

Simple linear regression models the mean response as a linear function of a predictor. The fitted line is written:

$\hat{y} = b_{0} + b_{1} x$

Here $\hat{y}$ is the predicted response, $b_{0}$ is the intercept, and $b_{1}$ is the slope. The slope is the predicted change in the response for a one-unit increase in the predictor. Its units are “response units per predictor unit.”

The intercept is the predicted response when $x = 0$ . An intercept can be mathematically necessary without being practically meaningful. If zero is far outside the observed range of $x$ , interpreting the intercept may be unreasonable.

Fehler beim Erstellen des Vorschaubildes:

The Least-Squares Principle

For each observation, the vertical difference between the observed and predicted response is called a residual:

$e_{i} = y_{i} - {\hat{y}}_{i}$

A positive residual means the observed response lies above the fitted line. A negative residual means it lies below. The least-squares regression line is the line that minimizes the sum of squared residuals:

$\sum e_{i}^{2}$

Squaring prevents positive and negative residuals from cancelling and gives larger errors more weight. In ordinary simple linear regression with an intercept, the fitted line passes through $(\bar{x}, \bar{y})$ , and the residuals sum to zero apart from rounding.

Datei:Residuals for Linear Regression Fit.png

Linking Correlation and the Regression Line

For simple linear regression with one predictor and an intercept, the least-squares slope can be written using the sample correlation and standard deviations:

$b_{1} = r \frac{s_{y}}{s_{x}}$

The intercept is:

$b_{0} = \bar{y} - b_{1} \bar{x}$

These formulas show a deep connection between correlation and regression. The sign of the slope is the same as the sign of $r$ . However, the slope depends on measurement units while $r$ does not.

Regression is not symmetric. Regressing $y$ on $x$ generally gives a different prediction problem from regressing $x$ on $y$ . Correlation, in contrast, treats the variables symmetrically.

Worked Example

Suppose six students record weekly practice hours $x$ and a performance score $y$ :

Practice hours	Performance score
1	50
2	62
3	58
4	72
5	75
6	81

For these data, the least-squares line is approximately:

$\hat{y} = 45.53 + 5.94 x$

The slope means that the model predicts an increase of about 5.94 performance-score points for each additional weekly practice hour within the range represented by the data. The sample correlation is approximately $r = 0.955$ , so the data show a strong positive linear association.

For a student with three practice hours, the predicted score is approximately $63.36$ . The observed score is $58$ , so the residual is about $- 5.36$ . The negative sign tells you that this observation lies below the fitted line.

This example is an association study. It does not, by itself, prove that increasing practice by one hour will cause a score increase of exactly 5.94 points. Motivation, prior skill, coaching, sleep, and many other variables may also matter.

Coefficient of Determination

The coefficient of determination, written $R^{2}$ , measures how much of the sample variation in the response around its mean is accounted for by the fitted regression model. One common formula is:

$R^{2} = 1 - \frac{\sum e_{i}^{2}}{\sum (y_{i} - \bar{y})^{2}}$

For ordinary simple linear regression with an intercept, $R^{2} = r^{2}$ . In the worked example, $R^{2}$ is approximately $0.912$ , so about 91.2 percent of the sample variation in performance scores around their mean is accounted for by the fitted linear relationship with practice hours.

This interpretation is about the fitted model and the observed data. A high $R^{2}$ does not prove causation, does not guarantee accurate predictions for new situations, and does not show that the model assumptions are satisfied.

Checking a Regression Model

Residual Plots

A residual plot displays residuals against the predictor or against fitted values. If a straight-line model is reasonable, you generally hope to see residuals scattered around zero without a clear curve or systematic pattern.

A curved pattern suggests that a linear model misses important structure. A funnel-shaped spread suggests that the variability changes across the range of predictions. Distinct groups may indicate a missing categorical variable. An isolated residual may reveal an unusual observation that deserves investigation.

Datei:Linear regression residuals.svg

Heteroscedasticity

Heteroscedasticity means that the spread of residuals is not roughly constant across the fitted range. This matters especially when you move from descriptive regression to statistical inference because standard errors and uncertainty calculations can be affected.

Datei:Linear regression residuals heteroscedastic.svg

At school level, the key diagnostic idea is visual: a pattern such as a widening fan tells you that “one typical prediction error” may not describe all parts of the data equally well.

Linearity, Independence, and Data Quality

A basic regression analysis should consider several conditions. The mean relationship should be reasonably linear if you use a straight-line model. Observations should be collected in a way that makes dependence understandable; repeated measurements on the same person, time-series data, or spatially related measurements may violate a simple independence assumption. The variability around the fitted relationship should be considered, especially if formal inference is planned. Measurement quality and sampling design also matter.

For inference about slopes or predictions, many textbook procedures additionally use assumptions about the distribution of errors. The exact requirements depend on the method. Do not turn an assumption checklist into a ritual: connect each assumption to the data-generating process and the question you are answering.

Interpolation and Extrapolation

Interpolation predicts within the range of predictor values represented by the data. Extrapolation predicts outside that range. Extrapolation is riskier because the relationship may change beyond the observed data.

For example, a regression line fitted to practice times between one and six hours should not automatically be used to predict performance at twenty hours. Physical limits, fatigue, different behavior, or a curved relationship may make the linear rule unrealistic.

Correlation, Regression, and Good Statistical Reasoning

Questions to Ask Before You Trust the Model

A strong analysis is more than a calculation. Ask what population the data are meant to represent and how the sample was obtained. Check whether both variables were measured reliably. Look for a plausible linear form before using Pearson correlation or a straight-line regression. Investigate unusual points. Examine residuals. State whether your purpose is description, prediction, or causal explanation. Finally, communicate uncertainty and limitations.

A model can be useful without being literally true. Statistical models are simplified representations of patterns. Their value comes from how well they answer a specific question within a justified range of conditions.

Prediction versus Explanation

Regression can be used for prediction or for explanation, but the goals are not identical. A predictive model is judged by how well it predicts new observations. An explanatory analysis tries to understand how variables are related and often needs stronger reasoning about design, confounding, and causality.

A predictor can improve prediction without being a direct cause. Conversely, a causal factor may have only a modest correlation with an outcome when many other factors also matter.

Responsible Communication

When you report correlation and regression, name the variables, units, sample, and context. Give the numerical result with sensible precision, describe the scatterplot, and mention model limitations. Avoid claims such as “X causes Y” unless the research design and evidence justify them.

Be especially cautious with sensitive data about people. Statistical patterns can reflect sampling decisions, measurement choices, social conditions, or historical inequities. Good analysis includes ethical judgment about how data were obtained, what the model leaves out, and how predictions might be used.

Technology and Reproducibility

A calculator, spreadsheet, or statistical program can compute $r$ , regression coefficients, residuals, and $R^{2}$ . Technology should reduce arithmetic burden, not replace reasoning. Save the data, record the steps you used, label graphs clearly, and make your analysis reproducible enough that another learner can check it.

A useful workflow is: enter paired data, create a scatterplot, calculate a correlation only if a linear summary is meaningful, fit a regression line if prediction is relevant, inspect residuals, investigate unusual observations, and write a conclusion in context.

Interactive Tasks

Quiz: Test Your Knowledge

What does Pearson correlation primarily measure? (Direction and strength of a linear association) (!The slope of every possible relationship) (!Proof that one variable causes another) (!The difference between two sample means)

What should you usually inspect before interpreting a correlation coefficient? (The scatterplot of the paired data) (!Only the largest observed value) (!Only the sample size) (!Only the response mean)

What does a positive regression slope mean in context? (Predicted response increases as the predictor increases) (!Predicted response always equals the observed response) (!The variables must have a causal relationship) (!Every residual is positive)

What quantity is minimized by a least squares regression line? (The sum of squared residuals) (!The sum of all predictor values) (!The number of data points) (!The range of the response variable)

What is a residual? (The observed response minus the predicted response) (!The predictor value minus its mean) (!The regression slope divided by the intercept) (!The correlation multiplied by the sample size)

What can a curved residual pattern suggest? (A straight line is missing systematic structure) (!The correlation must equal one) (!The data contain no variation) (!The intercept must be zero)

What does R squared describe in a fitted regression model? (The proportion of response variation accounted for by the model) (!The probability that the model is causal) (!The number of observations that are correct) (!The size of the regression intercept)

Why can an outlier be important in correlation and regression? (It may strongly change the numerical results) (!It always proves the data are wrong) (!It must always be deleted) (!It makes every correlation zero)

Why is extrapolation usually riskier than interpolation? (The relationship may change outside the observed range) (!Interpolation never uses a model) (!Extrapolation always gives a negative residual) (!Interpolation can only be used with categories)

What is true about correlation and causation? (Correlation alone does not establish causation) (!A large correlation always proves causation) (!A negative correlation rules out causation) (!A zero correlation proves independence)

Memory Game

Scatterplot	Graph of paired quantitative observations
Correlation	Unit free summary of linear direction and strength
Regression	Method for modelling a response from a predictor
Residual	Observed response minus fitted response
Outlier	Observation that departs noticeably from the main pattern
Interpolation	Prediction within the observed predictor range
Confounder	Third variable related to both variables of interest

Drag and Drop

Match the correct terms.	Topic
Measures linear association	Pearson correlation
Predicts a response from a predictor	Linear regression
Shows model prediction errors	Residual plot
Prediction inside the observed range	Interpolation
Prediction beyond the observed range	Extrapolation

...

Crossword Puzzle

Correlation	Which word names a standardized measure of linear association?
Regression	Which method fits a predictive relationship between variables?
Residual	What is the difference between an observed and predicted response called?
Scatterplot	Which graph displays paired quantitative observations as points?
Outlier	What do you call an observation that lies unusually far from the main pattern?
Extrapolation	What is prediction beyond the observed predictor range called?

LearningApps

Cloze Text

A

should usually be inspected before a numerical summary is interpreted. Pearson's coefficient describes the direction and strength of a

association. A correlation close to zero can still occur when the relationship is strongly

. The least-squares method minimizes the sum of squared

. A residual is the observed response minus the

response. In simple linear regression with an intercept, the sign of the slope matches the sign of the

. The coefficient of determination is commonly written as

. A patterned residual plot can indicate that the chosen model is

. Prediction inside the observed predictor range is called

. A statistical association by itself does not establish

.

Open-Ended Tasks

Easy

Human scatterplot: Collect one pair of quantitative measurements from at least ten willing participants, create a scatterplot, and describe its form, direction, strength, and unusual features without making a causal claim.
Correlation language audit: Find three correlation claims in news, advertising, or social media and rewrite each claim so that it distinguishes association from causation.
Residual cards: Create illustrated cards that show an observed value, a predicted value, and the resulting residual, including examples with positive, negative, and near-zero residuals.
One-minute explanation video: Record a short video in clear English explaining why a scatterplot should be inspected before a correlation coefficient is trusted.

Standard

Mini regression investigation: Gather a small paired dataset, fit a regression line with technology, interpret the slope and intercept in context, and state whether the intercept is practically meaningful.
Residual diagnostic poster: Create a poster comparing a random residual pattern with a curved pattern and a funnel-shaped pattern, then explain what each pattern suggests.
Prediction boundary study: Use a fitted line to make two interpolated predictions and one extrapolated prediction, then explain why the extrapolated result requires more caution.
Interview about data decisions: Interview a teacher, technician, researcher, or workplace professional about a situation in which correlation or regression is used, and summarize how they check whether a model is trustworthy.

Advanced

Reproduce Anscombe's quartet: Use a spreadsheet or programming tool to recreate the four scatterplots, calculate their main summaries, and explain why graphical analysis changes the interpretation.
Confounding investigation: Design an observational study on a school or community question, identify at least three plausible confounders, and explain what additional evidence would be needed before making a causal claim.
Compare two models: Find or generate a dataset with a curved relationship, compare a straight-line model with a suitable alternative, inspect residuals, and argue which model is more appropriate.
Reproducible analysis report: Produce a complete report containing data provenance, a scatterplot, correlation, fitted model, residual analysis, prediction limits, ethical considerations, and a short reflection on uncertainty.

Learning Assessment

Model critique: Given a scatterplot, correlation, regression equation, and residual plot, write a connected judgment about whether the linear model is appropriate and support every claim with specific evidence.
Causal reasoning: Analyze an observational correlation claim, propose at least two alternative explanations, and describe a study design that would give stronger evidence about causality.
Influence analysis: Compare correlation and regression results with and without an unusual observation, then explain why deleting or retaining the point must be justified by data quality and context.
Prediction reasoning: Evaluate several proposed predictions from one regression model, identify which are interpolation or extrapolation, and rank them by likely reliability with reasons.
Diagnostic transfer: Match several unfamiliar residual plots to plausible modelling problems such as curvature, unequal spread, or an unusual point, and explain the consequences for interpretation.
Evidence-based report: Write a concise statistical conclusion for a non-specialist audience that includes the direction and form of association, key numerical results, model limitations, and a warning against unjustified causal language.

Evidence of Learning

Evidence area	What successful learning looks like
Knowledge	You accurately explain scatterplots, Pearson correlation, least-squares regression, residuals, coefficient of determination, interpolation, extrapolation, outliers, and confounding.
Skills	You calculate and interpret correlation and regression results, create appropriate graphs, inspect residuals, compare models, and use technology without surrendering statistical judgment.
Products	You produce labelled scatterplots, regression analyses, residual diagnostics, short reports, presentations, posters, videos, or reproducible digital investigations.
Reasoning	You distinguish linear association from causation, recognize when a summary statistic hides structure, and justify whether a prediction is supported by the observed data range.
Transfer	You apply the same reasoning to unfamiliar contexts in science, economics, social research, technology, vocational data, and everyday claims.

OERs on the Topic

Explore these English Wikipedia articles as open reference material. Use them to extend the course, check terminology, and follow links to deeper topics.