aiMOOC

Classification, regression and model selection - AI Year 10

QR-Code

Introduction

Start with the target: a category calls for classification, while a numerical quantity calls for regression. Compare simple baselines, a restricted decision stump, one-nearest-neighbour methods and an ordinary fitted line on original tiny synthetic tasks. The complete exact-arithmetic lab was executed. It is not a spam filter, delivery service trial, generic model-import tool or proof that one family is universally best.

Choose the task from the meaning of the target

Classification predicts a member of a declared set of categories. Spam or not spam is the source example: the required output is a category, not a measured quantity. A class can be represented by a number such as zero or one without becoming a regression task. Predicting the numeric code for a tool type is still classification when the codes identify categories rather than meaningful amounts or distances.

Regression predicts a numerical quantity, such as delivery time in minutes. The difference between ten and twelve minutes has a quantitative meaning, and an error of two minutes can be compared with an error of twenty. A regression prediction may be written as a decimal even when a reporting system later rounds it. Rounding a number for display does not by itself change the underlying target into a classification problem.

Distinguish supervised targets from features. Features are inputs legitimately available at prediction time; the target is what the method is intended to predict. A delivery example could use distance available before a trip. The actual arrival time, or a field recorded only after arrival, must not secretly become an input for a claim about advance prediction. Feature units, missingness, permissions and collection context belong in a dictionary before model choice.

Target meaning	Suitable task family	Example comparison evidence
One of declared categories	Classification	Correct and incorrect categories, false positives and false negatives where relevant
Numerical amount or duration	Regression	Numerical errors in the target's units, with a suitable aggregation and baseline

The original classification experiment uses a synthetic count x and an authored binary review flag y. It is loosely inspired by message triage, but contains no actual message text, pupil data or evidence about spam. Counts alone can be insufficient: different fictional messages may share a count but have different intended flags. The original regression experiment uses invented distance-like inputs and delivery-minute targets, not observations collected from a real delivery service.

A baseline makes a model comparison interpretable. For classification, a majority-class baseline predicts the most common training target. For our tied two-zero two-one training labels, the declared tie rule chooses zero. For regression, a mean baseline predicts the average training target; the original four target values five, seven, nine and eleven have mean eight minutes. Other baselines can be more appropriate for another task, but they must be declared and fitted without using final test outcomes.

Do not choose an error measure merely because it produces an attractive percentage. Classification accuracy counts correct categories, while confusion counts explain different error types. A mean absolute regression error reports the average magnitude of numerical mistakes in the target's units. Reporting delivery-minute errors as category accuracy would require an explicit categorisation or tolerance rule; otherwise it changes the question without saying so.

CodeAI's AI: What is Machine Learning? supplies a broad introduction to learning from examples. The course's own written definitions, original datasets and executed code establish what we actually did. A video or a label such as AI does not decide the task type or validate an application.

Attempt before checking: identify the task type for six proposed targets: spam versus not spam; delivery minutes; tool type hammer saw or drill; weekly energy in kilowatt-hours; plant needs review yes or no; next-day rainfall in millimetres. The editorial target is five of six correct, followed by a comparison using a metric, baseline and limitation. The worked Learning Check gives the classifications after your attempt; this exercise target is not a qualification award.

Understand three model ideas and their assumptions

A decision tree follows feature-based branches to a prediction. It can support classification or regression depending on the target and fitted leaf values. A very shallow tree can be easy to inspect, while a large tree can fit peculiarities of training examples. Our original classification candidate is deliberately restricted: a one-split decision stump predicting one when the count is at least cut. It is fitted by minimising training classification mistakes across integer cuts zero through twenty-one. This is not a full implementation of a general tree-learning library or its usual splitting criteria.

Train that stump on feature-target pairs zero-zero, one-zero, four-one and five-one. Cuts two, three and four all separate the training labels perfectly. Our declared tie rule chooses the smallest, two. The training data alone does not determine how the gap between one and four should be labelled. Choosing a tie rule resolves the implementation contract, not the uncertainty about new examples in that gap.

One-nearest-neighbour learning retains the permitted training pairs and predicts the target of the closest feature. We use absolute distance in a single numerical feature. If two training examples are equally distant, prefer the smaller feature; if features also tie, prefer the earlier retained training row. The candidate stores training examples rather than fitting a slope. This is still a supervised prediction method, but its definition of near, scale, available examples and tie rule all affect its output.

For count two, training feature one is the nearest, with target zero. For count three, training feature four is the nearest, with target one. At two and a half, features one and four would tie in absolute distance, so the declared lower-feature rule would choose one; our classification input contract itself uses integer counts, so that fractional query is a conceptual tie illustration rather than a classification test call. A multidimensional task would require further choices about feature scaling and distance; none is implemented here.

An ordinary least-squares line predicts a numerical target from an input using an intercept and slope, chosen to minimise squared training residuals for the fitted line family. Our original exact-arithmetic implementation fits the four regression pairs zero-five, one-seven, two-nine and three-eleven. It obtains intercept five and slope two:

$\hat{y} = 5 + 2 x$

The target is minutes, so the intercept is in minutes and the slope is in minutes per input-distance unit. The exact fit follows from this authored straight-line fixture; it is not evidence that real delivery durations follow that relationship or that distance causes every delay. A line may miss nonlinear patterns, interactions or important omitted features. Extending beyond the observed input range is extrapolation and needs particular care.

StatQuest's Linear Regression, Clearly Explained!!! provides additional visual explanation. The coordinate graphic at the beginning is a generic reference, not our data plot; use the explicit pairs and equation to draw this course's original example. The written methods remain complete without video playback.

Model selection needs separated evidence. Our tiny classification validation pairs are two-zero and three-one. The majority baseline gets one of two correct. The cut-two stump also gets one of two correct, but makes a different error. One-nearest-neighbour gets both correct and is selected by validation correct count. A tie between candidate scores would choose the earlier predeclared candidate. None of these tiny results establishes reliability outside the fixture.

For regression validation inputs one half and two and a half, the authored targets are six and ten minutes. The mean baseline predicts eight and eight, so both absolute errors are two. The fitted line predicts six and ten, so both errors are zero. Nearest neighbour predicts five and nine under the stated tie rule, giving errors one and one. Select the line using the smaller validation mean absolute error, with the earlier candidate as a score tie-break. The classification and regression tasks choose different candidates for different evidence and target meanings.

Think first: why does a perfect training stump not identify the correct label everywhere between one and four? Hint: more than one cut has the same training mistakes. Partial check: the tie rule chooses cut two, but a new target label at count two can disagree with it. Explain why validation helps a decision without becoming a guarantee.

Execute comparisons and report held-out limits

The complete original program below fits candidates from the training pairs, compares them on validation pairs, fixes each task's chosen family, and only then prints test reports. For teaching it prints final test reports for all predeclared candidates as a retrospective comparison; those reports do not change the frozen selection. Choosing a new winner after seeing them would be test-informed redevelopment and would need appropriate new final evidence.

"""Original tiny classification and regression comparisons with exact arithmetic."""
import json
from fractions import Fraction as F


def checked_pairs(rows,task):
    if task not in ["classification","regression"] or not isinstance(rows,list) or not 1 <= len(rows) <= 100:
        raise ValueError("Expected a bounded nonempty supervised task")
    for r in rows:
        if not isinstance(r,list) or len(r)!=2:
            raise ValueError("Expected feature and target pairs")
        x,y=r
        if type(x) not in [int,F] or not 0 <= x <= 20:
            raise ValueError("Invalid numeric feature")
        if task=="classification" and (type(x) is not int or type(y) is not int or y not in [0,1]):
            raise ValueError("Classification uses integer counts and binary targets")
        if task=="regression" and (type(y) not in [int,F] or not 0 <= y <= 100):
            raise ValueError("Invalid numeric regression target")


def fit_classification(train):
    checked_pairs(train,"classification")
    majority=int(2*sum(y for x,y in train)>len(train))
    cut=min(range(22),key=lambda c:(sum(int(x>=c)!=y for x,y in train),c))
    return [dict(name="majority",value=majority),dict(name="stump",cut=cut),
            dict(name="nearest",points=[r[:] for r in train])]


def fit_regression(train):
    checked_pairs(train,"regression")
    n=len(train)
    sx=sum(F(x) for x,y in train)
    sy=sum(F(y) for x,y in train)
    sxx=sum(F(x)*F(x) for x,y in train)
    sxy=sum(F(x)*F(y) for x,y in train)
    denominator=n*sxx-sx*sx
    if denominator==0:
        raise ValueError("The line needs at least two distinct feature values")
    slope=(n*sxy-sx*sy)/denominator
    intercept=(sy-slope*sx)/n
    return [dict(name="mean",value=sy/n),dict(name="line",slope=slope,intercept=intercept),
            dict(name="nearest",points=[r[:] for r in train])]


def predict(model,x):
    if type(x) not in [int,F] or not 0 <= x <= 20:
        raise ValueError("Invalid prediction feature")
    name=model["name"]
    if name in ["majority","mean"]:
        return model["value"]
    if name=="stump":
        return int(x>=model["cut"])
    if name=="line":
        return model["intercept"]+model["slope"]*x
    if name=="nearest":
        index,row=min(enumerate(model["points"]),key=lambda item:(abs(item[1][0]-x),item[1][0],item[0]))
        return row[1]
    raise ValueError("Unknown internally fitted candidate")


def classification_report(model,rows):
    checked_pairs(rows,"classification")
    pairs=[(y,predict(model,x)) for x,y in rows]
    tp=sum(y==1 and p==1 for y,p in pairs)
    tn=sum(y==0 and p==0 for y,p in pairs)
    fp=sum(y==0 and p==1 for y,p in pairs)
    fn=sum(y==1 and p==0 for y,p in pairs)
    return dict(name=model["name"],predictions=[p for y,p in pairs],tp=tp,tn=tn,fp=fp,fn=fn,
                correct=tp+tn,total=len(rows),accuracy=str(F(tp+tn,len(rows))))


def regression_report(model,rows):
    checked_pairs(rows,"regression")
    predictions=[F(predict(model,x)) for x,y in rows]
    errors=[abs(p-F(y)) for p,(x,y) in zip(predictions,rows)]
    return dict(name=model["name"],predictions=[str(p) for p in predictions],
                absolute_errors=[str(e) for e in errors],mae=str(sum(errors)/len(rows)))


def demonstrate():
    class_train=[[0,0],[1,0],[4,1],[5,1]]
    class_validation=[[2,0],[3,1]]
    class_test=[[2,1],[6,0]]
    regression_train=[[0,5],[1,7],[2,9],[3,11]]
    regression_validation=[[F(1,2),6],[F(5,2),10]]
    regression_test=[[4,16],[5,14]]
    cm=fit_classification(class_train)
    rm=fit_regression(regression_train)
    cv=[classification_report(model,class_validation) for model in cm]
    rv=[regression_report(model,regression_validation) for model in rm]
    ci=max(range(3),key=lambda i:(cv[i]["correct"],-i))
    ri=min(range(3),key=lambda i:(F(rv[i]["mae"]),i))
    return dict(classification=dict(models=cm,validation=cv,selected=cm[ci]["name"],
                test=[classification_report(model,class_test) for model in cm]),
                regression=dict(models=rm,validation=rv,selected=rm[ri]["name"],
                test=[regression_report(model,regression_test) for model in rm]),
                authored_fixtures=True,selection_used_test=False,real_spam_or_delivery_trial=False,
                serialized_models_accepted=False,external_ai_used=False)


if __name__=="__main__":
    print(json.dumps(demonstrate(),default=str,indent=2))

The selected classification candidate is nearest neighbour. Its two test pairs are feature two with authored target one, and feature six with target zero. It predicts zero and one, so both predictions are wrong: one false negative and one false positive, with zero correct out of two. The majority and stump each get one of two correct in this retrospective test report. Do not now call either an independently selected winner. The point is that the validation choice can fail on new cases, especially with very little evidence and inadequate features.

The repeated count two appears in validation and test with different authored targets. These are declared separate synthetic cases sharing one feature value, not a duplicated real message with a hidden identity. This shows that a single count need not determine the target. In real data, genuine related observations must be identified and grouped appropriately; matching a feature value is not by itself proof that two records are the same source or that they are independent.

For regression, the selected line predicts thirteen minutes at input four and fifteen at input five. The separate authored test targets are sixteen and fourteen. Absolute errors are three and one minutes, so the mean absolute error is two minutes. The mean baseline has errors eight and six, giving seven minutes MAE. Nearest neighbour predicts eleven for both and has errors five and three, giving four minutes MAE.

$M A E = \frac{1}{n} \sum_{i = 1}^{n} | y_{i} - {\hat{y}}_{i} |$

Regression candidate	Validation MAE in minutes	Final test MAE in minutes
Training mean baseline	2	7
Fitted line	0	2
One nearest neighbour	1	4

The training inputs range from zero to three, so both test inputs four and five are outside that range. State this extrapolation limit. The perfect training line and zero validation error do not imply zero test error. MAE is an average magnitude, so retain individual errors too; an average can hide a large or consequential miss. A different application might need another metric, error-cost model or an abstention boundary agreed before selection.

The program uses exact rational arithmetic for regression fitting, predictions and errors. It rejects empty or malformed training pairs and incompatible feature or target types. A fitted line requires at least two distinct feature values; if they are all equal, the slope denominator is zero and the request is rejected rather than silently fabricating a line. The prediction functions are for the internally fitted candidates only. The program does not accept untrusted serialized models, import code from data or serve a public prediction API.

A useful comparison report states purpose, target units or categories, permitted features, training and evaluation separation, baseline, candidate assumptions, selection measure, actual outputs and limits. A tree can make understandable rules but may overfit. Neighbours need a meaningful distance and relevant stored examples. A line is compact and interpretable but can be misspecified. All can fail under changed conditions. These considerations refute a universal best-family claim without pretending this tiny exercise ranks all algorithms.

Transfer from a binary message-review flag to delivery time changes target type, metric and error consequences. Transfer to a permissioned greenhouse review task must revisit label meaning, input availability, seasonal coverage, group or time separation, privacy and a human review path. For a new numerical irrigation amount, check units and safety limits before considering regression. These are proposed follow-on investigations, not tasks run by the original program or automated decisions about people or living things.

Independent review: write one comparison paragraph containing a metric with units or class definition, the corresponding baseline, a concrete held-out error and at least one limitation. Hint: two test cases cannot support a precise claim about a real population. Partial check: a line's two-minute MAE belongs to this authored pair of test cases, while the nearest-neighbour classification failure belongs to its separate task. Neither result selects a universally best model.

Interactive Tasks

Quiz: Test Your Knowledge

Which source task is classification? (Spam versus not spam) (!Delivery time in minutes) (!Weekly energy in kilowatt-hours) (!Rainfall in millimetres)

Which source task is regression? (Delivery time in minutes) (!Tool type identified by category codes) (!Spam versus not spam) (!A yes or no review flag)

Why can a numeric category code still be classification? (The number identifies a class rather than a numerical quantity) (!Every integer automatically means regression) (!Category codes always measure distance) (!Classification cannot use numeric features)

What is the fitted regression mean baseline in the example? (8 minutes) (!5 minutes) (!2 minutes) (!0 minutes)

Why does the original stump choose cut two? (It is the smallest cut among tied zero-training-error candidates) (!The test labels chose it) (!Every dataset requires cut two) (!It is a full general decision-tree library)

How is a nearest-neighbour equal-distance tie resolved here? (Prefer the smaller training feature then the earlier row) (!Always choose the larger class label) (!Use the final test outcome) (!Randomly change the answer until correct)

Which family is selected by the classification validation cases? (One nearest neighbour) (!The majority baseline) (!The restricted stump) (!A fitted delivery-time line)

What happens to the selected classification method on its two test cases? (It makes one false positive and one false negative) (!It correctly classifies both) (!It has measured real spam accuracy) (!It becomes the universal best classifier)

What is the selected line's test mean absolute error? (2 minutes) (!0 minutes) (!4 minutes) (!7 minutes)

What does a final test comparison permit? (A bounded report of the predeclared candidates and limitations) (!Untouched winner selection after inspecting every test outcome) (!A claim that two cases certify deployment) (!Automatic use of private messages for training)

Memory Game

Classification	Predict a declared category
Regression	Predict a numerical quantity
Stump	A tree restricted to one split
Neighbour	A nearby training example under a declared distance
Residual	An observed target minus a numerical prediction
Extrapolation	Prediction beyond the observed input range

Drag and Drop

Spam versus not spam	A classification target
Delivery minutes	A regression target with meaningful numerical errors
Training mean eight	The original regression baseline
Same training score for several cuts	Parameter choice can remain uncertain in a data gap
Retrospective test report	Do not use it to redefine an untouched selected winner

Crossword Puzzle

CLASSIFICATION	Prediction of categories
REGRESSION	Prediction of numerical quantities
BASELINE	A simple comparison method
NEIGHBOUR	A nearby training example
RESIDUAL	Target minus numerical prediction
EXTRAPOLATION	Prediction beyond the observed input range

LearningApps

Optional extra practice. Choose an appropriate activity with your teacher. If the external collection is unavailable, use the native activities and independent tasks in this course.

Cloze Text

A category target calls for

. A numerical quantity calls for

. Category codes do not automatically become numerical

. The training mean is a regression

. The one-split tree is a

. Neighbours depend on a declared

. The fitted line has slope

. Its intercept is

. Candidate choice uses

. Final test errors do not prove universal

. Prediction beyond the observed range is

. MAE retains the target's

.

Open Tasks

Easy

Identify six target types: Classify the six proposed targets in the first chapter before checking the Learning Check. Explain why category codes do not imply regression.
Compute the two baselines: Use training labels zero zero one one for the majority tie rule and targets five seven nine eleven for the regression mean.
Read the fitted line: Use intercept five slope two to predict at inputs one and two. Give units and explain what was actually fitted.
Read a classification error: For test input six target zero and nearest prediction one, identify the error type and the declared positive class.

Standard

Explain a training gap: Show why stump cuts two three and four have the same training errors and why the smallest-cut rule does not prove the true label in the gap.
Apply the neighbour tie rule: For regression input one half, identify equally close training features, apply the stated tie rule and calculate the target error.
Compare validation candidates: Use both validation tables to justify the fixed selections and state which baseline belongs to each task.
Calculate regression test MAE: Work through the line baseline and nearest-neighbour predictions on inputs four and five and retain individual errors as well as averages.

Advanced

Critique retrospective winner selection: Explain why final test reports for all candidates must not redefine the winner while calling the evidence untouched. Propose a recorded redevelopment and fresh-evaluation path.
Investigate conflicting feature matches: Explain why the shared feature count two does not uniquely determine a label. Distinguish matching feature values source identity grouping and sufficient features.
Test a degenerate line fit: Design an all-equal-input training case, state why the slope is not identified, and explain an appropriate rejection or separately declared alternative model.
Transfer model and metric carefully: Move from a message review flag to a numerical delivery or greenhouse target. Recheck units input timing permissions grouping metric costs coverage and human review.

Learning Check

Check six task types: Identify all six task types and apply the editorial five-of-six target.
Check fitted candidates and assumptions: Explain the classification tie and the original fitted regression models.
Check validation choices: Give the actual validation evidence without using the final test to choose.
Check held-out errors: Interpret both tasks' final results and the regression arithmetic.
Check boundary and ambiguity claims: Explain all-equal regression inputs the data gap and shared feature count.
Write a bounded comparison: Give a metric baseline limitation and transfer plan for a new task.

Worked checks and assessment guidance

Spam versus not spam is classification; delivery minutes regression; tool type hammer saw or drill classification; weekly energy in kilowatt-hours regression; plant needs review yes or no classification; rainfall in millimetres regression. Correct at least five of these six and add a metric baseline limitation comparison. This is an editorial exercise target, not a qualification or deployment approval.
The majority baseline chooses zero from tied class counts. Integer cuts two three four all fit the four classification training pairs without error, and the smallest-cut rule chooses two. One neighbour uses absolute distance with lower-feature then earlier-row ties. Regression targets average eight; least squares fits intercept five slope two. Neighbour regression retains the permitted pairs. None of these assumptions is universally suitable.
Classification correct counts are one of two for majority and stump and two of two for nearest, selecting nearest. Regression MAEs are two minutes for the mean zero for the line and one for nearest, selecting the line. Fit parameters use training only and these choices use validation only. Final reports happen afterward and cannot become an untouched selection step.
Nearest classification predicts zero and one for targets one and zero, giving one false negative one false positive and zero correct of two. Retrospective majority and stump reports each show one correct of two without changing the fixed winner. The line predicts thirteen and fifteen versus targets sixteen and fourteen, so errors three and one average two minutes. Baseline errors eight six average seven; nearest errors five three average four.
All-equal feature values give a zero slope denominator for this line fit and are rejected; a constant alternative needs its own declared contract. Multiple perfect stump cuts leave new labels in the training gap uncertain. The same count two can describe different synthetic cases with different targets, so one feature is insufficient; real source relationships need evidence beyond matching feature values. Both regression test inputs extrapolate beyond training zero through three.
State category meaning or numerical units, permitted inputs, split and baseline. Report actual held-out errors and small authored sample limits. Revisit target type model assumptions distance or units metric error consequences grouping timing rights and review for the new context. Do not call a perfect training fit a generalisation guarantee or a tiny test a universal ranking; any redevelopment informed by test results requires appropriately new final evidence.

Learning evidence

Retain the six task classifications with the editorial tally, both training dictionaries and split fixtures, fitted parameters and tie rules, separate validation comparisons and fixed chosen families. Add final per-case predictions confusion counts numerical errors and MAE units, including the extrapolation and shared-feature ambiguity limits. Record retrospective comparisons without selecting a new untouched winner. Include a boundary rejection and an evidence-labelled transfer plan with permissions and human review.

OER and Wikipedia

DfE GCSE Computer Science subject content, fitness-for-purpose evaluation anchor inspected 15 November 2026. The AI Year 10 candidate comparison and five-of-six exercise target are editorial enrichment rather than statutory extra AI requirements.

scikit-learn decision-tree documentation, nearest-neighbour documentation and linear-model documentation, relevant task and model-family sections inspected 15 November 2026. Our original standard-library program implements a restricted error-fitted classification stump, one-neighbour methods and exact single-feature least squares. It does not import the library or claim to reproduce a full general tree algorithm, every estimator or a production evaluation.

2D Cartesian Coordinates.svg, H Padleckas, public domain, unchanged generic coordinate reference. It is not a plot of our fixture or fitted line. Exact metadata retained preview bytes and native embedding checked; preview visually inspected. All coordinates and numerical results are readable in the written examples.

https://www.youtube.com/watch?v=nk2CQITm_eo — StatQuest with Josh Starmer: Linear Regression, Clearly Explained!!! https://www.youtube.com/watch?v=OeU5m6vRyCk — CodeAI: AI: What is Machine Learning? Actual retained metadata embedding availability and preview bytes checked and previews visually inspected. Unchanged external embeds imply no transcript copying endorsement or relicensing. The course provides the complete required methods without playback.

All training validation and test pairs code tie rules scenarios and calculations are original authored constructions. Execution uses exact rational arithmetic and no real messages delivery service data private pupil information local generative LLM or paid AI API. A retrospective test comparison does not alter the already fixed validation choice. Author checks are not independent expert certification.