English:Machine Learning Foundations
Machine Learning Foundations
Introduction
Machine learning is a branch of artificial intelligence in which computer systems learn patterns from data so that they can make predictions, classifications, recommendations, or other decisions without every rule being written by hand. In this aiMOOC for Grades 11–13, you will learn the foundations needed to reason about machine-learning systems rather than treating them as black boxes.
By the end of the course, you should be able to distinguish major learning settings, explain a standard machine-learning workflow, interpret simple models and evaluation metrics, recognize overfitting, and discuss responsible use. You will also design and evaluate small investigations of your own.
A useful first distinction is between supervised learning, where training examples include target labels, and unsupervised learning, where the algorithm searches for structure without target labels. Modern systems may also use other paradigms, but these two provide a strong foundation for further study.
What a Machine-Learning System Learns
A machine-learning model is a mathematical function whose behavior depends on adjustable parameters. Training means choosing parameter values that make the model perform well on examples. The input variables are called features. In supervised learning, the value the model is asked to predict is the label or target.
Suppose you want to predict the energy use of a school building. Features might include outside temperature, floor area, day of the week, and occupancy. The target could be electricity consumption in kilowatt-hours. A model does not understand these variables in the human sense. It detects statistical relationships represented in the training data.
A basic workflow is:
- Data collection: Define the question and gather examples that are relevant to it.
- Data preprocessing: Check data types, missing values, outliers, labels, scales, and possible leakage.
- Feature engineering: Choose or construct representations that make useful information available to the model.
- Model training: Fit parameters using a training set and an objective such as a loss function.
- Model validation: Tune choices such as model complexity or learning rate using validation data or cross-validation.
- Model evaluation: Estimate performance on untouched test data with metrics suited to the real task.
- Model deployment: Use the model carefully, monitor outcomes, and update or retire it when conditions change.
Data, Features, Labels, Parameters, and Hyperparameters
Data are the examples from which a system learns or on which it is evaluated. A feature is an input variable given to the model. A label is the desired output in supervised learning. A parameter is learned during training, such as the slope of a regression line. A hyperparameter is chosen outside the fitting process, such as the number of clusters in k-means or a regularization strength.
These distinctions matter because they help you ask precise questions. If a model performs poorly, is the problem the data, the representation, the training process, the model family, a hyperparameter choice, or the evaluation method?
Supervised Learning
In supervised learning, each training example includes input information and a target. The model tries to learn a mapping from inputs to targets that generalizes to new examples.
Regression
Regression predicts a numerical quantity. A simple linear regression model with one feature can be written as:
y-hat = b0 + b1x
Here, x is the feature, y-hat is the prediction, b0 is the intercept, and b1 is the slope. With several features, the model can contain several weighted inputs. Training chooses parameter values that make predictions close to observed targets according to a chosen loss function.
A common regression loss is mean squared error. It averages the squared differences between predictions and true values. Squaring makes large errors count more strongly and prevents positive and negative errors from canceling each other.
When you inspect a regression model, do not rely on a single score. Plot predictions, inspect residuals, and ask whether the data cover the range where the model will be used.
Classification
Classification predicts a category, such as whether an email is spam or not spam. Many classifiers first produce a score or probability-like value and then apply a threshold to choose a class. Changing the threshold changes the types of errors the system makes.
A binary classifier produces four possible outcomes: true positive, true negative, false positive, and false negative. These outcomes form a confusion matrix and support several important metrics.
Accuracy is the fraction of all predictions that are correct. It can be misleading when one class is much more common than the other. Precision asks what fraction of predicted positives are actually positive. Recall asks what fraction of actual positives are detected. The best metric depends on the consequences of different errors.
A ROC curve shows how true-positive rate and false-positive rate change as a classification threshold changes. It is useful for comparing threshold behavior, but model evaluation should always be connected to the real costs, risks, and class balance of the application.
Learning by Minimizing Loss
A loss function turns prediction error into a number that training attempts to reduce. Different tasks use different losses. For regression, mean squared error is common. For classification, losses based on predicted probabilities are often used.
Gradient Descent
Gradient descent is an optimization method used to reduce loss. Imagine standing on a landscape and trying to move downhill. The gradient describes the direction of steepest increase, so moving in the opposite direction tends to reduce the objective.
A simplified update rule is:
new parameter = old parameter - learning rate × gradient
The learning rate controls step size. If it is too large, training may overshoot useful solutions or become unstable. If it is too small, training may progress very slowly.
For complex models, training usually repeats parameter updates over many batches of examples. The goal is not merely to minimize training loss; it is to learn patterns that work on unseen data.
Generalization, Overfitting, and Validation
Generalization means performing well on data that were not used to fit the model. A model that memorizes accidental details of the training set may achieve very low training error while performing poorly on new examples. This is overfitting.
The opposite problem, underfitting, occurs when the model is too simple or insufficiently trained to capture important structure. Good modeling searches for an appropriate balance between fitting the available data and remaining robust to new cases.
Train, Validation, and Test Sets
A common data split has three roles. The training set is used to fit parameters. The validation set is used to compare models or tune hyperparameters. The test set is kept untouched until the end and is used to estimate final performance.
If you repeatedly make choices after looking at test results, the test set begins to influence development and no longer provides a clean final estimate. This is one reason careful experiment design matters.
Cross-Validation
Cross-validation repeatedly divides the available development data into training and validation folds. In k-fold cross-validation, each fold acts as validation data once while the others are used for training. The resulting scores can be averaged to estimate how stable a method is across different splits.
Cross-validation is especially useful when the dataset is not very large. If you use cross-validation to choose a model or hyperparameters, it is still valuable to keep a separate final test set when possible.
Bias and Variance
Bias describes error caused by overly restrictive assumptions or an inability to represent the needed pattern. Variance describes sensitivity to the particular training sample. Increasing model complexity can reduce bias but may increase variance.
The bias-variance idea helps explain why more complexity is not always better. Regularization, more representative data, simpler models, and better validation strategies can all help improve generalization.
Unsupervised Learning and Clustering
In unsupervised learning, the data do not provide target labels for the task. The goal may be to find groups, compress information, detect unusual patterns, or discover useful representations.
K-Means Clustering
K-means partitions data into k clusters. A common version of the algorithm repeats two steps: assign each example to its nearest centroid, then recompute each centroid as the mean of the assigned examples. Iteration continues until assignments or centroids stop changing substantially.
K-means can reveal structure, but its output is not automatically meaningful. You must choose k, and results can depend on initialization, outliers, and the scale of features. It also works best when its distance-based assumptions fit the shape of the data.
Clustering should be interpreted cautiously. A cluster is a mathematical grouping, not proof that a natural or socially meaningful category exists.
Feature Representation and Data Preparation
Models only receive the information represented in their inputs. Poor representation can make an easy problem difficult, while careless preprocessing can leak information that would not be available at prediction time.
Numerical scaling can matter when algorithms use distances or gradient-based optimization. Categorical variables often need an encoding such as one-hot encoding. Missing values require a documented strategy rather than silent deletion. Text, images, and audio need representations that convert raw input into numerical form.
The image illustrates why feature scale can change a distance-based clustering result. Standardization is not automatically required for every model, but you should understand how a model uses numerical magnitude before choosing a preprocessing method.
Data Leakage
Data leakage happens when training uses information that would not legitimately be available when the model makes a real prediction. Examples include calculating preprocessing statistics from the entire dataset before splitting, using future information to predict the past, or including a feature that directly reveals the target.
Leakage can make evaluation results look excellent even though the system will fail in real use. A strong experiment keeps the intended prediction moment clear and applies data-dependent preprocessing within the training process.
Neural Networks as a Foundation for Deep Learning
An artificial neural network is built from layers of computational units. Each unit combines inputs with learned weights and a bias, then applies an activation function. By composing many such transformations, a network can represent complex nonlinear relationships.
The input layer represents features, hidden layers construct intermediate representations, and the output layer produces the prediction. Training typically adjusts weights and biases using gradient-based optimization. A network with many learned layers is commonly described as a deep learning model.
Neural networks are powerful, but they are not automatically the best choice for every dataset. Simpler models can be faster, easier to interpret, and competitive when the data and task are relatively structured.
Responsible Machine Learning
A technically accurate model can still be inappropriate. Responsible machine learning asks whether the system is useful, fair enough for its context, privacy-aware, secure, understandable to affected people, and subject to appropriate human oversight.
Dataset bias can arise when training data underrepresent some situations or record past inequalities. Measurement bias can occur when a convenient variable is only a weak proxy for what you actually care about. Deployment bias can occur when a model is used in a setting different from the one for which it was tested.
Fairness cannot be reduced to one universal number. Different fairness goals may conflict, and the right analysis depends on the social and legal context. You should examine performance across relevant groups when appropriate, document limitations, and ask who benefits, who may be harmed, and who can challenge an automated outcome.
Privacy also matters. Collect only data that are justified, protect sensitive records, and consider whether a model could expose information about individuals. For consequential decisions, model output should support accountable human processes rather than replace them automatically.
A Small End-to-End Example
Imagine that your class wants to predict daily bicycle counts near a school. You could use weather, weekday, school holidays, and time of year as features. The target would be the number of bicycles counted.
First, define the prediction question precisely. Next, inspect the data and decide what information would actually be available before each prediction. Split the data by time if future days are the intended test cases, because a random split could hide time-related leakage. Train a simple baseline, such as predicting the training-set mean, and then compare it with a regression model.
Evaluate the model using a regression metric and residual plots. If performance differs strongly by season, investigate whether useful features are missing. Record the model's scope and limitations. This workflow is more informative than simply choosing the algorithm with the most impressive name.
Sources and Further Learning
For additional practice, you can use the Google Machine Learning Crash Course, which includes interactive exercises on regression, classification, data, overfitting, neural networks, and fairness. The scikit-learn User Guide provides practical documentation for many classical machine-learning methods. The English Machine learning article provides a broad reference overview.
Interactive Tasks
Quiz: Test Your Knowledge
Which description best matches supervised learning? (Learning from examples that include target labels) (!Finding structure only in completely unlabeled data) (!Choosing random rules without data) (!Storing every input without making predictions)
What kind of output is typical of regression? (A numerical quantity) (!A cluster identifier only) (!A confusion matrix) (!A learning rate)
What is the main purpose of a final test set? (To estimate performance on unseen data after model choices are finished) (!To fit the model parameters) (!To create target labels) (!To increase the number of features)
Which situation is a sign of overfitting? (Very low training error but much worse error on new data) (!Similar performance on training and new data) (!A model with no adjustable parameters) (!A dataset with no features)
What does gradient descent try to do during training? (Adjust parameters in a direction that reduces loss) (!Increase the number of labels) (!Turn every feature into a category) (!Replace evaluation with random guessing)
What does precision measure in binary classification? (The fraction of predicted positives that are actually positive) (!The fraction of actual positives that are detected) (!The fraction of all examples used for training) (!The number of clusters found by k-means)
What does recall measure in binary classification? (The fraction of actual positives that are detected) (!The fraction of predicted positives that are correct) (!The average squared regression error) (!The size of a gradient step)
Which value must be chosen for standard k-means clustering? (The number of clusters) (!The true class label for every example) (!The final test accuracy) (!The slope of a regression line)
What is a feature in a machine-learning dataset? (An input variable supplied to a model) (!A guaranteed correct prediction) (!A final evaluation report) (!A type of test-set error)
Which action best supports responsible machine learning? (Evaluate limitations and possible harms in the real use context) (!Use the most complex model regardless of the task) (!Ignore group differences whenever overall accuracy is high) (!Reuse sensitive data without checking whether it is needed)
Memory Game
| Training set | Data used to fit model parameters |
| Validation set | Data used to compare choices during development |
| Test set | Data reserved for final performance estimation |
| Feature | Input variable given to a model |
| Label | Target value in supervised learning |
| Hyperparameter | Setting chosen outside parameter fitting |
Drag and Drop
| Match the correct terms. | Topic |
|---|---|
| Mean squared error | Common regression loss |
| Confusion matrix | Table of binary classification outcomes |
| Cross-validation | Repeated training and validation across folds |
| K-means | Centroid-based clustering method |
| Gradient descent | Iterative loss minimization method |
...
Crossword Puzzle
| Regression | Which learning task predicts a numerical quantity? |
| Classification | Which learning task predicts a category? |
| Clustering | What unsupervised task groups similar examples? |
| Overfitting | What is the term for fitting training details that fail to generalize? |
| Gradient | What mathematical direction is used by gradient descent? |
| Precision | Which metric asks how many predicted positives are correct? |
LearningApps
Cloze Text
Open-Ended Tasks
Easy
- Dataset: Choose a small public dataset, identify its features and possible target, and explain which variables are numerical, categorical, or missing.
- Scatter plot: Create a scatter plot from two numerical variables, describe the visible relationship, and state whether a linear model seems plausible.
- Confusion matrix: Design a one-page infographic that explains true positives, false positives, true negatives, false negatives, precision, and recall with one original example.
- Supervised learning: Produce a 60–90 second video or storyboard that teaches the difference between supervised and unsupervised learning to another student.
Standard
- Linear regression: Fit a simple regression model in a spreadsheet or notebook, separate training from testing, plot predictions and residuals, and explain at least two limitations.
- K-means clustering: Create or find a two-dimensional dataset, run k-means with several values of k, visualize the results, and explain which solution you find most defensible.
- Artificial intelligence: Interview a teacher, student, researcher, or professional who uses an AI-enabled tool, then distinguish what the tool predicts from what the user decides.
- Algorithmic bias: Examine a dataset or realistic case study for representation, measurement, or deployment bias and propose concrete checks that could reduce risk.
Advanced
- Statistical classification: Train two classification models on the same dataset, use cross-validation, select an evaluation metric that fits the scenario, and justify your final model choice.
- Learning curve: Investigate how training and validation performance change as you vary training-set size or model complexity, then relate the pattern to underfitting and overfitting.
- Feature engineering: Compare at least two feature-preparation strategies such as scaling or categorical encoding, keeping the evaluation protocol fixed, and explain why the results change.
- Model card: Create a short model card plus a presentation or video that documents a model's purpose, data, metrics, intended users, limitations, ethical risks, and conditions under which it should not be used.
Learning Assessment
- Problem formulation: Given a real-world scenario, decide whether it is best framed as regression, classification, clustering, or not as a machine-learning task at all, and defend your decision.
- Data leakage: Analyze an experimental design that preprocesses the entire dataset before splitting it, explain why this can inflate results, and redesign the pipeline.
- Evaluation metric: Compare two classifiers with different precision and recall, state which one you would choose under two different cost scenarios, and justify each choice.
- Generalization: Interpret training and validation curves for several model complexities and recommend a model while explaining the bias-variance trade-off.
- Gradient descent: Explain how learning rate affects optimization and predict what could happen when it is extremely small or extremely large.
- Responsible AI: Evaluate a proposed school-related prediction system for purpose, data quality, fairness, privacy, transparency, and human oversight, then recommend whether and how it should be used.
Evidence of Learning
| Evidence area | What successful learning can look like |
|---|---|
| Knowledge | You accurately distinguish supervised and unsupervised learning, regression and classification, parameters and hyperparameters, training and test data, and common evaluation metrics. |
| Reasoning | You can explain why a particular split, metric, feature representation, or model family fits a specific problem and what assumptions it introduces. |
| Practical skill | You can prepare a small dataset, train a baseline model, evaluate it on unseen data, visualize results, and document the experiment so another learner could reproduce it. |
| Products | Your portfolio can include plots, a notebook or spreadsheet model, a confusion-matrix analysis, a clustering visualization, a short explainer, and a model card. |
| Transfer | You can critique an unfamiliar machine-learning claim, identify what evidence is missing, and propose a safer or more informative evaluation. |
| Responsibility | You can recognize leakage, biased measurement, inappropriate metrics, privacy concerns, and deployment risks instead of treating accuracy as the only goal. |
OERs on the Topic
Linked Learning Areas
aiMOOC Projects
MOOCwiki · Deutsch
Nach dem Lernen ist vor dem Lernen
Entdecke direkt den nächsten Lernkurs. Weitere Inhalte erscheinen, wenn Du weiter nach unten scrollst.
Zur MOOCwiki-HauptseiteMediathek
Mediathek
Mediathek wird aus dem Wiki geladen ...
Keine passenden Inhalte gefunden. Bitte ändere Suche oder Filter.