English:Communicating model uncertainty to users - AI Year 6
Introduction

A confident-looking score is not enough. Build six clear messages that distinguish supported records, conflicting evidence, missing inputs and out-of-scope tasks. Run a complete original policy program, compare an overconfident baseline and design a meaningful way to ask for help.
Explain uncertainty without inventing certainty
Help a user decide what to do next
A useful uncertainty message says what the system suggests, what evidence or limitation matters, and what the user can do next. It should not merely display a reassuring number or a vague warning at the bottom of the page. The purpose is an appropriately informed decision, not making the system sound confident or persuading someone to accept every result.
Our fictional classroom museum-caption helper suggests bowl or vase for an invented item record. It has not inspected a real object or run a real image model. The candidate labels and scores below are supplied teaching examples. The separate evidence field represents a teacher-prepared review of a fictional reference card: support, conflict or none. A real product would need to obtain and check such evidence; the policy cannot manufacture it.
Keep three questions separate. What label is proposed? Does the input fall within the declared task? What does the inspected reference support? A score may describe a model's internal ranking or another measure, but its meaning must be established before you interpret it. Here scores from zero to one hundred are invented metadata, not calibrated probabilities of correctness.
A high score cannot overrule missing evidence
In C2 the supplied score is 99, but the candidate bowl conflicts with the fictional reference card, which says vase. The responsible message flags the conflict instead of announcing 99 percent certainty. In C5 the score is also 99, but the input lies outside this caption task's declared coverage. Neither number fixes the context problem.
An overall error table can help explain what types of mistakes occurred in a test set, but it cannot by itself tell you the chance that one new caption is correct. The opening confusion-matrix illustration shows error categories, not our six invented records or a validated probability estimate. Do not convert a diagram, a ranking score or a reassuring tone into evidence the source did not provide.
CodeAI: How Chatbots and Large Language Models Work. With teacher support, discuss why fluent generated language is not a factual guarantee. Our caption review policy is hand-written and its examples are prepared; the video does not supply model scores or prove that a real caption model was run.
Build six evidence-aware messages and test their policy
Read the six complete review records
The task is to prepare a classroom caption for teacher review, not automatically publish a museum description. Input-present and coverage fields must be checked before interpreting the label. C1 and C6 have fictional records supporting bowl. C2 has a conflicting vase reference. C3 has no reference, C4 lacks required input, and C5 is outside scope.
| Case | Input present | Coverage | Reference review | Invented score | Candidate | Expected state |
|---|---|---|---|---|---|---|
| C1 | True | within | support | 92 | bowl | SUPPORTED_IN_RECORD |
| C2 | True | within | conflict | 99 | bowl | CHECK_CONFLICT |
| C3 | True | within | none | 73 | vase | UNVERIFIED |
| C4 | False | within | support | 95 | bowl | NEED_INPUT |
| C5 | True | outside | support | 99 | bowl | OUT_OF_SCOPE |
| C6 | True | within | support | not supplied | bowl | SUPPORTED_IN_RECORD |
Try writing a short useful message for each case before reading the policy table. Hint: start with the problem that prevents interpretation. Partial step: C4 needs the missing input first, even though its leftover metadata includes a large score and a support flag. A stale support flag is not a substitute for the required current input.
Connect every state with an explanation and action
| Policy state | Plain-language message | Useful next action |
|---|---|---|
| SUPPORTED_IN_RECORD | The supplied worksheet record supports this candidate. | Inspect the record and context before using the caption; this is not a general guarantee. |
| CHECK_CONFLICT | The candidate conflicts with the supplied reference. | Keep it unconfirmed and ask a teacher to compare the exact records. |
| UNVERIFIED | No supporting reference is supplied. | Find and inspect appropriate evidence or leave the caption unverified. |
| NEED_INPUT | The required input is missing. | Provide the allowed fictional input before interpreting a result. |
| OUT_OF_SCOPE | This input is outside the declared task coverage. | Do not apply the caption; ask for an appropriate method or human review. |
| CHECK_METADATA | The supplied review fields are not valid for this policy. | Check the worksheet metadata; do not invent a confidence or evidence verdict. |
SUPPORTED_IN_RECORD is deliberately limited wording. It says what this supplied record supports, not that the object, every future input or the model as a whole has been verified. The learner still needs to inspect the actual card and task context. The other states explain different barriers rather than put every problem behind one ambiguous error label.
Trace the original policy program
The complete Python below was actually executed during preparation. It selects a message state from supplied review metadata; it does not classify an image, retrieve evidence or run a language model. An omitted score is allowed and is never invented. An invalid score is flagged as a metadata problem for within-scope cases. Missing-input and out-of-scope checks have priority over interpreting the candidate.
def status(present, scope, evidence, score, candidate):
if type(present) is not bool:
return "CHECK_METADATA"
if not present:
return "NEED_INPUT"
if scope not in ("within", "outside"):
return "CHECK_METADATA"
if scope == "outside":
return "OUT_OF_SCOPE"
if evidence not in ("support", "conflict", "none"):
return "CHECK_METADATA"
if score is not None and (type(score) is not int or not 0 <= score <= 100):
return "CHECK_METADATA"
if candidate not in ("bowl", "vase"):
return "CHECK_METADATA"
if evidence == "conflict":
return "CHECK_CONFLICT"
if evidence == "none":
return "UNVERIFIED"
return "SUPPORTED_IN_RECORD"
cases = [
("C1", True, "within", "support", 92, "bowl"),
("C2", True, "within", "conflict", 99, "bowl"),
("C3", True, "within", "none", 73, "vase"),
("C4", False, "within", "support", 95, "bowl"),
("C5", True, "outside", "support", 99, "bowl"),
("C6", True, "within", "support", None, "bowl"),
]
for row in cases:
print(row[0], status(*row[1:]))Expected states in order are SUPPORTED_IN_RECORD, CHECK_CONFLICT, UNVERIFIED, NEED_INPUT, OUT_OF_SCOPE, SUPPORTED_IN_RECORD. C6 lacks a score but retains its record-support explanation. It must not display a made-up percentage. Candidate and evidence fields remain distinct; the program does not independently verify that a supplied support flag is true.
Compare with an overconfident baseline
The deliberately poor baseline says verified, use this caption for all six cases. It hides four known barriers: C2's conflict, C3's absent evidence, C4's missing input and C5's coverage problem. Even for C1 and C6, verified without the record/context limitation overstates what was checked. A useful comparison reports those actual communication failures rather than invent a measured increase in user trust.
Our six-case program demonstrates a consistent finite policy. It does not show that real users understood the messages or made better decisions. That would require an appropriately planned user study, with accessible materials and permission where relevant. Keep the distinction between correct program output, clear wording and observed human understanding.
Check boundaries access and real user understanding
Test boundaries instead of worshipping a score
Hold C2's conflict fixed and change its supplied score from zero to one hundred. Every valid score still gives CHECK_CONFLICT. An extreme score does not make the reference agree. Likewise, C6 with no score still gives SUPPORTED_IN_RECORD because the evidence field, not a percentage threshold, supports that limited statement.
Test an unknown evidence value or a candidate outside bowl/vase: the within-scope policy must return CHECK_METADATA rather than guess an explanation. If the input is absent, ask for it; if the task is outside scope, do not apply the caption. These are design choices for our declared workflow, not a general formula for all uncertainty or high-stakes decisions.
Make the explanation perceivable and manageable
Use short descriptive text with a clear next action. Do not rely only on a red or green dot, a star or a hover message that a touch-screen user may not discover. In a real interface, important status changes should also be available to assistive technology without unnecessarily moving focus or interrupting every small step. Appropriate implementation and user testing are still needed; a printed policy table is not an accessibility certification.
W3C Web Accessibility Initiative: Web Accessibility Perspectives, compilation of ten topics. Choose clear-layout, keyboard or feedback examples to connect the message with an access need. The recording is context, not a completed test of our prototype or evidence that a colour alone communicates uncertainty.
Keep optional detail available without hiding the main limitation. A user should be able to find the supplied source, the reason for uncertainty and a way to ask for help. Human review requires time, relevant evidence and authority to reject or correct the suggestion. A compulsory final click with no explanation is not meaningful oversight.
Transfer the message pattern to another learning task
For a fictional reading recommendation, a high ranking score does not establish source reliability or the learner's complete interests. State which signals were used, the current learning goal and available controls. The museum's bowl/vase scope and reference states would need redesign. Preserve the pattern of suggestion, evidence/limit and next action, not the old labels blindly.
Build an evidence log with all six inputs, expected and actual states, six written messages, extreme-score and missing-score tests, and the overconfident-baseline comparison. Add two limits: the review fields were supplied rather than independently obtained, and real user understanding has not been tested. Propose a small comprehension study in which a learner explains when to check a source or ask a teacher; label that proposed study unperformed.
No pupil identities, private images or sensitive work are needed. Use original fictional records and an approved classroom document or programming environment. Your final product should help someone recognise when information is limited and how to respond, rather than simply look more persuasive. Clear uncertainty supports a defensible decision; it does not eliminate error.
Interactive Tasks
Quiz: Test Your Knowledge
What should a useful uncertainty message provide? (A suggestion its important limit and a next action) (!Only a reassuring colour) (!Only a large score with no meaning) (!A guarantee that no review is needed)
What do the supplied scores mean in this lesson? (Invented metadata with no calibrated probability claim) (!Measured chances that each real object was classified correctly) (!A completed user-study result) (!Proof that every reference was opened)
Which state does C2 require despite its score of ninety-nine? (CHECK_CONFLICT) (!SUPPORTED_IN_RECORD) (!NEED_INPUT) (!An automatic public caption)
What is missing in C3? (A supporting reference) (!A candidate label) (!A supplied score) (!The entire declared task)
What takes priority for C4? (Ask for the required missing input) (!Trust leftover support metadata) (!Treat the large score as a current image) (!Publish the caption first)
Why is C5 not suitable for this caption workflow? (Its input is outside declared coverage) (!Its score is too low) (!It has no label) (!All museum objects are unsafe)
What should C6 show about the absent score? (No invented percentage while retaining the limited support message) (!A fabricated score of one hundred) (!A claim that evidence is impossible without a number) (!A hidden certainty badge)
Does a correct six-case policy prove user comprehension? (No an appropriate actual user study is still needed) (!Yes every reader must understand it) (!Yes a diagram proves comprehension) (!Yes if the wording looks professional)
Why should a message not rely only on hover? (Touch and other input modes may not expose the information easily) (!Every device uses a mouse) (!Hover proves a source is reliable) (!A hidden message is always clearer)
What must change when transferring to reading recommendations? (Redefine relevant signals scope and controls for the new need) (!Reuse bowl and vase labels unchanged) (!Treat popularity as factual proof) (!Assume old tests establish all new outcomes)
Memory Game
| Candidate | The proposed label or answer |
| Scope | The task conditions a method is declared to cover |
| Evidence | Relevant material inspected to support or challenge a claim |
| Unverified | Not established by the evidence available |
| Calibration | Whether stated probabilities correspond appropriately to observed outcomes |
| Human review | A meaningful opportunity to inspect evidence and disagree |
Drag and Drop
| A large score with conflicting evidence | Flag the conflict and request a source comparison |
| A missing required input | Ask for the input before interpreting a result |
| A candidate outside task coverage | Do not apply the old method to the new context |
| A record supporting a candidate without a supplied score | Retain the limited support message without inventing a percentage |
| A proposed comprehension study | Do not report improved user understanding before the study occurs |
Crossword Puzzle
| SCOPE | What defines the task conditions a method covers? |
| INPUT | What required information is absent in C4? |
| CONFLICT | What disagreement must be flagged in C2? |
| SOURCE | What should a reviewer be able to inspect? |
| ACTION | What useful next step should a message offer? |
| REVIEW | What human route permits an informed challenge? |
LearningApps
Optional extra practice. Choose an appropriate activity with your teacher. If the external collection is unavailable, use the native activities and independent tasks in this course.
Cloze Text
Open Tasks
Easy
- Find three message parts: Mark suggestion, limitation and next action in a short original example.
- Explain the score caution: Explain why the invented 99 does not overrule C2 reference conflict.
- Write a missing-input message: Give C4 a useful next action without interpreting leftover metadata as evidence.
- Retain a missing-score limit: Write C6 without inventing a percentage or claiming universal verification.
Standard
- Complete six case reviews: Trace all expected states and write an accurate action-oriented message for each.
- Challenge the baseline: Identify four barriers hidden by verified-use-this for every case and explain the overstatement in the two supported-record cases.
- Test all valid scores for a conflict: Hold other C2 fields fixed and confirm that extreme scores do not change the evidence verdict.
- Design accessible messages: Specify visible text, clear controls and assistive-technology status needs; label implementation and user testing as still required.
Advanced
- Inspect untrusted review metadata: Explain why the policy cannot manufacture independent evidence from a supplied support flag.
- Plan a comprehension study: Ask someone to explain when to check a source or seek help; specify appropriate conditions and label the proposed study unperformed.
- Transfer to a reading recommendation: Redefine signals, evidence, scope, user controls and possible errors for a different learning need.
- Prepare a bounded evidence handoff: Retain all six records, actual states, messages, boundary tests, baseline comparison, two limits and meaningful human review.
Learning Check
- Explain the communication goal: Separate useful information from persuasion.
- Review all six records: Give states with case-specific reasons.
- Interpret the boundary test: Change score while preserving evidence.
- Compare the baseline and evidence limits: Evaluate a purported improvement.
- Design an accessible human route: Specify what the user can perceive and change.
- Transfer responsibly: Redesign the explanation for a reading recommendation.
Worked checks and assessment guidance
- A message should state the candidate, the evidence or relevant limitation and the next action. Here model scores are invented uncalibrated metadata. A high number or fluent sentence cannot establish that the claim or reference is correct, and a confusion matrix alone does not give a new case probability.
- C1 SUPPORTED_IN_RECORD for its supplied support; C2 CHECK_CONFLICT despite 99; C3 UNVERIFIED without evidence; C4 NEED_INPUT before interpreting leftover metadata; C5 OUT_OF_SCOPE despite 99; C6 SUPPORTED_IN_RECORD with no score invented. Both supported states retain the worksheet-record limitation rather than certify the model.
- Every integer score from 0 through 100 leaves a within-scope conflict as CHECK_CONFLICT. Missing score may be allowed, invalid score gives CHECK_METADATA in an otherwise valid within-scope case. Unknown evidence/candidate fields must not be guessed. The fixed policy, not a measured model-confidence calibration, produced these states.
- Verified-use-this hides C2 conflict, C3 absence, C4 missing input and C5 scope problem; it also overstates the two supplied-record support cases. The new policy documents those distinctions, but no actual human-comprehension or trust study was run. Correct code output and useful user understanding are different evidence types.
- Use meaningful visible text and a next-action control, not colour or hover alone. Real dynamic status must be available to assistive technology with appropriate implementation/testing. Human reviewers need source access, time and authority to disagree or stop; a final click alone is insufficient.
- Describe actual ranking signals, incomplete preference inference, source reliability and user controls for the new learning goal. Replace museum labels/scope with the new dictionary and tests. Keep six-case evidence, two limitations and an unperformed study plan, avoiding private learner data and unsupported certainty claims.
Learning evidence
Retain all six synthetic records, expected/actual policy states, six written messages with next actions, all-score conflict boundary and missing/invalid-score tests. Include the overconfident-baseline critique, access specification, context transfer, human-review route and two explicit limits: supplied review metadata and untested user comprehension. Actual program execution is not image classification, a live model run, calibrated probability, interface certification or a completed user study.
OER and Wikipedia
Microsoft HAX: communicate system performance and Google People+AI: explainability and trust inform the limited introductory advice about expectations and meaningful confidence displays. The six records, all scores, messages, policy, program and tests are original synthetic classroom materials, not publisher model results or user-study findings.
W3C: understanding status messages informs the short accessibility implementation note. DfE Computing supplies the KS2 programming anchor; this specific AI communication sequence is editorial. Sources read 15 November 2026. No actual interface accessibility conformance or human comprehension study is certified.
ConfusionMatrix.png, Jackverr, CC BY-SA 3.0. Original illustration unchanged; error-category context, not the six-case dataset or a calibrated score.
https://www.youtube.com/watch?v=X-AWdfSFCHQ — CodeAI: How Chatbots and Large Language Models Work.
https://www.youtube.com/watch?v=3f31oufqFSM — W3C Web Accessibility Initiative: Web Accessibility Perspectives, compilation of 10 topics/videos. Both original recordings embedded unchanged with actual verified playability/embed/preview evidence; teacher-supported context, no live learner account required.
Linked learning areas
aiMOOC Projects
Schulfach+


aiMOOCs


aiMOOC Projekte


NEWSLernweltNOAH fragen