AI & Computer Vision, lesson 37 of 52
Claude as Your Match Commentator
Introduction
Your system knows a goal was scored at 4:10 by the blue team. Wouldn't it be great if it could say so, like a commentator? Today you'll connect your detector to Claude, a large language model, to write live commentary from the facts your code produces, and learn the most important rule of working with AI that writes: it must never be the one deciding what's true.
By the end of this lesson you'll be able to:
- Explain the split: code decides, Claude describes
- Turn detections into a small JSON of facts
- Call the Claude API with a system prompt, and read the reply
- Explain hallucination and check output against the facts
- Keep the match loop fast with a queue and a worker thread
- Handle API keys, privacy and cost responsibly
Working principle
What a language model is good at
Claude is excellent at turning structured facts into good writing: a summary, a commentary line, a match report, an email to a parent. Your CV system is excellent at producing facts. Together: your system sees, Claude tells the story.
camera -> YOLO -> tracker -> rules (lesson 31) -> FACTS (JSON) -> Claude -> wordsFacts in, words out
Claude cannot see your video here. It only knows what you put in the message. This is a feature: the words can only be as true as the facts, so you control the truth.
The system prompt sets the rules
The system message is where you say how to behave: tone, length, and above all what it must not do. Our rules say: use only the facts, never invent names, say so when the facts do not cover something. Clear rules beat clever wording.
Hallucination: fluent, confident, false
A language model writes the most plausible next words. If you ask "who gave the assist?" and the facts say nothing, it may invent a plausible answer. This is called hallucination. Three defences, all used today:
| Defence | How |
|---|---|
| Give it the facts | So it does not need to guess |
| Tell it what to do when it does not know | "Say so briefly" |
| Check afterwards | Code that compares the text to the facts (Cell 4) |
The number checker is crude. It cannot catch a wrong name or a made-up foul. It shows the habit: never publish model output unchecked when it states facts.
Output varies
Ask twice, get two different answers. For commentary that is lovely. For anything where the answer must be the same every time (a score, a rule decision), do not ask a language model. Use code. A good rule: code decides, Claude describes.
Choosing the model
| Need | Pick |
|---|---|
| Short live lines, speed, low cost | A small, fast model (claude-haiku-5-5) |
| A full match report, nuance | A larger model (claude-sonnet-5-5) |
| Hardest reasoning | claude-opus-5-5 |
Each call costs money and time. Count your calls. A goal event is a good moment for a call; every frame is a terrible one.
Never block the match
The internet is slow and sometimes down. The queue-and-worker pattern keeps the detector running whatever happens to the network. We also limit the queue so a slow API cannot build up a backlog of stale news.
Keys, privacy and cost
- API key: it is a password that spends money. Keep it in an environment variable, never in code, and never in Git. If it leaks, delete it in the console and make another.
- Privacy: facts go to a company's servers. Send only what you need: ids, times, counts. Not faces, not children's names, not video of identifiable minors. Re-read lesson 14. For school use, ask the school before any data leaves the building.
- Cost: set a spending limit in the console and a per-session call cap in your code.
- Honesty: if commentary is machine-written, say so on the screen.
What you'll need
- Your goal events from lessons 31 to 36 (or the sample event list below)
- Your
cv_courseenvironment with the Anthropic SDK:pip install anthropic - An Anthropic API key, set up by your teacher or a parent, stored in an environment variable
Build it
Install: pip install anthropic. Get a key from the Claude Console (console.anthropic.com) with a teacher's account, and set it as an environment variable, never inside the code: export ANTHROPIC_API_KEY=... (macOS/Linux) or setx ANTHROPIC_API_KEY ... (Windows, then reopen the terminal). No key? Cells 1 to 4 and 6 work with the pretend client, and the tests need no internet.
Cell 1: from detections to facts
Needs events.py and goal_detector.py (lesson 31) and detections.json (lesson 30).
"""Turn detections.json into a short list of FACTS. Claude never sees video; it sees these."""
import json
import math
from events import EventBus
from goal_detector import GoalDetector
def nearest_player(frame):
"""Which player id is closest to the ball in this frame (None if there is no ball or no players)."""
b = frame["ball"]
if not b or not frame["players"]:
return None
return min(frame["players"], key=lambda p: math.hypot(p["x"] - b["x"], p["y"] - b["y"]))["id"]
def build_facts(path="detections.json"):
data = json.load(open(path))
frames = data["frames"]
goals = []
bus = EventBus()
by_time = {round(f["t"], 2): f for f in frames}
def on_goal(zone, t):
# who was nearest the ball a second before the goal? (the ball is in the net at t, so look back)
before = min(frames, key=lambda f: abs(f["t"] - (t - 1.0)))
goals.append({"time_seconds": round(t, 1), "nearest_player": nearest_player(before)})
bus.on("goal", on_goal)
det = GoalDetector(data["goal_zone"], bus)
for fr in frames:
det.update(fr["ball"], fr["t"])
ids = sorted({p["id"] for f in frames for p in f["players"]})
return {
"clip_length_seconds": round(frames[-1]["t"], 1),
"players_tracked": len(ids),
"goals_detected": len(goals),
"goals": goals,
"note": "Positions come from a camera system and may contain mistakes. Player numbers are tracker ids, not shirt numbers.",
}
if __name__ == "__main__":
print(json.dumps(build_facts(), indent=2))With the demo data you should see:
{
"clip_length_seconds": 20.0,
"players_tracked": 8,
"goals_detected": 3,
"goals": [
{
"time_seconds": 5.7,
"nearest_player": 1
},
{
"time_seconds": 12.4,
"nearest_player": 7
},
{
"time_seconds": 19.1,
"nearest_player": 1
}
],
"note": "Positions come from a camera system and may contain mistakes. Player numbers are tracker ids, not shirt numbers."
}Claude will never see the video. It only sees this. Whatever is not in the facts, it cannot know.
Cell 2: the prompt and the call
"""Ask Claude to commentate from facts. The model name is one constant, so you can swap it."""
import json
import os
MODEL = "claude-haiku-5-5" # fast and cheap: good for short live lines. Try "claude-sonnet-5-5" for a full match report.
SYSTEM = """You are a lively but honest football commentator for a school match.
Rules:
- Use ONLY the facts in the JSON the user sends. Never invent names, scores, times, shots, fouls or reasons.
- Player numbers are tracker ids, not shirt numbers or names. Say "player 4", never a name.
- If the facts do not cover something, say so briefly rather than guess.
- Keep it under 80 words. Plain text, no markdown."""
def build_user_message(facts, style="excited radio commentator"):
return f"Style: {style}.\nFacts (JSON):\n{json.dumps(facts, indent=2)}\n\nWrite the commentary."
def get_client():
"""Real client. Needs `pip install anthropic` and the ANTHROPIC_API_KEY environment variable."""
import anthropic
if not os.environ.get("ANTHROPIC_API_KEY"):
raise RuntimeError("Set ANTHROPIC_API_KEY first (never write the key in your code or commit it).")
return anthropic.Anthropic()
def commentate(facts, client=None, model=MODEL, style="excited radio commentator", max_tokens=300):
client = client or get_client()
msg = client.messages.create(
model=model,
max_tokens=max_tokens,
system=SYSTEM,
messages=[{"role": "user", "content": build_user_message(facts, style)}],
)
return "".join(block.text for block in msg.content if block.type == "text")Notice three things: the model name is one constant; the rules are in the system message; the facts travel as JSON in the user message. To use a bigger model for a full match report, change MODEL to claude-sonnet-5-5. Names change over time, so check the current list at docs.claude.com if one stops working.
Cell 3: a pretend Claude for tests
"""A pretend Claude for tests: it returns whatever text you give it, in the same shape as the real SDK."""
from types import SimpleNamespace
class FakeClient:
def __init__(self, reply="Goal!", fail_times=0):
self.reply, self.fail_times, self.calls = reply, fail_times, []
self.messages = SimpleNamespace(create=self._create)
def _create(self, **kwargs):
self.calls.append(kwargs)
if self.fail_times > 0:
self.fail_times -= 1
raise ConnectionError("network down")
return SimpleNamespace(content=[SimpleNamespace(type="text", text=self.reply)])Cell 4: catch invented numbers
The most common failure of a language model is to say something fluent that is false. This function checks that every number in the commentary exists in the facts.
"""Catch invented numbers. Every number in the commentary must be traceable to the facts."""
import re
def numbers_in(obj):
"""All numbers that appear anywhere in the facts, as strings in a few spellings (5.7, 5, 6...)."""
out = set()
def walk(x):
if isinstance(x, dict):
for v in x.values():
walk(v)
elif isinstance(x, list):
for v in x:
walk(v)
elif isinstance(x, (int, float)) and not isinstance(x, bool):
out.add(f"{x:g}")
out.add(str(int(round(x)))) # a commentator may round 5.7 s to "6 seconds"
out.add(f"{x:.1f}")
elif isinstance(x, str):
for m in re.findall(r"\d+(?:\.\d+)?", x):
out.add(m)
walk(obj)
return out
WORDS = {"one": "1", "two": "2", "three": "3", "four": "4", "five": "5", "six": "6", "seven": "7",
"eight": "8", "nine": "9", "ten": "10"}
def unsupported_numbers(text, facts):
allowed = numbers_in(facts)
found = re.findall(r"\d+(?:\.\d+)?", text)
for w in re.findall(r"[a-z]+", text.lower()):
if w in WORDS:
found.append(WORDS[w])
return sorted({n for n in found if n not in allowed}, key=float)Cell 5: try the real thing
from facts import build_facts
from commentary import commentate
from check_numbers import unsupported_numbers
facts = build_facts()
text = commentate(facts) # real call
print(text)
print("numbers not in the facts:", unsupported_numbers(text, facts))Run it three times. Is the text the same each time? (No: the output varies.) Change style to "a nature documentary narrator", "a nervous parent" and "a robot". Then remove the first rule from SYSTEM and ask for "who gave the assist?". Does it invent one? Put the rule back. Write down what you saw: this is your evidence for why the rule matters.
Cell 6: live lines without slowing the match
A call to Claude takes a second or more. The goal detector must never wait for it. Events go in a queue and a worker thread talks to Claude.
"""Live lines: the detector must NEVER wait for the internet. Events go in a queue; a worker thread talks to Claude."""
import queue
import threading
import time
from commentary import commentate
class CommentaryWorker:
def __init__(self, client=None, model=None, retries=2, on_line=print):
self.q = queue.Queue(maxsize=10) # if Claude is slow, drop old news instead of piling up
self.client, self.model, self.retries, self.on_line = client, model, retries, on_line
self.lines, self.errors, self.dropped = [], [], 0
self.thread = threading.Thread(target=self._run, daemon=True)
self.thread.start()
def submit(self, facts, style="excited radio commentator"):
"""Called from the match loop. Returns immediately."""
try:
self.q.put_nowait((facts, style))
except queue.Full:
self.dropped += 1
def _run(self):
while True:
item = self.q.get()
if item is None:
return
facts, style = item
for attempt in range(self.retries + 1):
try:
kwargs = {"client": self.client, "style": style}
if self.model:
kwargs["model"] = self.model
text = commentate(facts, **kwargs)
self.lines.append(text)
self.on_line(text)
break
except Exception as e: # network, rate limit, bad key: log and carry on
self.errors.append(f"attempt {attempt + 1}: {type(e).__name__}: {e}")
time.sleep(0.01 * (attempt + 1)) # tiny wait here; use seconds with a real API
self.q.task_done()
def close(self):
self.q.put(None)
self.thread.join(timeout=5)Cell 7: tests (no internet needed)
import time
from check_numbers import unsupported_numbers
from commentary import SYSTEM, build_user_message, commentate
from facts import build_facts
from fake_client import FakeClient
from live_commentary import CommentaryWorker
bad = 0
def check(name, ok, extra=""):
global bad; bad += not ok; print("PASS" if ok else "FAIL", name, extra)
facts = build_facts()
check("3 goals in the facts", facts["goals_detected"] == 3)
fc = FakeClient("And it is in! Player 1 scores at 5.7 seconds, and that is goal number 1 of 3 in this clip.")
text = commentate(facts, client=fc)
call = fc.calls[0]
check("system prompt forbids inventing", "ONLY the facts" in call["system"])
check("facts are in the user message", '"goals_detected": 3' in call["messages"][0]["content"])
check("model constant is used", call["model"] == "claude-haiku-5-5")
check("text returned", text.startswith("And it is in"))
good = "Three goals: player 1 at 5.7 seconds, player 7 at 12.4 seconds and player 1 again at 19.1 seconds."
check("true numbers pass", unsupported_numbers(good, facts) == [], unsupported_numbers(good, facts))
made_up = "Player 9 smashes it in at 7.2 seconds and it is 4-0 after 90 minutes."
print(" invented numbers found:", unsupported_numbers(made_up, facts))
check("invented numbers are caught", {"9", "7.2", "4", "0", "90"} <= set(unsupported_numbers(made_up, facts)))
check("number words are caught", "4" in unsupported_numbers("That is four goals.", facts))
check("rounding is allowed", unsupported_numbers("Around 6 seconds in.", facts) == [])
# worker: success
got = []
w = CommentaryWorker(client=FakeClient("Goal!"), on_line=got.append)
t0 = time.time(); w.submit(facts); submit_time = time.time() - t0
w.q.join(); w.close()
check("submit returns at once", submit_time < 0.05, f"{submit_time*1000:.1f} ms")
check("line delivered", got == ["Goal!"])
# worker: network fails once, then works
w = CommentaryWorker(client=FakeClient("Back online!", fail_times=1), retries=2, on_line=got.append)
w.submit(facts); w.q.join(); w.close()
check("retry works", w.lines == ["Back online!"] and len(w.errors) == 1, w.errors)
# worker: always fails -> never raises, match goes on
w = CommentaryWorker(client=FakeClient("x", fail_times=99), retries=1, on_line=got.append)
w.submit(facts); w.q.join(); w.close()
check("permanent failure is logged, not raised", w.lines == [] and len(w.errors) == 2, w.errors)
# worker: flood -> drop
class Slow(FakeClient):
def _create(self, **kw):
time.sleep(0.2); return super()._create(**kw)
w = CommentaryWorker(client=Slow("ok"), on_line=lambda s: None)
for _ in range(30): w.submit(facts)
check("a flood is dropped, not queued forever", w.dropped > 0, f"dropped {w.dropped}")
print("failures:", bad)All should print PASS. The last two matter most: a permanent network failure is logged but never stops the match, and a flood of events is dropped rather than piling up.
Questions: why does the worker drop events instead of waiting? What would you add if you wanted Claude to say "player 1" as a real name? (Think about lesson 14 before you answer.) What does max_tokens do?
If it goes wrong
| Error | Cause | Fix |
|---|---|---|
RuntimeError: Set ANTHROPIC_API_KEY | The environment variable is not set in this terminal | Set it, then reopen the terminal |
AuthenticationError | Wrong or revoked key | Create a new key in the console |
NotFoundError for the model | Model name changed or mistyped | Check the current list; change MODEL |
RateLimitError / 429 | Too many calls in a short time | Fewer calls; the worker retries; wait |
| Commentary invents things | Weak rules, or the question is outside the facts | Strengthen the system prompt, and run the number checker |
| Markdown symbols in the text | The model chose to format | The system prompt says plain text; say it again, or strip them |
| Output too long | max_tokens high, no length rule | Lower max_tokens, keep the 80 word rule |
| Test hangs | Worker not closed | Call close(); daemon threads exit with the program |
Try it
- Ask for the commentary in Konkani, Hindi or Spanish and check it with a speaker
- Add a second check: every player id in the text exists in the facts
- Make the commentary spoken with text-to-speech
Check what you learnt
Who decides that a goal happened: your code or Claude?
What is a hallucination?
Which is a good defence against hallucinations?
Why put commentary requests in a queue?
Where should the API key live?
Conclusion
A language model is great at turning facts into words, and bad at being the source of facts. Your code decides what happened, a system prompt sets the rules, Claude writes the commentary, and code checks it for invented details before it's shown. Keep the key in an environment variable and the detector off the internet's clock.
What next?
Next: Multimodal AI: send images to Claude. You'll show Claude pictures, not just facts.

