AI & Computer Vision, lesson 41 of 52

Player Heatmaps: Footprints in the Snow

advanced26 min read

Introduction

Where did your striker spend the match? Hanging around the box, or dropping deep to help? A heatmap answers that in one picture. Today you'll turn the tracked positions from the live event into heatmaps on a real pitch, in metres, and learn which choices make a heatmap honest and which make it misleading.

By the end of this lesson you'll be able to:

  • Explain a heatmap as counting positions in cells
  • Build a seconds-per-cell grid from positions in metres with a 2D histogram
  • Smooth and render the grid on a pitch
  • Compare players on a shared colour scale
  • Explain how cell size, smoothing and colour change the picture
  • Name three things that can make a heatmap wrong, and how to check

Working principle

From positions to a picture

Every frame of tracked data is a dot: a player at (x, y) metres. A heatmap answers "where did they spend their time?" by counting dots per area. Many dots in a cell = a hot cell.

positions (lesson 33) -> grid of cells -> seconds per cell -> smooth -> colours on the pitch

Use pitch metres, not pixels

If you count in pixels, a player near the camera looks like they cover more ground than one far away (lesson 33). On the flat top-down pitch every metre is the same size, so the counts are fair.

Counting: a 2D histogram

np.histogram2d puts every point in a cell of a grid and counts. Divide by the frame rate and the count becomes seconds. That is a number a coach understands: "player 7 spent 128 seconds in this square".

Each dot is a position (in metres). The grid counts how many land in each cell: the more footprints, the hotter the cell. 0 positions so far.

Choices that change the picture

ChoiceEffect
Cell sizeSmall cells look noisy; big cells hide detail. 5 m is a good start.
SmoothingMakes a glow instead of boxes. Too much hides the real shape.
Colour scaleShared scale to compare players; own scale to see one player's pattern
Time windowWhole match, a half, or the last five minutes

None of these is "correct". Say which you used when you show a heatmap.

Be careful with colour

Rainbow scales (blue, green, yellow, red) look dramatic, but the brain does not read them evenly, and they can make small differences look large. For serious work, use a scale that goes from light to dark in one colour. We use the rainbow for fun and say so. Never use colour alone to carry the meaning: add the numbers.

Summaries: thirds

Pictures are for people; numbers are for decisions. The share of time in the defensive, middle and attacking thirds is a simple summary. Compare two matches and you can see whether a team has moved up the pitch.

What can go wrong

  • Bad calibration (lesson 33): players land off the pitch. The dropped count is a free quality check.
  • ID switches (lesson 26): when the tracker mixes up two players, their heatmaps blend. A per-player map is only as good as the tracking.
  • Missing frames: if a player is not detected, their time is not counted. A heatmap shows where we saw them.
  • Few minutes of data: a five-minute heatmap is a snapshot, not a style of play.

People, not just dots

A heatmap is data about a person, often a child. Think before sharing. Show a child their own map first. Avoid ranking children by "most active". Don't publish named heatmaps without the consent you collected in lesson 14. Coaching uses of this data (with agreement) are fine; shaming is not (lesson 43 will come back to this).

What you'll need

  • Tracked positions from lesson 39 (or the sample CSV)
  • Your homography from lesson 33
  • Your cv_course environment with Matplotlib

Build it

Cell 1: pretend positions in metres

No real data yet? This makes ten minutes of made-up positions for eight players, each with a favourite area. The format is the one to_pitch_json.py (lesson 33) writes, so your real file drops straight in.

"""Pretend positions in METRES (the lesson 33 format), so heatmaps can be practised before you have real data.
Eight players, each with a favourite area, wandering for 10 minutes at 5 frames per second."""
import json
 
import numpy as np
 
rng = np.random.default_rng(3)
FPS, MINUTES = 5, 10
W, H = 105.0, 68.0
roles = {1: (10, 34, "keeper"), 2: (30, 20, "defender"), 3: (30, 48, "defender"), 4: (55, 34, "midfielder"),
         5: (60, 15, "winger"), 6: (60, 53, "winger"), 7: (80, 30, "striker"), 8: (85, 40, "striker")}
n = FPS * 60 * MINUTES
pos = {pid: np.array([ax, ay], float) for pid, (ax, ay, _) in roles.items()}
frames = []
for i in range(n):
    players = []
    for pid, (ax, ay, _) in roles.items():
        step = rng.normal(0, 0.9, 2)
        pull = (np.array([ax, ay]) - pos[pid]) * 0.03            # drift back towards the favourite area
        pos[pid] = pos[pid] + step + pull
        x, y = pos[pid]
        players.append({"id": pid, "x": round(float(x), 2), "y": round(float(y), 2)})
    frames.append({"frame": i, "t": round(i / FPS, 2), "players": players, "ball": None})
json.dump({"units": "metres", "fps": FPS, "frames": frames}, open("positions_pitch.json", "w"))
print("frames:", len(frames), "| roles:", {k: v[2] for k, v in roles.items()})

Cell 2: the heatmap tools

Needs positions_pitch.json. (pitch_map.py from lesson 33 is not required here.)

"""Heatmaps: where did the players spend their time? Input: positions in metres (lesson 33)."""
import json
 
import cv2
import numpy as np
 
PITCH_W, PITCH_H = 105.0, 68.0
 
 
def load_positions(path="positions_pitch.json"):
    d = json.load(open(path))
    return d["fps"], d["frames"]
 
 
def collect(frames, player_id=None):
    """All (x, y) points for one player, or for everyone. Returns (points, how many were off the pitch)."""
    pts = [(p["x"], p["y"]) for fr in frames for p in fr["players"] if player_id is None or p["id"] == player_id]
    pts = np.array(pts, float).reshape(-1, 2)
    on = (pts[:, 0] >= 0) & (pts[:, 0] <= PITCH_W) & (pts[:, 1] >= 0) & (pts[:, 1] <= PITCH_H)
    return pts[on], int((~on).sum())
 
 
def seconds_grid(points, fps, cell=5.0):
    """Count the time spent in each cell of a grid (5 m by 5 m). Result is in SECONDS."""
    nx, ny = int(np.ceil(PITCH_W / cell)), int(np.ceil(PITCH_H / cell))
    counts, _, _ = np.histogram2d(points[:, 0], points[:, 1], bins=[nx, ny], range=[[0, nx * cell], [0, ny * cell]])
    return counts.T / fps                                     # rows = y, columns = x
 
 
def smooth(grid, sigma=1.0):
    """Blur so the picture looks like a smooth glow, not boxes. Blurring keeps the total (almost) the same."""
    return cv2.GaussianBlur(grid.astype(np.float32), (0, 0), sigma, borderType=cv2.BORDER_CONSTANT)
 
 
def thirds(points):
    """Share of time in the defensive, middle and attacking third (attacking left to right)."""
    if len(points) == 0:
        return (0.0, 0.0, 0.0)
    x = points[:, 0]
    d = float((x < PITCH_W / 3).mean())
    a = float((x >= 2 * PITCH_W / 3).mean())
    return (d, 1.0 - d - a, a)
 
 
def render(grid, scale=8, margin=20, vmax=None, title=None):
    """Draw the grid on a green pitch. Pass the same vmax to compare two players fairly."""
    w, h = int(PITCH_W * scale) + 2 * margin, int(PITCH_H * scale) + 2 * margin
    base = np.full((h, w, 3), (50, 120, 50), np.uint8)
    vmax = vmax or float(grid.max()) or 1.0
    norm = np.clip(grid / vmax, 0, 1)
    heat = cv2.resize((norm * 255).astype(np.uint8), (int(PITCH_W * scale), int(PITCH_H * scale)), interpolation=cv2.INTER_CUBIC)
    colour = cv2.applyColorMap(heat, cv2.COLORMAP_JET)
    alpha = (heat.astype(np.float32) / 255.0)[..., None] * 0.8        # cold cells stay see-through
    roi = base[margin:margin + heat.shape[0], margin:margin + heat.shape[1]].astype(np.float32)
    base[margin:margin + heat.shape[0], margin:margin + heat.shape[1]] = (roi * (1 - alpha) + colour * alpha).astype(np.uint8)
    cv2.rectangle(base, (margin, margin), (w - margin, h - margin), (255, 255, 255), 2)
    cv2.line(base, (w // 2, margin), (w // 2, h - margin), (255, 255, 255), 2)
    cv2.circle(base, (w // 2, h // 2), int(9.15 * scale), (255, 255, 255), 2)
    if title:
        cv2.putText(base, title, (margin, 14), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)
    return base

Cell 3: your first heatmap

import cv2
from heatmap import load_positions, collect, seconds_grid, smooth, render, thirds
 
fps, frames = load_positions("positions_pitch.json")
points, dropped = collect(frames, player_id=7)
print("points off the pitch:", dropped)
 
grid = seconds_grid(points, fps, cell=5.0)         # seconds per 5 m by 5 m cell
cv2.imwrite("heat_player7.png", render(smooth(grid), title="player 7"))
 
d, m, a = thirds(points)
print(f"defensive {d:.0%}   middle {m:.0%}   attacking {a:.0%}")

Open heat_player7.png. The glow should sit in the right-hand (attacking) half, around x = 80 m. Which player would have a glow near the left goal?

Cell 4: compare players fairly

Two heatmaps with different colour scales cannot be compared. Use one vmax for both.

k, _ = collect(frames, 1)
s, _ = collect(frames, 7)
vmax = max(seconds_grid(k, fps).max(), seconds_grid(s, fps).max())
cv2.imwrite("heat_1.png", render(smooth(seconds_grid(k, fps)), vmax=vmax, title="player 1"))
cv2.imwrite("heat_7.png", render(smooth(seconds_grid(s, fps)), vmax=vmax, title="player 7"))

Cell 5: cell size changes the story

From our run on player 7:

Cell sizeGridBusiest cellCells used
2.5 m28 by 4247 s57
5 m14 by 21128 s21
10 m7 by 11185 s8

The same ten minutes. The busiest cell is four times bigger with larger cells, because larger cells hold more. A number from a heatmap only means something with its cell size. Try your own sizes.

Cell 6: tests

import numpy as np
from heatmap import *
 
bad = 0
def check(name, ok, extra=""):
    global bad; bad += not ok; print("PASS" if ok else "FAIL", name, extra)
 
fps = 5
# a player who stands in one spot for 10 seconds
frames = [{"players": [{"id": 1, "x": 12.0, "y": 8.0}]} for _ in range(10 * fps)]
pts, off = collect(frames, 1)
g = seconds_grid(pts, fps)
check("10 s in one place gives 10 s in that cell", abs(g.sum() - 10) < 1e-9 and abs(g[1, 2] - 10) < 1e-9, (g.sum(), g[1, 2]))
check("grid shape is 14 rows by 21 columns", g.shape == (14, 21), g.shape)
 
# off-pitch points are dropped and counted
frames = [{"players": [{"id": 1, "x": -5.0, "y": 10.0}, {"id": 1, "x": 50.0, "y": 70.0}, {"id": 1, "x": 50.0, "y": 30.0}]}]
pts, off = collect(frames, 1)
check("two off-pitch points dropped and counted", len(pts) == 1 and off == 2, (len(pts), off))
 
# smoothing keeps the total
fps, fr = load_positions("positions_pitch.json")
pts, off = collect(fr)
g = seconds_grid(pts, fps)
s = smooth(g, 1.0)
check("total time = players x minutes x 60 (8 x 600 s) minus off-pitch", abs(g.sum() - (len(pts) / fps)) < 1e-6, g.sum())
print(f"   total on-pitch time {g.sum():.0f} s, dropped {off} points")
check("smoothing keeps most of the total (edges lose a little)", 0.9 * g.sum() < s.sum() <= g.sum() + 1e-3, (s.sum(), g.sum()))
 
# roles show up
k, _ = collect(fr, 1); st, _ = collect(fr, 7)
dk, mk, ak = thirds(k); ds, ms, as_ = thirds(st)
check("keeper spends most time in the defensive third", dk > 0.8, f"{dk:.0%}")
check("striker spends most time in the attacking third", as_ > 0.5, f"{as_:.0%}")
check("thirds add to 1", abs(sum(thirds(st)) - 1) < 1e-9)
check("empty input does not crash", thirds(np.zeros((0, 2))) == (0.0, 0.0, 0.0))
 
img = render(smooth(seconds_grid(st, fps)), title="player 7")
check("render gives a picture", img.shape == (584, 880, 3) and img.std() > 5, img.shape)
cv2.imwrite("heat7.png", img)
# fair comparison: same scale
vmax = max(seconds_grid(k, fps).max(), seconds_grid(st, fps).max())
cv2.imwrite("heat1_shared.png", render(smooth(seconds_grid(k, fps)), vmax=vmax, title="player 1 (shared scale)"))
print("failures:", bad)

All should print PASS. The first checks the basic promise: ten seconds standing still gives ten seconds in one cell.

Questions: what does the dropped number tell you about your calibration? If a tracker swaps two players' ids halfway (lesson 26), what happens to both heatmaps? Why is a rainbow colour scale risky? Would you show a child's heatmap to the whole team? To their parents? To nobody but them?

If it goes wrong

ErrorCauseFix
Empty pictureEveryone is off the pitch (bad calibration)Check the dropped count; recalibrate (lesson 33)
Heatmap mirroredx and y swapped, or the pitch flippedCheck which way attack goes; flip the grid
One bright cell, nothing elseCell too big, or too few pointsSmaller cells, more data
Everything looks the same temperatureEach map has its own scale, shown aloneUse a shared vmax when comparing
Two players' maps look alikeTracker swapped idsCheck the tracks (lesson 26); smooth ids
Seconds do not match the clockWrong fps valueUse the fps from the file
ValueError in histogramNo points for that playerCheck the id exists; handle an empty list
Rainbow hides small differencesColour scaleUse a one-colour scale and print the numbers

Try it

  • Make a team heatmap and a heatmap of the first half versus the second
  • Add a one-colour scale option and compare
  • Add a 'distance covered' number (preview of lesson 45)

Check what you learnt

  1. Why use metres, not pixels?

  2. What does a hot cell mean?

  3. Why use one shared colour scale for two players' maps?

  4. What does the count of dropped positions tell you?

  5. Why be careful sharing heatmaps?

Conclusion

A heatmap is footprints in the snow: map each player's feet to pitch metres, count them in a grid (a 2D histogram), choose the cell size and smoothing with care, and use a shared, honest colour scale. Check how many positions fell off the pitch, and share heatmaps of people only with consent.

What next?

Next: Auto-highlight reel generator. You'll make the computer edit a highlight reel by itself.