Browser Detection & Triggers in JavaScript

Introduction

Seeing boxes is nice; making something happen is the point. Today you'll run a detector in a web page and draw a zone on the screen. When a person steps into it, the page reacts once: a banner, a log line and a sound, like a shop door chime, not a car alarm.

By the end of this lesson you'll be able to:

Run COCO-SSD on a webcam in the browser and read its output
Draw detection boxes on a canvas laid over the video
Test whether a point is inside a zone, and choose the right point (the feet, not the box centre)
Explain edge-triggered events versus "on every frame", and why a cooldown is needed
Fire a custom DOM event and react to it with a banner, a log line and a sound
Explain why browsers block sound until the user clicks, and how that affects the design

Working principle

Detector in, decision out

camera frame -> COCO-SSD -> list of people (boxes) -> is a foot inside the zone? -> enter event? -> alert

The detector is the easy part. The interesting work is turning a stream of noisy detections into one clean event.

COCO-SSD versus YOLO

	COCO-SSD (today)	YOLO (lesson 23 to lesson 26)
Where it runs	In the browser, nothing to install	Python (or a server)
Classes	Fixed: 80 COCO classes (`person`, `sports ball`...)	Whatever you trained
Speed	Light and quick on most laptops	Heavier
Accuracy	Fine for people in clear view; poor for a small ball	Better, and tuned to your data
Output box	`[x, y, width, height]`	`[x1, y1, x2, y2]` (careful: they differ!)

Which point do we test?

A person's box centre is in the middle of their body. Their feet are where they stand on the pitch. For "is the player in this part of the pitch?" use the bottom-centre of the box. This matters again in lesson 32 and lesson 33.

Events, not polling

Instead of checking a flag everywhere, the page dispatches an event (zone-enter) and any part of the page can listen for it: change the banner, write to a log, play a sound, later send a message to a server or a microcontroller (lesson 35). This is the same idea as a button click.

Edge trigger and cooldown

Idea	Meaning	Why
Level	"A person is inside"	True for as long as they stand there. Firing on this alerts 30 times a second.
Edge	"A person just became inside"	One event per entry
Cooldown	Ignore re-entries for N milliseconds	A noisy detector flickers: the person disappears for a frame and reappears. Without a cooldown that is two alerts.
Confidence threshold	Ignore weak detections	Fewer false alarms, more misses (lesson 19)

Outside the zone. The page is waiting for the feet to cross the line.

Browser rules to remember

The camera needs localhost or https (lesson 22).
Sound: browsers block audio until the person has clicked or tapped the page. That's why there is a "Turn sound on" button.
The video and the canvas can have different on-screen and internal sizes. We convert mouse positions to video pixels.

What you'll need

Chrome or Edge, a webcam and VS Code
Python to run python -m http.server

Build it

Part 1: the logic on its own (Node, or paste into the browser console)

This is the same logic the page uses (the first part of the script). It runs without a camera. Save it as logic.js and run node logic.js.

 
    // ---------- 1. The logic (pure functions: easy to test) ----------
    function centreOf(bbox) {                         // bbox = [x, y, width, height]
      return { x: bbox[0] + bbox[2] / 2, y: bbox[1] + bbox[3] / 2 };
    }
    function footOf(bbox) {                           // where a person stands: bottom-centre of the box
      return { x: bbox[0] + bbox[2] / 2, y: bbox[1] + bbox[3] };
    }
    function inZone(point, zone) {                    // zone = {x, y, w, h}
      return zone !== null && point.x >= zone.x && point.x <= zone.x + zone.w &&
             point.y >= zone.y && point.y <= zone.y + zone.h;
    }
    // Fire an event only when someone ENTERS (from outside to inside), then wait for a cooldown.
    function makeTrigger(cooldownMs) {
      let wasInside = false, lastFire = -Infinity;
      return function (isInside, nowMs) {
        let fire = false;
        if (isInside && !wasInside && nowMs - lastFire >= cooldownMs) { fire = true; lastFire = nowMs; }
        wasInside = isInside;
        return fire;
      };
    }
 
    
const assert=require('assert');
const zone={x:100,y:100,w:200,h:100};
assert.strictEqual(inZone({x:150,y:150},zone),true);
assert.strictEqual(inZone({x:50,y:150},zone),false);
assert.strictEqual(inZone({x:150,y:150},null),false);
assert.deepStrictEqual(footOf([10,20,30,60]),{x:25,y:80});
assert.deepStrictEqual(centreOf([10,20,30,60]),{x:25,y:50});
const t=makeTrigger(2000);
const seq=[[false,0],[true,100],[true,200],[false,300],[true,400],[false,2500],[true,2600],[true,2700]];
const fired=seq.map(([i,n])=>t(i,n));
console.log('fired:',JSON.stringify(fired));
assert.deepStrictEqual(fired,[false,true,false,false,false,false,true,false]);
console.log('logic tests passed');

Read the fired list: false, true, false, false, false, false, true, false. The second item is the first entry. The re-entry at 400 ms is ignored because the cooldown (2000 ms) has not passed. The entry at 2600 ms fires again.

Question: if the cooldown were 0, which items in the list would change?

Part 2: the page

Save as index.html in a new folder. Run python -m http.server 8000, open http://localhost:8000, allow the camera, click "Turn sound on" once, and drag a rectangle on the video to draw the zone.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Zone alert</title>
  <style>
    body { font-family: system-ui, sans-serif; max-width: 760px; margin: 1rem auto; padding: 0 1rem; }
    #stage { position: relative; width: 100%; max-width: 720px; }
    #cam { width: 100%; display: block; border-radius: 8px; background: #111; }
    #overlay { position: absolute; left: 0; top: 0; width: 100%; height: 100%; cursor: crosshair; }
    #banner { margin: .6rem 0; padding: .6rem .9rem; border-radius: 8px; background: #e8f0fe; font-weight: 600; }
    #banner.alert { background: #ffb300; color: #000; }
    button { font-size: 1rem; padding: .45rem .8rem; margin: .2rem .2rem .2rem 0; }
    #log { font-family: ui-monospace, monospace; font-size: .85rem; max-height: 9rem; overflow: auto; background: #f4f4f4; padding: .5rem; border-radius: 6px; }
    #status { color: #555; min-height: 1.4em; }
  </style>
</head>
<body>
  <h1>Zone alert</h1>
  <p id="status">Loading...</p>
  <div id="stage">
    <video id="cam" autoplay playsinline muted></video>
    <canvas id="overlay"></canvas>
  </div>
  <div id="banner">Drag on the video to draw a zone.</div>
  <button id="sound">Turn sound on</button>
  <button id="clearZone">Clear zone</button>
  <label>Confidence <input id="conf" type="range" min="0.2" max="0.9" step="0.05" value="0.5"> <span id="confVal">0.50</span></label>
  <div id="log"></div>
 
  <!-- Version numbers were current when written: check them on npmjs.com before class. -->
  <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@4.22.0/dist/tf.min.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/coco-ssd@2.2.3"></script>
  <script>
    // ---------- 1. The logic (pure functions: easy to test) ----------
    function centreOf(bbox) {                         // bbox = [x, y, width, height]
      return { x: bbox[0] + bbox[2] / 2, y: bbox[1] + bbox[3] / 2 };
    }
    function footOf(bbox) {                           // where a person stands: bottom-centre of the box
      return { x: bbox[0] + bbox[2] / 2, y: bbox[1] + bbox[3] };
    }
    function inZone(point, zone) {                    // zone = {x, y, w, h}
      return zone !== null && point.x >= zone.x && point.x <= zone.x + zone.w &&
             point.y >= zone.y && point.y <= zone.y + zone.h;
    }
    // Fire an event only when someone ENTERS (from outside to inside), then wait for a cooldown.
    function makeTrigger(cooldownMs) {
      let wasInside = false, lastFire = -Infinity;
      return function (isInside, nowMs) {
        let fire = false;
        if (isInside && !wasInside && nowMs - lastFire >= cooldownMs) { fire = true; lastFire = nowMs; }
        wasInside = isInside;
        return fire;
      };
    }
 
    // ---------- 2. The page ----------
    const video = document.getElementById("cam");
    const overlay = document.getElementById("overlay");
    const ctx = overlay.getContext("2d");
    const statusEl = document.getElementById("status");
    const banner = document.getElementById("banner");
    const logEl = document.getElementById("log");
    const confInput = document.getElementById("conf");
    const confVal = document.getElementById("confVal");
 
    let model = null, zone = null, dragStart = null, soundOn = false, audioCtx = null;
    const trigger = makeTrigger(2000);
    let scaleX = 1, scaleY = 1;
 
    confInput.oninput = () => { confVal.textContent = Number(confInput.value).toFixed(2); };
 
    function log(message) {
      const line = document.createElement("div");
      line.textContent = new Date().toLocaleTimeString() + "  " + message;
      logEl.prepend(line);
    }
 
    // The event system: any other part of the page can listen for "zone-enter"
    document.addEventListener("zone-enter", (e) => {
      banner.textContent = "Someone entered the zone (confidence " + e.detail.score.toFixed(2) + ")";
      banner.classList.add("alert");
      log("zone-enter");
      beep();
      setTimeout(() => { banner.classList.remove("alert"); banner.textContent = "Watching the zone..."; }, 1500);
    });
 
    function beep() {
      if (!soundOn || !audioCtx) return;
      const osc = audioCtx.createOscillator(), gain = audioCtx.createGain();
      osc.frequency.value = 880;
      gain.gain.setValueAtTime(0.2, audioCtx.currentTime);
      gain.gain.exponentialRampToValueAtTime(0.001, audioCtx.currentTime + 0.4);
      osc.connect(gain).connect(audioCtx.destination);
      osc.start();
      osc.stop(audioCtx.currentTime + 0.4);
    }
 
    document.getElementById("sound").onclick = (e) => {      // browsers only allow sound after a click
      audioCtx = audioCtx || new (window.AudioContext || window.webkitAudioContext)();
      soundOn = !soundOn;
      e.target.textContent = soundOn ? "Turn sound off" : "Turn sound on";
      if (soundOn) beep();
    };
    document.getElementById("clearZone").onclick = () => { zone = null; banner.textContent = "Drag on the video to draw a zone."; };
 
    // Draw the zone with the mouse. Coordinates are converted from screen pixels to video pixels.
    function toVideoPoint(ev) {
      const r = overlay.getBoundingClientRect();
      return { x: (ev.clientX - r.left) * (overlay.width / r.width), y: (ev.clientY - r.top) * (overlay.height / r.height) };
    }
    overlay.onpointerdown = (ev) => { dragStart = toVideoPoint(ev); };
    overlay.onpointermove = (ev) => {
      if (!dragStart) return;
      const p = toVideoPoint(ev);
      zone = { x: Math.min(dragStart.x, p.x), y: Math.min(dragStart.y, p.y), w: Math.abs(p.x - dragStart.x), h: Math.abs(p.y - dragStart.y) };
    };
    overlay.onpointerup = () => { dragStart = null; if (zone && zone.w > 10) banner.textContent = "Watching the zone..."; };
 
    async function loop() {
      overlay.width = video.videoWidth || 640;
      overlay.height = video.videoHeight || 480;
      ctx.clearRect(0, 0, overlay.width, overlay.height);
 
      let personInside = false, bestScore = 0;
      try {
        const detections = await model.detect(video);
        for (const d of detections) {
          if (d.class !== "person" || d.score < Number(confInput.value)) continue;
          const foot = footOf(d.bbox);
          const inside = inZone(foot, zone);
          if (inside) { personInside = true; bestScore = Math.max(bestScore, d.score); }
          ctx.strokeStyle = inside ? "#ffb300" : "#0084ff";
          ctx.lineWidth = 3;
          ctx.strokeRect(d.bbox[0], d.bbox[1], d.bbox[2], d.bbox[3]);
          ctx.fillStyle = ctx.strokeStyle;
          ctx.fillRect(foot.x - 4, foot.y - 4, 8, 8);
          ctx.font = "16px sans-serif";
          ctx.fillText("person " + d.score.toFixed(2), d.bbox[0], Math.max(16, d.bbox[1] - 6));
        }
      } catch (err) { statusEl.textContent = "Error: " + err.message; }
 
      if (zone) {
        ctx.strokeStyle = "#e53935"; ctx.lineWidth = 3; ctx.setLineDash([8, 6]);
        ctx.strokeRect(zone.x, zone.y, zone.w, zone.h);
        ctx.setLineDash([]);
      }
      if (trigger(personInside, performance.now())) {
        document.dispatchEvent(new CustomEvent("zone-enter", { detail: { score: bestScore } }));
      }
      requestAnimationFrame(loop);
    }
 
    async function main() {
      try {
        await tf.ready();
        statusEl.textContent = "Starting the camera (allow it when asked)...";
        const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: false });
        video.srcObject = stream;
        await new Promise((resolve) => { video.onloadeddata = resolve; });
        statusEl.textContent = "Loading the detector (a few MB)...";
        model = await cocoSsd.load();                   // COCO-SSD "lite" model: small and quick
        statusEl.textContent = "Ready. Backend: " + tf.getBackend();
        loop();
      } catch (err) {
        statusEl.textContent = "Could not start: " + err.message;
      }
    }
    main();
  </script>
</body>
</html>

Questions: what is the difference between using footOf and centreOf when you stand close to the zone's edge? Why does the alert not repeat while you stand inside? What happens if the detector misses you for one frame while you are inside?

Part 3: choose an extension

Extension	Hint
Count entries	Add a counter that goes up on every `zone-enter` event
Exit event	Make the trigger also fire when someone leaves, and dispatch `zone-exit`
Two zones	Keep an array of zones, each with its own trigger and colour
Dwell time	Measure how long someone stays inside and show it
Ball alert	Also watch for the class `sports ball` (it is small, so expect misses: why?)
Send it somewhere	`fetch` a URL when the event fires (a preview of lesson 34 and lesson 35)

If it goes wrong

Error	Cause	Fix
Camera does not start	Not `localhost` or `https`	Serve with `python -m http.server`
`cocoSsd is not defined`	Script failed to load, or wrong order	Check the network tab. Keep TensorFlow.js first.
Boxes are in the wrong place	Canvas size does not match the video	Set `overlay.width` and `height` from `video.videoWidth` and `videoHeight`, as the code does
Zone drawn in the wrong place	Mouse position not converted	Use `toVideoPoint`
No sound	The browser blocks audio until a click	Click "Turn sound on" first
Alerts repeat constantly	Using the level, not the edge	Use `makeTrigger`
Alert missed when someone briefly leaves detection	Flickering detector	Lower confidence a little, or keep a "recently seen" timer
Slow and choppy	Weak laptop or a busy tab	Close other tabs; check `tf.getBackend()` is `webgl`

Try it

Add two zones with different sounds
Write the dwell-time extension and show a "time in zone" bar
Add a "smooth" step: only count a person as inside after 3 consecutive frames

Check what you learnt

Why test the foot position instead of the box centre?
What is an edge trigger?
Why add a cooldown?
Why is there a 'Turn sound on' button?
Why can't COCO-SSD find goal posts?

Conclusion

A detector gives boxes; a trigger turns them into decisions. Test the feet against the zone, fire on the edge (outside to inside), add a cooldown so flicker doesn't spam alerts, and use DOM events to drive a banner, a log and a sound (after the user has clicked to allow audio).

What next?

Next: FIELD TRIP: football field data capture. Unit 6 starts with Field Trip 1: capturing football data on a real pitch.