AI & Computer Vision, lesson 6 of 52
JavaScript for Computer Vision: Your Webcam in the Browser
Introduction
Python is great on your laptop, but your friends won't install it to see your project. A web browser is already on every phone and laptop, and it can use the camera too. Today you rebuild the lesson 2 webcam loop in JavaScript, then change the pixels yourself to make live filters.
By the end of this lesson you'll be able to:
- Ask for webcam access with
navigator.mediaDevices.getUserMedia() - Show the stream in a
<video>element and copy frames to a<canvas> - Read the pixels of a frame with
getImageData()and change them - Write a loop that runs once per frame with
requestAnimationFrame - Explain why the page must run from
localhostorhttps - Compare the Python loop from lesson 2 with the JavaScript loop
Working principle
Why the browser?
Python needs installation. A web page runs on any phone or laptop with a link. Later, browser CV (lesson 22, lesson 27) means models that run on your own device with no server, and dashboards (lesson 30) that anyone can open.
The Python to JavaScript map
| Python (lesson 2) | JavaScript (today) |
|---|---|
cap = cv2.VideoCapture(0) | navigator.mediaDevices.getUserMedia({ video: true }) |
ret, frame = cap.read() | ctx.drawImage(video, 0, 0) |
while True: | requestAnimationFrame(loop) |
frame (a NumPy array) | ctx.getImageData(0, 0, w, h) (an array of numbers) |
cv2.imshow(...) | the <canvas> already on the page |
cap.release() | stream.getTracks().forEach(t => t.stop()) |
Pixels in JavaScript
getImageData returns a data array of RGBA numbers in a single long row: R, G, B, A, R, G, B, A, .... Each pixel takes 4 numbers (alpha is see-through-ness, usually 255). Pixel number i starts at data[i * 4].
Note the order: JavaScript uses RGB, unlike OpenCV's BGR.
Secure context
Browsers only allow the camera on https pages or on localhost. Opening the file by double-clicking (file://) usually fails or asks every time. Run a tiny server instead:
python -m http.server 8000
# then open http://localhost:8000/webcam.htmlWhat you'll need
- A laptop with a webcam and Chrome, Edge or Firefox
- VS Code
- Python (only to run a tiny local web server:
python -m http.server)
Build it
Step 1: webcam to canvas
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>lesson 6 Browser Vision</title>
<style>
body { font-family: sans-serif; background: #111; color: #eee; text-align: center; }
canvas { border: 2px solid #0084ff; max-width: 95%; }
#hud { margin: 8px; }
button { padding: 8px 14px; margin: 4px; }
</style>
</head>
<body>
<h1>Browser vision</h1>
<video id="video" autoplay playsinline muted hidden></video>
<canvas id="canvas" width="640" height="480"></canvas>
<div id="hud">Waiting for camera...</div>
<div>
<button id="mode">Filter: none</button>
<button id="snap">Snapshot</button>
</div>
<script>
const video = document.getElementById("video");
const canvas = document.getElementById("canvas");
const ctx = canvas.getContext("2d", { willReadFrequently: true });
const hud = document.getElementById("hud");
const filters = ["none", "grayscale", "invert", "green only"];
let filterIndex = 0;
document.getElementById("mode").onclick = (e) => {
filterIndex = (filterIndex + 1) % filters.length;
e.target.textContent = "Filter: " + filters[filterIndex];
};
document.getElementById("snap").onclick = () => {
const a = document.createElement("a");
a.download = "snapshot.png";
a.href = canvas.toDataURL("image/png");
a.click();
};
let last = performance.now();
function loop() {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height); // like cap.read() + imshow
const frame = ctx.getImageData(0, 0, canvas.width, canvas.height);
const d = frame.data; // R, G, B, A, R, G, B, A ...
const mode = filters[filterIndex];
for (let i = 0; i < d.length; i += 4) {
const r = d[i], g = d[i + 1], b = d[i + 2];
if (mode === "grayscale") {
const y = 0.299 * r + 0.587 * g + 0.114 * b;
d[i] = d[i + 1] = d[i + 2] = y;
} else if (mode === "invert") {
d[i] = 255 - r; d[i + 1] = 255 - g; d[i + 2] = 255 - b;
} else if (mode === "green only") {
d[i] = 0; d[i + 2] = 0; // keep only the green channel
}
}
ctx.putImageData(frame, 0, 0);
const now = performance.now();
hud.textContent = "FPS: " + (1000 / (now - last)).toFixed(1) + " Filter: " + mode;
last = now;
requestAnimationFrame(loop); // schedule the next frame
}
navigator.mediaDevices.getUserMedia({ video: { width: 640, height: 480 } })
.then((stream) => {
video.srcObject = stream;
video.onloadedmetadata = () => { video.play(); loop(); };
})
.catch((err) => { hud.textContent = "Camera error: " + err.name; });
</script>
</body>
</html>Run it with python -m http.server 8000 from the same folder and open http://localhost:8000/webcam.html.
Question: where does the loop get its frames, and what stops it from running faster than the screen can show?
Step 2: extension, a brightness slider and a pixel readout
Add the slider to the HTML and the two lines to the loop. The mouse position shows the colour under the pointer.
<input id="bright" type="range" min="-100" max="100" value="0">
<span id="readout"></span>const bright = document.getElementById("bright");
const readout = document.getElementById("readout");
let mouse = { x: 0, y: 0 };
canvas.onmousemove = (e) => {
const box = canvas.getBoundingClientRect();
mouse.x = Math.floor((e.clientX - box.left) * canvas.width / box.width);
mouse.y = Math.floor((e.clientY - box.top) * canvas.height / box.height);
};
// Inside loop(), after getImageData and before the filter loop:
const k = Number(bright.value);
for (let i = 0; i < d.length; i += 4) {
d[i] = Math.min(255, Math.max(0, d[i] + k)); // red
d[i + 1] = Math.min(255, Math.max(0, d[i + 1] + k)); // green
d[i + 2] = Math.min(255, Math.max(0, d[i + 2] + k)); // blue
}
// Anywhere in loop(), to show the colour under the mouse:
const p = (mouse.y * canvas.width + mouse.x) * 4;
readout.textContent = ` RGB(${d[p]}, ${d[p + 1]}, ${d[p + 2]})`;Question: in lesson 3 you learned to avoid uint8 wrap-around. Why does the JavaScript version still need Math.min and Math.max? (Uint8ClampedArray clamps automatically, but this makes the idea visible, and it matters if you ever store values in a plain array.)
If it goes wrong
| Error | Cause | Fix |
|---|---|---|
NotAllowedError | Camera permission blocked | Click the camera icon in the address bar, allow it, reload |
NotFoundError | No camera found | Plug one in, or try another device |
NotReadableError | Another app holds the camera | Close Zoom, Teams, other tabs |
getUserMedia is undefined | Page opened with file:// or plain http on another host | Use localhost or https |
| Black canvas | The loop started before the video had loaded | Start the loop in onloadedmetadata, as in Step 1 |
| Colours look swapped | Mixing up RGB (JavaScript) and BGR (OpenCV) | Remember the order in each language |
| Page is very slow | Big canvas plus a heavy loop | Lower the canvas size (320 by 240) |
Canvas has been tainted error | Drawing a cross-origin image, then getImageData | Use the webcam or same-origin images only |
Try it
- Add a sepia filter or a threshold filter (white if brighter than 128, else black)
- Add a mirror mode so the image flips left to right
- Count the FPS for each filter and say which is slowest
Check what you learnt
What is the JavaScript version of cap.read()?
How many numbers does each pixel use in getImageData?
What does requestAnimationFrame do?
Why can't you just double-click the HTML file to use the camera?
What's the colour order in OpenCV, and in a canvas?
Conclusion
The browser can do what your Python loop does: getUserMedia opens the camera, drawImage grabs a frame onto a canvas, getImageData gives you the pixels (RGBA, four numbers each) and requestAnimationFrame repeats it every screen refresh. Pages that use the camera must run from localhost or https.
What next?
Next: Filters, thresholds & edges. Unit 2 starts: classic computer vision with filters, thresholds and edges.

