Browser app self-drives a Steam Controller into its charging puck using computer vision

An open-source Vue 3 web app pilots a Steam Controller across a desk and into its magnetic charging puck with no human input. An overhead webcam feeds OpenCV.js, which uses Lucas-Kanade optical flow to track the controller and puck, while a WebHID connection streams telemetry directly from the hardware. The app then steers the controller by firing 70Hz asymmetric haptic pulses through its two internal linear resonant actuators, effectively turning the rumble motors into a crude locomotion system. A ‘proximity creep’ mode halves the pulse rate once the controller is within 150 pixels of the puck so the magnet can seat gently.

The build leans on some notable browser-native tricks. Object avoidance runs on a Rust convolutional neural network compiled to WebAssembly and offloaded to a Web Worker to keep the tracking loop responsive, and battery state is read by parsing raw HID reports (0x43 for live voltage and percentage, 0x79 to confirm charging began). The whole thing needs only a Chromium-based browser with WebHID support and Nix as the sole build dependency, which handles fetching packages and compiling the WASM module across Windows, Mac, and Linux.

The project is mostly a playful demonstration rather than a practical tool, but it’s a tidy showcase of how far in-browser hardware and vision APIs have come. Optical flow, native HID access, and WASM-accelerated inference all run client-side, and the author credits a Very Lazy Pixel video as the inspiration. It’s MIT-licensed.