Case study · 05 · Computer vision + physical computing
Turning a hand gesture into a physical output.
A webcam‑based vision system that detects a face, counts raised fingers, and drives LEDs through an Arduino in real time. My first complete sensing‑to‑actuation loop.
Overview
This project connected two kinds of computing I had been exploring separately. MediaPipe identifies 21 landmarks on the hand; the Python application compares fingertip and joint positions to work out how many fingers are raised, then passes that count through PyFirmata to an Arduino, which lights the same number of LEDs on a breadboard.
It was a small project, but the loop was complete: observe the real world, interpret the signal, make a decision, drive a physical response. That pattern shows up again in everything I have built since, from FitForm to FitandFine.
How it works
- Hand landmarks. MediaPipe Hands returns 21 keypoints per hand; comparing fingertip and knuckle positions gives a reliable finger count.
- Face detection. A face model draws a bounding box and confirms a person is present before the system acts.
- Serial bridge. PyFirmata talks to the Arduino UNO over USB, so Python sets digital pins directly with no custom firmware.
- Actuation. Each finger count maps to a pin pattern that lights one to five LEDs in real time.
Impact
Gallery
