Overview
Hold your hand up to a webcam, make a few gestures, and the system enters Morse code and decodes it into text in real time. Conceived as an input method that lets people communicate without speech or a physical keyboard, it was designed with accessibility in mind — supporting communication for people with hearing or speech difficulties. This HCI (Human-Computer Interaction) project runs entirely on a standard laptop CPU with no specialised hardware.
Team & my role
An international collaborative project carried out during a one-week visit to our sister university in Thailand, in my third year. We formed a single team of five — three Thai students and two Japanese students — and I led the development as team lead.
It was my first time holding technical discussions in English, and communication was far from smooth. So I broke the work down into individual features and shared a diagram of the overall structure, so every member could see both their own piece and how it fit into the whole. As a result, we managed to ship a working system within the tight one-week deadline.
Controls & gesture design
Morse code's "dit (·)" and "dah (–)" are expressed through hand gestures and how long each one is held.
- Thumbs up: start input
- Open palm (short, ≤ 1s): "·" (dot / dit)
- Open palm (long, > 1s): "–" (dash / dah)
- "I love you" sign: space (word separator)
- Victory / peace sign: confirm and convert the current Morse sequence into one character
- Pointing up: clear the current signal
- Thumbs down: reset the entire input
- Plus editing commands such as backspace (delete one character) and word delete
Key Technologies
- Gesture recognition: MediaPipe Gesture Recognizer (gesture_recognizer.task) classifies hand shapes into predefined gestures — Open_Palm / Victory / Thumb_Up / ILoveYou, and more
- Timing logic: measures how long each gesture is held — a short hold is a dot, a long hold is a dash — to build the Morse signal sequence
- Morse decoding: a lookup table covering A–Z, 0–9, and symbols (? ! . , ; : + - / =) decodes the sequence into a character, triggered by the confirm gesture
- Real-time UI: OpenCV shows the mirrored camera feed alongside a Morse reference chart, visualising the live signal, the decoded text, and a "Now Reading" indicator
- Design: the gesture-recognition class extends a hand-landmark detection class (built through class inheritance)
- Stack: Python / MediaPipe / OpenCV / NumPy