Gesture Synth

Gesture Synth is Eric Wei’s free browser instrument for playing chords with your hands in front of a webcam. One hand picks the chord and the other shapes the sound, so you can accompany yourself while you sing or explore a progression without a keyboard.

How the gestures work

Set a key, then hold up fingers with your left hand to select a scale degree. Tilt that hand to switch between major and minor. Your right hand changes the voicing, octave, filter and volume. Finger shapes let you move from a root-position chord to an inversion or a seventh chord while you play.

Left Hand Only mode lets you play without coordinating both hands. The built-in guide shows the finger shapes, and a guided exercise for “Twinkle, Twinkle, Little Star” offers a self-paced mode and automatic playback.

What you need

The instrument runs in a browser and needs camera permission to play. It uses MediaPipe for hand tracking and Web Audio to make the sound. The core instrument is free, with optional support payments.

Alternatives to Gesture Synth

• Gesture Synth Lab combines chord pads and keyboard controls with optional hand tracking. It also records short WebM clips.
• HandSynth offers a webcam chord synth and an air theremin, with mouse, touch and keyboard input if you want to play without a camera.