Overview
Gesture Synth is a groundbreaking, browser-based musical instrument that removes the barriers between technology and creativity. By integrating advanced computer vision with the Web Audio API, it allows users to perform "air music" using only a standard webcam. No MIDI controllers or software downloads are required—your hands become the interface for immediate, expressive sound creation.
Key Features
- Real-time Gesture Tracking: Powered by MediaPipe, it tracks 21 landmarks per hand. The left hand controls chord degrees and harmonic color, while the right hand manages voicing, octave, volume, and filter resonance.
- Playable Sound Engine: Features 12 musical keys and multiple synth waveforms (Warm, Bright, Retro). The engine is entirely browser-native for low-latency, real-time response.
- Integrated Local Recording: Capture performances instantly. The tool mixes synth audio, microphone input, and the visual canvas into a high-quality MP4 file for local download.
- No Hardware Required: Works on any modern browser (Chrome/Edge) across Windows, macOS, and Linux without specialized sensors.
Use Cases
- Rapid Sketching: Quickly test chord progressions or sonic textures before opening a heavy DAW.
- Interactive Performance: Ideal for live streaming and digital art where visual movement is central to the performance.
- Music Education: A visual way to teach intervals, chord qualities, and synthesis without complex interfaces.
- Creative Prototyping: A prime example of human-computer interaction (HCI) in a creative web context.
Getting Started
- Enable Camera: Visit gesturesynth.co and grant camera access in a well-lit space.
- Initialize Sound: Click the center "Play" button once hand-tracking markers appear.
- Shape Harmony: Use your left hand to select chord degrees and tilt for Major/Minor qualities.
- Refine Sound: Use your right hand to adjust volume, voicing, and filter sweeps.
- Record: Use the red "Record" button to save your performance directly to your device.
Privacy & Security
Gesture Synth is built with a "Privacy First" architecture. All camera processing and audio synthesis happen locally on your device. No video or audio data is ever sent to a server. No account or personal data is required, ensuring total anonymity.
Refinement Notes
- Character Count: Significantly reduced the wordiness of the "Overview" and "Privacy" sections to ensure it fits within your 3000-character limit.
- Clarity: Kept the bulleted structure for "Key Features" and "Use Cases" to ensure readability on the submission page.
- Technical Accuracy: Retained mentions of MediaPipe and Web Audio API as they are strong selling points for this type of tool.
