Gestura

Gestura is a wearable performance instrument designed to empower artists with cerebral palsy and limited speech abilities. By translating subtle finger bends and arm trajectories into real-time audio and visual modulation, Gestura expands stage expression beyond vocal communication.

Concept & Inclusive Design#

Each movement acts as a brushstroke on a live stage canvas, seamlessly coupling physical gesture with reactive projection visuals and synthesized soundscapes. Built around customizable calibration profiles and a handmade adjustable leather glove and arm strap, Gestura adapts to each performer's unique range of motion rather than requiring the performer to conform to a rigid interface.

Hardware & Software Architecture#

• Wearable Sensing: Handmade leather glove and arm strap housing an Arduino Nano 33 IoT powered by a 9V battery, reading finger curvature via flex sensors, touch position via a soft potentiometer, and 6-axis arm orientation via the onboard accelerometer and gyroscope.
• Wireless Telemetry: Streams calibrated sensor packets over Wi-Fi to the stage workstation in real time.
• Audio & Visual Synthesis: Drives reactive stage projection graphics in TouchDesigner and real-time spatial audio synthesis in Max/MSP/Jitter.

Team & Contributions#

• Alan Ren — Physical Computing & Wearable Engineering
• Hanyong (Kyrie) Yang — Product Design, Branding & Wearable Engineering
• Kefan Lyu — Interactive Sound Output (Max/MSP) & Video Editing
• Muqing Wang — Generative Visual Output (TouchDesigner) & Project Outreach