Biofeedback you can hear.
MusicFit matches exertion, sport performance and music in real time, at full biometric granularity. Train your vital parameters, reach your performance goals and stay in control of your musical image.
An industrial B2B demo for makers of medical and fitness equipment and of wellness apps.
Bespoke beats for bespoke performance.
A run, heard from the inside
The runner steers an EDM track with his own vital parameters while he runs. Each footfall lands on a kick, and the arrangement tells him whether he is inside his target zone.
Below your zone
Stripped back: kick and pad
An illustration running in your browser: a simulated run and a simple EDM generator. It is not the production engine. The weight of the headline follows the beat too. The piece: festival EDM in A minor, supersaw chords and a sidechain pump.
What the match is for
Granular, in real time
Exertion, sport performance and music are matched at full biometric granularity, moment by moment.
Music biofeedback
The music is the feedback. Hearing your effort, you train yourself to improve your vital parameters and reach your performance goals.
Your musical image
You stay in control of your musical image while you train.
From the band to the beat
A chest band or a wrist band streams the heartbeat. Heart rate variability sets the energy thresholds of the music, and the session looks for what repeats in the athlete's movement in order to consolidate it: the athlete hears their own control.
Heart rate, thresholds and the level the music followed
Beat-to-beat intervals
Movement against the beat
- BaseOpen: held Rhodes, soft kick and hats
- AerobicGroove: bass, backbeat and Rhodes comping
- ThresholdDrive: lead line and horns join, everything on
- MaxRelease: half-time and a cymbal swell, back off
- Bass motif
- Lead motif
- Hat pattern
When the gesture repeats steadily for a few bars, the pattern is locked into the music; when it breaks, the music lets it go.
| Age | HRmax (bpm) | Base to aerobic (bpm) | Aerobic to threshold (bpm) | Threshold to max (bpm) | Resting RMSSD (ms) | Tempo (bpm) | Music structure |
|---|
A demo model, not a medical device. HRmax follows the Tanaka formula (208 − 0.7 × age) at the midpoint of each band, the thresholds come from the HRV model of the demo, and the resting values are indicative. Real thresholds have to be validated on each individual.
More MUSI-CO biofeedback demos: Mindkestra, MUSI-CO.
Boxing: music that follows your rhythm
The music is produced as separate stems, rendered offline in high quality and stretched in time to the pace of the boxer without changing the pitch. Every punch answers with MIDI: when the punches land on the beat and repeat the same combination, the music builds and rewards; when the rhythm is lost, it loses layers and sours.
Space bar: the next punch of the combination.
Rhythm lane: beats and punches
Pose tracking (simulated)
MIDI feedback
- Drums
- Bass, from 4
- Chords, from 8
- Lead, from 12
On the beat within 35 ms: a perfect hit, the music opens and plays the melody of your combination. Within 70 ms: good. Beyond that, or a wrong punch in the repeated combination: the layers drop, the filter closes, the pitch sours, and two misses in a row stutter.
A simulation. In the final service the punches come from our computer-vision blocks, which send the same event (type of punch, hand, time) to this audio and MIDI engine through window.musicfitBoxing.punch(type, hand, time); nothing else changes.
Two versions, one logic
Offline
The offline version is built for medical and gym machines, where the sound has to be reliable and self-contained.
Realtime
The realtime version lives in mobile apps, where the music follows the user's movement as it happens.
Sonification techniques
The techniques that turn movement into sound are derived from GLooper, our synthesis environment. GLooper exists as a synth on the web and, locally, as an extension of the DAW.
On the web
A synth in the browser, where sound behaviour can be designed and tried.
In the DAW
A local extension, so the work can happen inside the tools composers already use.
Films
Same logic as Mindkestra
MusicFit follows the model of Mindkestra, our immersive therapeutic audio environments: an offline version for machines and a realtime version for mobile apps.
Request an industrial demo
MusicFit is offered to businesses. Tell us what you build and we will arrange a demonstration.