Imagine being able create a continuous stream of music by merely wearing a sensor on your finger. Now, imagine that the music changes in real-time, based on changes in your own mood.
This is ‘biomusic’, an interface that produces music from the physiological signals from our bodies. Biomusic is a premier example of how art and science can collaborate to create innovative technologies.
The idea of biomusic was originally conceptualized by Dr. Stefanie Blain-Moraes while she was volunteering at the Complex Continuing care unit for Disabled Children at the Bloorview Hospital in Toronto. Many of the children were completely reliant on technology including respirators, and required 24-hour personal care. They had little to no ability to move or communicate voluntarily because, for many of them, their somatic nervous system had deteriorated due to either injury or neurodegenerative disease. It was tough for these pediatric patients to communicate their wants and needs, as even the simple motion of pushing a button or flipping a switch proved difficult.
However, Dr. Blain-Moraes wondered if one could understand these children through the activity of their autonomic nervous system, which regulates unconscious functions, such as heartbeat, breathing rate, and sweating. Even though these children could not voluntarily communicate, one might still gain insight into their emotional world through these physiological indicators. Stefanie was eventually able to show that children who didn’t communicate verbally felt stimulated from being in contact with other people through changes in breathing and heart rate. The challenge was in translating the graphs and charts she used as evidence for this effect into a meaningful interpretation for caregivers and parents who were not scientifically trained. So, she sought a way to translate these physiological signals into a medium that was relatable. Given her background, which includes classical training as a pianist in addition to formal scientific training, Dr. Blain-Moraes wondered if this sort of scientific data could be communicated through an artistic medium instead. This was the start of the biomusic project.
How Biomusic Works
The biomusic interface considers four main bodily signals: heart rate, skin temperature, breathing rate, and electrodermal activity (sweating) – all of which can be detected by wearing a device on one’s finger. The device plays a default tone, which then fluctuates based on changes in one’s autonomic nervous system activity. For example, breathing and heart rate are usually correlated in a relaxed individual – when you inhale, your heart rate accelerates and when you exhale, your heart rate decelerates. However, when excited, the two signals start to de-sync, and the correlation is disrupted. This can be recognized by the interface, and is then reflected in an accelerando in the music. In its early stages, biomusic simply mapped changes in physiological signals to typical music tropes: heart rate controlled the tempo of the music, skin temperature affected the key, spikes in electrodermal activity created little embellishments in the melody.
The iteration of biomusic currently in development is more advanced. It is now customizable, allowing users to choose their own soundscapes rather than sticking with the default classical music one. Sounds of nature, Indian drums, or even a purring cat can be worked into the program’s musical output. Additionally, machine learning techniques are being integrated to enable the interface to recognize specific emotional states of an individual through their physiological signals and adjust the music accordingly.
Caption: A demonstration of biomusic at its early stages. Note the dramatic change in music (at 13:43) after the researcher asks the subject to think of something stressful! Source: TEDxMontreal (2010)
A Plethora of Applications with Biomusic
Biomusic was initially employed to interpret the emotional states of non-communicative children. The interface was linked to children in complex continuing care units during time with their caregivers. The use of biomusic drastically changed the interactions between caregivers and patients. Caregivers felt that even though the patients were technically non-communicative, and some even vegetative, the changes in their biomusic let the caregivers know that the children were in fact responding emotionally during the interactions. Biomusic became a tool to communicate these emotions.
One other application of biomusic is in individuals with autism-spectrum disorder as a way of regulating one’s emotional state. Dr Blain-Moraes teamed up with Spectrum Productions, a non-profit organization that does audiovisual design and exclusively employs individuals on the autism spectrum, to create workshops where autistic individuals could try Biomusic. In one workshop, participants went to art galleries while wearing their Biomusic interfaces. At one point during the visit, a participant noticed that their Biomusic became erratic, and wondered why. He realized that he was on the threshold between two different rooms, and that this abundance of sensory input made him feel overwhelmed. So, he stepped back into one room, and the music calmed down. Thus, Biomusic appeared to make it easier for individuals on the autism spectrum to self-regulate their emotions.
In addition to being a tool for therapeutic means, biomusic can also be used for other exciting avenues. For meditation purposes, an individual can listen to their biomusic as a means of monitoring their heart rate and breathing. Athletes can also use the interface to gauge their performance. Even musicians and composers can use biomusic by adding samples of their own biomusic to their compositions. One would not only be writing music, but could actually intrinsically create different types of music based on one’s emotional state.
Biomusic is a project born at the intersection of science and art. While Dr. Stefanie Blain-Moraes was formally trained in science and engineering, her background in classical music lent her creative ideas to tackle problems in science. Biomusic has a promising future as both a therapeutic and creative device for patients and the general public.
Stefanie Blain-Moraes holds a B.A. Sc and PhD from the University of Toronto in biomedical engineering and rehabilitation sciences, and is an Assistant Professor in the Department of Physical and Occupational Therapy at McGill University. She has also completed her ARCT diploma in classical piano from the Royal Conservatory of Music. Her favorite composer is Beethoven. You can follow more of her work here (https://www.mcgill.ca/spot/stefanie-blain-moraes, http://www.moraeslab.com/biapt/).
“The Black Box – Biomusic” could not be possible without the efforts of volunteers at Convergence Initiative (www.convergenceinitiative.org), a non-profit organization dedicated to science outreach and fostering collaboration between scientific and artistic disciplines.