Converting soundwaves into vibrations - experimenting with new ways of experiencing sound

Last summer’s quiet period of reflection was the ideal time for Paraorchestra Executive Producer, Hannah Williams Walton, and I to set about considering ways in which we could further enhance the experience of live music for D/deaf audiences.

In my last blog I touched on some of our research into existing technology; from established products like the SUBPAC, to more analogue tools such as using balloons to transmit soundwaves. But what we established very quickly was that we were interested in developing something that gave the user more autonomy over their experience and, rather than focussing on electronic instruments (like the SUBPAC – which would be perfect for, say, a rave) we were interested in creating something that could also work on acoustic instruments.

Many Paraorchestra projects - such as The Nature of Why or The Anatomy of the Orchestra - are immersive experiences, they invite the audience to move about the performance space, empowering individuals to choose what they would like to experience. Unlike other audio enhancement products available we wanted something more discreet, that could perhaps be handheld, or placed in a pocket.

So, our device had to be something very portable, ergonomic, and flexible with different types of music and instruments.

With designer, technologist, and musician Steve Symons – a resident at Pervasive Music Studio – on board we started to explore how we might translate the vibrations caused by the sound waves produced by orchestral instruments to something that could be experienced by a hearing-impaired individual. We quickly ruled out visual translations – some technology like the Audiolux use patterns of flashing lights to convey sound patterns. Instead, we began to look at technology that translated sound into vibrations.

Earlier this year, a small, wooden, and rather beautiful egg-shaped device arrived by courier to my flat. Steve’s prototype - the TouchSound, as is its working title - had an output for a microphone, a power button, and ‘volume’ control, and was roughly the size of an elongated tennis ball. I cleared my diary and tested it out immediately. Though it quickly transpired that playing the clarinet one-handed while holding the device and microphone in the other hand was, shall we say, not a wholly accurate reflection of how the device might work in practice!

How much of what I was experiencing with the device was informed by my other senses?

Our first priority had always been to start testing with a number of people who identify as D/deaf so we can start to get feedback on the device from its intended audience, but with the current COVID restrictions in place it has proved very challenging. Unable to undertake that testing as hoped, we took the opportunity when we were rehearsing for a recorded project last month at Bristol Old Vic to focus instead on how the TouchSound might react to multiple instruments.

With Steve on hand as well, several members of the Paraorchestra team, myself included, took turns to move about the orchestra – as close as social distancing permitted – pointing the microphone towards the direction of the sound we wanted to experience. It’s hard to explain, but it did appear that there was a clear difference in the vibration intensity, and pattern of vibration, depending on what instrument the user was engaging with. Of course, some of the louder instruments with more bass – reportedly like my bass clarinet – did at times overwhelm. But, it is my feeling that in a world when we can safely interact with the musicians more intimately, and after some more development this may be less of an issue.

I did wonder, however, how much of what I was experiencing with the device was informed by my other senses.

“Rather than used in isolation” Steve explained “It’s designed to supplement your other senses. It’s that correlation between what you see, and what you feel. The same notes on different instruments will feel the same as they have the same frequencies, but if you are seeing the sound being made as well, that is when you begin to augment those visual clues with the experience of sound”

We’re really excited about the potential for further development but appropriate testing is key. Hence, our next step will be to create more devices; if we are unable to test in person we will then have the option to send them out to willing participant’s for wider feedback on the experience.

We are also looking to hone the prototype; to make it smaller, as ergonomic as possible, and to strike the balance between using a mic that is powerful enough to capture the sound, yet small enough to remain discreet.

We set out with the aim of researching how we could develop and utilise technology to enhance the experience of live performances for D/deaf audiences. We’re really pleased with what we have achieved so far with the TouchSound and are looking forward to seeing what the next stage of testing brings.

If you identify as D/deaf or have a hearing impairment and are interested in helping us test the TouchSound please do email me on lloyd@paraorchestra.com