Sonic Boom: Building an Immersive World Through Audio

With a strong technical ability and creative flare, Mike loves to work on short films and various immersive projects outside of his commercial sound design work at 750mph. He also leads 750's VR offering.

Here, Mike discusses ambisonic audio and the importance of immersiveness. 

It’s easy to understand why the visuals of a VR film are so immersive, essentially you are in a full 360-degree all-encompassing environment, which gives you the freedom to look around at what you want. What many people don’t realise is that this immersive world building can be achieved with audio as well. This is something commonplace in video games, where sounds are attached to objects in a fully 3D world, so while you move through the level the sounds move dynamically with you making them sonically and spatially accurate to the space. What’s really interesting is that a similar level of sonic immersiveness and directionality can be achieved in a 360 video with the utilisation of spatial audio, or ambisonics.

Ambisonic audio is essentially a sphere of sound. Ordinarily audio comes in channels; a stereo mix has a left and right channel, a 5.1 mix has Centre, left, right, left surround and right surround and so on. The difference with ambisonic audio is that it has no channel information, everything is placed within a sphere, this is then decoded to play out of whatever speaker setup you want (5.1, 7.1 even Dolby Atmos) which makes it a powerful method to mix in. The key to ambisonic audio within VR is the ability to decode it into binaural - this means that, when wearing headphones, the audio convincingly sounds like it’s coming from the world around you.

What’s even cooler is that even once decoded into binaural you still have a full 360 degree sonic landscape. So, with a VR film you can have the audio placed within the same spherical plain as the visuals turning the audio into a dynamic, directional sonic landscape, one which convincingly (thanks to the binaural decode) moves with your view. Having this really adds to the engaging experience as visually and audibly you are fully immersed in the video. 

There are numerous examples of projects utilising ambisonic audio within their 360 films. I’ve had the pleasure to work on a number of them myself. One of the most notable and something that was able to shine because of this format was The Wayback project. The Wayback recreates historical UK events, telling stories of regular people at the time. Currently there are two episodes: one set during the Queen's coronation in 1955 and another during the 1966 world cup. The goal behind this project was to help people with dementia. By immersing them in these recreations, in which many of the people watching would have been present, it was found to help increase the chance of them recalling intact memories of the time, opening up conversations.

Another project I worked on that utilised the immersiveness of VR in an interesting way was for Guinness. This was a film that was used while people sampled the different drinks made at the Guinness factory. The customer would don a pair of VR goggles and headphones and while they tried each drink there would be an accompanying film with visuals and spatial audio specifically designed to help the customer pick out the flavours of the drinks. I find it really interesting how both these projects successfully use the medium but in completely different ways, it really shows how diverse VR can be.

Early on when researching VR and 360 videos I found that, unlike a regular 2D film, you have no physical control as to where the viewer is looking. To get around this sound becomes a vital part of the experience. Because all of the sounds are directional, they can be used as a cue to peak the viewer's interest and have them turn to look at whatever it is you want them to see. The sound essentially becomes your director. Without this, say for example you use a stereo mix on the 360 video, you have no way of accurately prompting people to look in specific directions as, unlike with ambisonics, the audio is essentially locked.

An example of using the audio as a prompt would be another project I worked on for Greenpeace. In this there’s a full 360-degree jungle you can look around. There’s the sound of a waterfall behind, which helps prompt the viewer to look around the scene, as they turn their head the sonic world moves around with them, bringing the crashing waterfall to the front. After around 30 seconds of settling them into this world there’s a rustle in a tree at another section of the jungle, this is intended to get the viewer’s attention, when they turn to look, they are confronted with an orangutan. This then becomes our focal point and the character we want the viewer’s attention to be on. It’s the use of these “spot” sounds that make you able to direct the interest and focus of the viewer.

Because it’s still a developing medium there are some technical hurdles when implementing ambisonic audio into VR and 360 films. One of the hurdles is that these films can only be played using specific platforms, these include Facebook, YouTube and bespoke apps, but as the medium becomes more popular the choice in software and platforms is also increasing. One other thing to note is that with a 360 video the audio is still mixed beforehand so levels can’t be dynamically changed in the same way as a video game, it is just the direction of audio that becomes dynamic. But even this is leaps and bounds more immersive than even a cinema mix.

VR and 360 videos are only just finding their feet and so currently have some limitations, but the future is definitely very exciting. With the introduction of higher order ambisonics (the higher the order the more precise the sonic landscape becomes) and higher quality VR films, the fidelity is only going to improve and as the community continues to play with the medium and see what can be pushed things can only get more interesting. Even now though the immersiveness is unrivalled making it an extremely interesting medium to look into.