Finally, The Secret Format Behind True Virtual Reality Revealed…
My obsession with immersive sound began before it became ‘cool’, before every gamer had immersive sound embedded into their virtual worlds and multi-directional sound had infiltrated pop-culture. But now, from ASMR to Christmas-gifted VR headsets, immersive sound is undoubtedly having a moment. So how is this audio manipulated to make listeners feel part of the action, what exactly does this mean for the future of audio and why does it matter?
Audio is the ‘thing’ that makes us spatially aware - without it, our experience is based solely on visual cues, which simply put, is just not enough. Sound designers have found that the quality of the visual content can be substantially reduced when paired with well made immersive audio and the result will still be a believable environment for players. Reversed, average audio with great video - not so much.
Immersive sound can infer a number of things. There’s ‘binaural’ which is a method of recording sound in the same way humans hear - often recorded using omnidirectional microphones that mimic human ears. Yet binaural, whilst a giant step up in immersive quality and realistic feel is not the most immersive method of recording sound. To feel completely enveloped by the sound - as if the audio was a sphere surrounding you above, below, left, right, forward, behind and on every angle - ambisonics is the key.
Technically speaking, ambisonics makes a sound field perfect at a very small point in space, by breaking it down into “spherical harmonics”. The more harmonics that are recorded, the bigger the reconstruction of the space. These are then easily manipulated; soundfields can be rotated, sounds can be moved up or down within the space, and zoomed to and from.
Imagine for a moment a busy pub. You’re speaking to a friend whilst other conversations at the surrounding tables are occurring. You can hear your drinking partner just fine in person but had the event been recorded it would be impossible to discern one conversation from another, the general din drowning out all identifiability. However, recorded ambisonically, each conversation could be zoomed in on - to the same quality as if you were sat next to them, eavesdropping on their discussion. Kind of incredible, right?
With ambisonics, the sound surrounds you completely. For comparison, a stereo recording works with one plane of sound - left and right. Binaural recordings are static, mimicking human ear canals and as such are wedded to the use of headphones to experience the full effect. But with ambisonics, the full sphere of possible audio direction is condensed into one single output - placing the listener at the centre. And which industry is largely hinged on the centralization of the listener? Yep, virtual reality and the gaming industry.
Video games and virtual reality require the addition of immersive sound to keep the illusion of reality alive. Head tracking - the phenomenon of sound moving counter to you in immersive environments - is at the core of what makes VR, well, VR. The ‘responding’ of sound to your movements is the core of what makes an immersive environment. Take a fire pit for example; crackling and sparking, the insistent hisses and pops of burning kindling filling your ears. If you’re facing the fire, the sound will be in front of you. The moment you turn to walk away from the fire the sound will move around and settle behind you. This change is the subtle but crucial difference between a realistic virtual environment and one that jars with the player and feels fake.
I approach all my mixes ambisonically - no matter what the required final format is. iT
So how can ambisonics and immersive sound be used commercially? Is there scope for brands to harness the power of ambisonics too?
Absolutely! VR and Game audio is leading the field, but audio mixed for TV and Radio has a lot of catching up to do.
Unsurprisingly, the big players are already using the format. Google and Facebook pair their 360-degree videos with ambisonic audio, enabling users to pan around an image with responsive sound. As a key development in our ability to build convincing and enveloping words, ambisonics is the sound of the future, but there’s still so much more that could be transformed. If live concerts, club nights, performances and shows could be experienced remotely, if listeners and revellers could feel as if the music was wholly surrounding and including them with immersive audio - the music industry could expand its reach with a new level of experience hierarchy.
So, maybe I can’t make it to a Hans Zimmer show in Sydney, but if I could don a pair of headphones in London, open my laptop and virtually transport myself to being sat in the Harbour-side location - well that would be incredible, wouldn’t it?