Sonic Boom: Innovations in Sound with Weston Fonger

Weston Fonger is the senior mixer and sound designer at VinylMix & Yessian NY. His work can be heard on a myriad of long and short form content as well as experiential and interactive entertainment. Throughout a nearly two decade long career he has won multiple awards for both mix and sound design, as well as served as a judge and panellist at many industry events. Since joining with Yessian in 2011 and helping to launch VinylMix in 2017 some notable projects have included working as supervising sound editor on Taylor Swift’s 1989 World Tour documentary on Apple Music with director Jonas Åkerlund, creating sound design and mixing all the audio content for NYC’s One World Observatory, films that have shown at SXSW, Tribeca, Annecy, and Venice, as well as numerous Super Bowl ads.

I’d like to focus on sound creation for Augmented Reality / AR. From a sound design perspective, I’m fascinated by the idea of creating sounds that will be heard alongside environmental sounds in order to give the user a heightened experience in their games, presentations, or other AR content. Because AR only requires a smart device that most already own and a recommended set of headphones it is a much more approachable technology for most as opposed to purchasing and carrying around a set of VR goggles. For this reason, I think we are going to see an increasing amount of AR content making its way into our daily lives.    

Sound design techniques for AR stem from game sound design coupled with spatial audio workflows and programming to inject sound elements into a particular experience. From a creative perspective the sound designer would create and assemble a library of sound triggers / cues with other static sounds and textures to support whatever story is being told. This is similar to creating sounds for traditional gaming platforms, but instead of those cues being used fill out and create fully in-the-box soundscapes with ambiences, footfalls, and often times music, the sounds in an AR experience must often work alongside the sounds of the real world and add another layer to the listening experience.

In the past few years, we’ve had some really fun opportunities to create audio for AR. A memorable project from a while back is one where we created sounds for a tabletop display for an investment management company where users could choose their own paths through various manufacturing and shipping centres in order to highlight the company’s role in business growth. The visual display was in augmented reality and everything seen by the user on a tablet was mapped to the top of a pedestal which the user could walk around and explore the different scenes in depth. Our sounds had to create the full environments in this case, replacing the sounds happening around the listener in the real world to make the experience fully immersive with voiceover guiding the story. Each scene was composed of static sounds as well as triggered sounds which the user could interact with.  

Another project for a Japanese cosmetic brand had us create sounds for a new pop-up store they were opening in Tokyo. Upon entering the store, guests were greeted with a large abstract projected image of their head and face which would change colour and speed with every mood they displayed on their face. We were tasked with creating dynamic sounds to coincide with every movement and emotion as they happened in real time. 

Recently we partnered with a major brand to design a library of bespoke sounds for a retro, 8bit stylized video game with an AR component that we are excited to see released in the near future. 

Because the term AR can be applied to a variety of user experiences from interactive to static and various hybrids of the two, each project presents its own opportunity to learn and grow from, building a foundational knowledge base that can aid how we approach future work.

I would say that it’s important to understand that there are often two roles involved in creating and implementing sound for augmented reality. There is the design phase where the creative sound designer works to make a bespoke sound set, and an implementation phase where an audio developer takes those sounds and incorporates them into the experience adjusting timings, levels, placement in their implementation. This is a departure from traditional audio post-production workflows, though depending on the specific AR experience there may be a blend of mixed content as well as triggered content. As a sound designer many of us are now learning the implementation component in order to provide a full audio workflow at a single facility.  

One of the challenges in designing sound for augmented reality is understanding that the created sounds need to be able to cut through whatever real-world environment the listener is in. The experience has to translate equally well in both a quiet bedroom and loud subway station. Just as important is knowing when a created sound needs to be in the forefront or background of an experience so as to call attention to something when needed or to simply provide subtle support for a particular moment. This would affect not only the level of a sound used, but also the sound’s timbre and frequency in order to help it stand out or blend in.

There are so many applications for AR that already exist. There are games that users can interact with as they move about during their day.  There are retailers creating virtual fitting rooms in AR allowing customers to try on clothing, accessories and makeup, or even design how their spaces could look with new furniture, wall colours etc.  Museums are using AR to help visitors engage with their collections in a more in-depth way.  There are also AR applications for tourism and maps where a user can hold their device up as they move through a city and gather more information about their location and the landmarks and objects around them that they may not see.  There are more creative uses of AR seemingly all the time. From an audio perspective there are a lot of new ways to consider how we use can use sounds to support them, whether that is a set of simple UI sounds as a user navigates a retail experience, or aiding the learning experience by adding sounds to otherwise silent displays like hearing the sound a dinosaur might have made in a museum exhibit, or even just to have fun and manipulate the sound of your own voice with a social media filter. Sound is such an important part of the way we consume media and tell stories, so as new ways to utilise AR emerge so will the need to consider how we hear them.