Beyond 2D: A Love Affair with Volumetric Video
Volumetric video is another giant step in the convergence of traditional and interactive media. With the ability to capture subjects and understand shape and volume from multiple angles in full, live 3D, volumetric video has allowed businesses and creators to think outside the box to create immersive content and experiences that give the viewer a truly unique perspective.
Michelle Excell, the founder of The Antipodean, creative tech consultants, fell in love with volumetric video when she put on a VR headset at Lytro’s office in 2016 and watched a demo of what volumetric would become.
"It was amazing and hilarious. It was a short faux-recreation of Kubrick’s faked moon landing," Michelle says."It felt like a standard 360 video experience at first, but they’d shot the actor playing Kubrick with their crazy new camera rig. The Lytro team member told me to lean in, and I actually got closer to the actor. It was my first experience of a real video with partial 6dof, and it seriously blew my mind!"
To further discuss her love affair with volumetric video, see examples of it in action, and highlight how this groundbreaking tech can elevate a brand's presence in the market, we sat down with Michelle.
So, what exactly is volumetric video?
Volumetric video is a production technique involving cameras that surround and capture subjects to create a 3D moving image that can then be viewed from any angle.
"What you end up with is the re-materialising of the subject in 3D. You're capturing them from all angles with standard cameras, but also infrared cameras that capture depth. You are literally recording their volume. What you are actually getting is a 3D video file of that subject — talking, moving, dancing," Michelle says.
The volumetric capture can then be used for campaigns using augmented reality, virtual reality, holographic displays, and even web and traditional video production. As a result, and given the right planning, the technique is highly flexible.
Above: A Microsoft Mixed Reality Capture Studios representative talks about volumetric video in Michelle’s living room.
Along with a vast selection of other immersive technology available today, Michelle believes volumetric video allows us to do things we can’t do any other way. You can feel volumetric experiences. And feeling the presence of the characters creates next-level immersion. No other tech can do that with real video, it can feel like immersive theatre It is also broadly applicable to a wide array of businesses and use cases.
"Volumetric video is still a fairly new technology, and the sky's the limit for where it can go," Michelle says. "YOU could come up with the next best way to use it. That's the beauty of it — you don't have to be technical to get creative with it. Costs are coming down to produce it, and while fidelity is getting better by the day, it doesn’t depend on it to be impactful. You don’t have to wait for it to be perfect."
How can volumetric video be used?
Michelle thinks some of the most creative volumetric work has been from the music industry.
"Whether it's a big name and studio, or an indie band and creator, some of the best work I've seen in this space is from music videos and special projects. Maybe it's got to do with the performative nature of their work, or the creators they collaborate with, but they've figured out some really incredible ways to work with it," she says.
Michelle remembers the 2018 mixed-reality campaign by AMEX Music featuring Justin Timberlake being a stand-out. "Outside In" used volumetric video to allow Justin to come to life in your living room, launched through the AMEX music app. You could explore a song from his album as elements of it unfold around you, and you could also shop for merch.
"Having a life-size, hyper-realistic three dimensional Justin Timberlake appear in your living room — chatting away — is surreal and oh-so-memorable. Whether you're a fan or not," says Michelle.
"But look, it doesn't have to be what we would deem 'high quality' to be good. I see so many music video creators and directors playing with volumetric. They don't overcomplicate it; they get straight to the essence of what makes it unique and compelling and don't obsess over photo realism," Michelle explains. "There are two projects that stand out for me from the last year: ASAP Ferg — Green Juice. and Caoilfhionn Rose — Fireflies."
Above: Outside In, from American Express Music and RadicalMedia, took users inside the creative mind of Justin Timberlake thanks to volumetric video.
LA-based volumetric capture studio, Metastage — who Michelle highly recommends working with — has compiled a helpful list of popular use cases for volumetric video. They are quick to remind us, though, that this only represents the tip of the iceberg:
Sports - bring sports fans closer to their favourite athletes.
Education - bring the world's best teacher into your living room or classroom.
Training - create interactive training scenarios or simulations.
Virtual tours - create a realistic experience with an expert guide.
Trade shows - bring your lead executives with you to present at trade shows.
Museums - use holographic capture to bring installations to life.
Inspirational - create connections between your customer and inspirational figures.
Games - create celebrity cameos to elevate video games.
“If you’re a brand and you have a known and loved celebrity or ambassador for your campaign, you’ve solved the hardest question: ‘will people engage with it?’” Michelle says, “Then you just need to make darn sure you make something special, and not just rely on that star power.”
An early example of a project Michelle was involved with is a powerful PSA made by VISYON and Left Field Labs for RK Venture and New Mexico Department of Transportation. It’s an immersive VR experience about the risks of drunk driving, delivered via a reenactment from the perspectives of the drunk driver, the victim in the accident, and the responding officer. It was shot at the Microsoft Mixed Reality Capture Studio in San Francisco, and highlights how PSAs can be even more impactful when the immersion of VR is coupled with the realism of volumetric video.
"You completely captured all the emotions and pain I experienced, truthfully and accurately.”
- A real DWI victim commenting on New Mexico Department of Transportation’s experience.
The fashion industry was an early adopter of the technology, along with the music industry. Holograms were initially all the rage, but we’re now seeing practical prototypes beginning to emerge. These include a Holographic telepresence test in association with Telefónica, shot with Evercoast and premiered at MWC2022.
"Like most immersive tech, you need to see it and feel it, to 'get it'”, says Michelle. “Then it becomes a case of, 'Oh my god, I need this! How can we use this for our next campaign/installation/app?'. I don't think many other technologies have that immediate emotional effect.".
Michelle's love for new and exciting creative tools spans the immersive industry.
"My love affair is really with all things spatial because I'd be remiss not to mention volumetric video's cousin, photogrammetry," Michelle says, "and you can't have volumetric video without spatial audio."
As the name suggests, photogrammetry involves taking (a lot of) photographs of an object or scene from all angles and stitching them together to create a 3D model. Photogrammetry has many applications and is used commonly in mapping, architecture, engineering, archaeology, and increasingly in retail.
Above: Emmanuel de Maistre’s one-scan-a-day 2021 project displayed on Sketchfab. Emmanuel is co-founder of photogrammetry start-up Scenario.
Who is using volumetric video?
As we have learned from Michelle, music is one of the biggest industries benefiting from volumetric video, but some of the world's more prominent brands have started to utilise the technology for advertising, product services and even customer services.
- Balenciaga completed their real-time Unreal Engine experience Afterworld with volumetric video models for their Fall 2021 collection.
- Nike's Virtual View allows customers to try on their products virtually before purchasing. Being WebAR makes it super accessible.
- “Jadu is an app I’ve been promoting for years”, says Michelle. It allows fans to perform alongside holograms of their favourite musicians and performers, and they’ve recently expanded into NFT AR collections. “I’m a huge fan!”
- Burger King created a WebAR experience that allowed customers to experience a private AR performance by Tinie on a Whopper burger.
- The New York Times Magazine created an augmented reality experience that brings a hologram of model Ashley Graham into their living room.
"I was so excited to see volumetric capture show up in SpaceJam," Michelle tells us. "As a technologist, you can love a tech all you want, but others have to realise its potential and embrace it with you. You have a spidey sense for when techs are about to go mainstream or reach a tipping point, when you see them ‘in the wild’ and not just in our bubble. So when I saw capture-tech show up in SpaceJam last year, I knew millions could easily grasp the idea."
Above: Nike’s Virtual View leveraged volumetric video to let their customers try on products digitally.
To keep her finger on the pulse with these advancements in tech and to connect with her ever-expanding network, Michelle invests a lot of her time with tech groups and communities and even hosts workshops and speaking events that highlight how immersive technologies are for every brand.
"I went to a special Volumetric Video AWE Night NYC event in 2018 with Intel, Metastage, NY Times, Google and Scatter. It was an amazing intro to the companies on the cutting edge in the early days, companies I would later have the opportunity to work with," Michelle tells us.
Michelle stays active in volumetric community groups and in regular contact with her partners at capture stages and supporting tech platforms such as Arcturus, the makers of HoloSuite (an industry-leading tool for powering volumetric experiences). Arcturus also shine a light on the tech’s growth by publishing a global capture stage map every quarter.
Diving Into Immersion
One thing is clear: Viewers expect more than 2D experiences now, and consumers value brands that embrace new technologies. In this rapidly changing digitised world, businesses need to meet demands and find new ways of producing content to keep up with the competition.
"Volumetric video allows you to create experiences you simply can't make any other way — a prerequisite for me to be an advocate for a tech. We almost got through the whole interview without saying "holograms", but low or high fidelity, the feeling of presence is real. It's live theatre. It brings digital into the real world. It puts you into the action.
You don't need to understand how it works — we can take care of that — you just need to feel the potential of it, to feel that magic and know instinctively there's a reason for using it,” says Michelle.
“And I still have a-ha moments. On that shoot at Microsoft's studio, between takes watching playback on the monitor, the director reached out and rotated the volumetric clip of an actor to check his pants weren't causing issues with the capture when he crouched. I keep that clip on my phone to show people why volumetric still gives me goosebumps."
With a clear passion for volumetric video, Michelle Excell sees an incredible opportunity and untapped potential in how brands and creators of all kinds can take advantage of this unique and powerful media tool.
Michelle's creative technology consultancy, The Antipodean, works with brands and agencies to unlock the potential of new technologies. She understands that immersive technology can be overwhelming, so she works with her clients to create unique strategies to implement the right tools for their business.
The Antipodean's offering extends to production, where a collective of the world's leading immersive tech and experiential production companies work together with Michelle to make her client's immersive tech dreams a reality.
To schedule an inspiration call and discover the potential for your brand, reach out to michelle@theantipodean.us.
