Camera Obscura: Mixing CG with Live-Action with Aggressive

Headed by the Grammy award-winning filmmakers Alex Topaller and Dan Shapiro, the creative production company crafts commercials, live experiences and design-driven content for some of the world’s biggest brands and artists such as Amazon, Boeing, Honda, Intel, Michael Jackson, Pharrell, Alicia Keys and The Beatles among many others.

They have earned many awards and nominations since our formation, including Grammy, AICP Next, MTV VMA, ADC and Communication Arts awards. Their work has been featured across the globe as Vimeo Staff Picks, Behance features and festival showcases, as well as publicized by The New York Times, American Cinematographer, Vogue, Vanity Fair, Wired, Creativity and NOWNESS among dozens of other publications.

LBB> What is your niche craft obsession?

Aggressive> It’s probably no secret, looking at our work, that we are positively hooked on shooting with LED screens on set - from giant realtime tracking XR volumes to small custom-sized LED panels integrated into practical set designs.

LBB> Where/ when/ how did you first come across this thing? 

Aggressive> Our very first LED project was a Huawei spot that we shot in Shanghai in 2016 - in which we featured talent in a fantastic graphical space built out of large-scale devices. The agency originally briefed it to be done with green screen - but we convinced them we could get a much more immersive, spontaneous, and light-driven film (with a much shorter post process!) if captured it all “in-the-lens.”  

Ironically we initially tried to do it with projectors - but at the time projection techs were too expensive in China, but they had boatloads of inexpensive LED panels since they were all manufactured there, so we used them to build the set - dynamic walls, ceiling, and floor, all active and driven with animation and light - and the results were incredible!

LBB> Was it an obsession straight away or something that has evolved over the years?

Aggressive> Mixing design and CG with live-action has always been in our DNA - but the fact that this could now be captured live in-camera was a lightning strike. 

That we could now shoot inside of these design-driven environments, without green screen, and see the real light reflecting off or through our subjects, see the real organic reactions in our talent performances to what they are seeing, and move our camera freely to find shots without concern for tracking or keying in post was liberating. 

And now, with the frontier of this technology advancing so rapidly, with the constantly improving visual quality and responsiveness of real-time CG engines like Unreal, paired with more powerful GPUs, camera tracking, and motion capture - we’re constantly trying to push ourselves to figure out new and interesting ways of using it all.

LBB> What are the most interesting debates or conversations you are having around this obsession?

Aggressive> LEDs and Virtual Production are such new techniques, that there’s still a good amount of learning curve among our clients, and exploration for us, on how and when it can be best used as opposed to more traditional methods, to make the most interesting and suitable result for the project and the budget - ie. build something practically, vs. build it in Unreal… shoot on location vs. shoot in a volume…  

Ultimately we don’t view LEDs and Virtual Production as replacements for other methods of filmmaking, but simply as another exciting new set of tools among all the others we have at our disposal as live-action filmmakers, designers, and CG/VFX artists - and we’ve found the strongest results emerge when the pursuit of the most interesting visual concept drives the technique and not the other way around.

LBB> How widespread do you think this obsession is with your peers?

Aggressive> XR, LED, and Virtual Production are all hot catch-words now for sure, both among our colleagues among studios, directors and artists, as well as agencies and clients with whom we’re chatting about the possibilities of things to come.

LBB> Can you share any examples of work where that obsession really came to the fore and elevated the final production? Can you tell us about it and share links if possible?

Aggressive> OnePlus “Nord 2 5G”

Filmed over three days on stage in Warsaw against a massive LED wall - we took a truly mixed media approach that seamlessly combined richly crafted practical sets, experimental camera moves, forced perspective (and an inflatable bird puppet!) with dynamic LED-projected background environments that mixed collage and motion-design with cutting-edge Unreal Engine animation. Rather than use the LED wall simply as a substitute for conventional location plates or set backdrops - we turned it into an active motion element for dynamic scene transitions and effects, animating and composing our projected backdrops to exaggerate the surreal camera moves, perspectives and energy we wanted to imbue into each shot: time-lapse forest wind-storm, a first-person video game run, an LED driven dolly-zoom, a subway tunnel, and hyper-color motorcycle ride.

Aggressive> 'AMEX Unstaged'

We shot this hour-long live XR concert on a pre-built LED stage - where we crafted an arresting three-act digital world structure - a desert, a city, and an abstract architectural setting. With only three weeks lead time, and a final set list not arriving from Alicia’s team till morning of - we relied on the versatility of the realtime engine Notch to create a variety of triggerable effects, light and color changes, and animated scene objects, that could be cued and directed in real time to match the emotional arc of her performance. 

In addition, we were able to use augmented reality XR to transform and extend the small space far beyond its physical boundaries, not only making a visual feast of expansive scale for viewers at home, but also immersing Alicia in the changing environments, so she could play off her surroundings during the set. The energy and spontaneity was electric - she was quoted in Forbes afterwards saying it was “This Was Her Most Amazing Performance Ever”, 

LBB> For anyone just getting into your field, what advice would you share to help them get their head around this particular thing?

Aggressive> I think given the technical nature of LED and virtual production work, there many “avenues of attack” here… 

On a conceptual level, it helps to check out any number of good BTS and “Making of” videos of LED and virtual productions, ie. our work for Amazon “Inside the Boys”, or the BTS’ for “The Mandalorian”, “The Batman,” “The Joker” train scene, and many many others…   To wrap one’s head around what it is at a top level. 

In essence, it’s a form of compositing but in-the-lens - how do virtual background and practical foreground perspectives play together, what kinds of camera moves and lensing works or not, and what’s the interplay between practical vs. virtual lighting and color-correction, to make sure the immersive illusion holds up.

On a technical level, it helps to get acquainted, at least on a high level, with the workflow and interplay between a real-time graphics engines like Unreal Engine, content  servers like D3 or TouchDesigner, LED screens, and real time camera tracking systems - basically, how all the pieces fit together to make a believable final image.