How This R&D Studio Is Revolutionising In-Camera Tech for the Likes of Taika Waititi and ‘Thor’
Carlo Van de Roer is a director who is repped by CASEY in the US. But his filmmaking influence stretches further than what he oversees behind the camera.
Carlo is also head of filmmaking and a co-founder of Satellite Lab, an R&D Studio that has created a series of revolutionary, patented in-camera technologies, with which he recently collaborated with Taika Waititi, on Thor: Love and Thunder, and previously on Thor: Ragnarok.
Satellite Lab's patented systems are:
- DynamicLight: Enables the use of moving light sources within footage captured at any speed, including the frozen world of high-speed imaging. When using a high-speed cinema camera and light sources moving at over 10,000 feet per second, this technology creates the ability to move light sources within images frozen in time.
- PlateLight: Provides the ability to capture multiple lighting set-ups simultaneously on the same camera. Each is recorded as separate footage of the same live action, without time offset. Works with moving camera(s) and subjects.
Intrigued to know more about the steps that lead to filmmaking innovations such as these, and his general thoughts on the link between technology knowhow and directing prowess, LBB's Addison Capper picked Carlo's brains.
LBB> You’re a director with a passion for technology - how do the two go hand in hand?
Carlo> I think that the more you know and understand about the tools you use, the more you can use them in favour of your creative work. Part of what I'm interested in doing as an artist is intervening in the way we are used to seeing our world represented through photographic technology, and a satisfying way I've found to engage in that is through intervening in the technology itself. So, around 2012, I started to shift into a practice of building processes and technology I wanted to use, and I’ve found a lot of creative freedom in doing so.
LBB> You’ve created a series of revolutionary, patented in-camera technologies - can you tell us more about those? What do they involve and how do you go about building them?
Carlo> I founded an R&D studio called Satellite Lab with my friend Stuart Rutherford. Initially, this was to create a technology we now call DynamicLight. This started as an idea I had for an approach I wanted to use in my work. I was in a program at the New Museum in NY which was focused on artists building sustainable practices when we were working on this tech, so rather than just build it for a single project, I looked at Satellite Lab as a place that could help push the R&D of ideas like DynamicLight further and turn them into toolsets which could continue to be built upon.
DynamicLight is essentially a way of controlling moving light sources within footage and we can do this at very high speeds – effectively moving light sources many times the speed of sound. This makes it an interesting tool for slow motion work because, for the first time, we can control moving light sources within images that are arrested in time - time appears frozen for the scene, yet light sources are still able to move within these frozen moments. The original idea for this came from wanting to upend the way we are used to seeing time represented in moving images. The frozen world of high-speed footage is at one end of the spectrum and timelapse at the other end, where moving light represents the passing of time. I wanted to mess with those constructs; the upshot is that DynamicLight is now a way of controlling time for the scene and for moving lights as independent variables.
This led to another area of R&D, exploring the idea of filming multiple lighting conditions at the same time on the same camera. For example, looks from different times and places, or different ‘recollections’ of the same moment. A simple example would be day and night, a more complex version would be the various light sources used on a film production, each recorded as a separate piece of footage which can be combined after the shoot. I spend a lot of my time currently exploring the possibilities offered by this approach which we call PlateLight.
LBB> I imagine this R&D side of your work also means you’ll work with other directors, if they need to use your creations. How do you enjoy that challenge?
Carlo> To date there have been a couple of collaborations with Taika Waititi and one with our friends at Psyop. I am interested in seeing these filmmaking approaches used with new ideas, especially in film and, in particular, I’m interested in seeing where we can take PlateLight in the near future.
LBB> How did you become a director in the first place and how did your interest in technology build alongside it?
Carlo> I’ve been using photography and filmmaking in my work as an artist for a long time, and when the opportunity arose to extend this into commissioned work it seemed like a great opportunity to collaborate with larger teams and play with new ideas.
Working across the different fields of feature films, fine art, experiential / installation, and commercial work is really fun as they all cross inform each other but have quite different demands. An idea that percolates in an art project might get rethought and refined as a filmmaking approach in a commissioned project and then scaled up for a film, only to lead to a new idea, etc. For Satellite Lab, it’s kept us on our toes, developing and building.
Carlo
LBB> How important is knowledge of filmmaking technology to be a director, do you think?
Carlo> I’m someone who likes to know something about each process that is used in a production. I worked briefly enough as a DOP and in VFX to be annoying to each department when working on a project now. But I find that knowledge really helpful in understanding the potential for in-camera approaches and how they can integrate with post processes. I think that understanding your tools well can help you know when to use what and how to apply new uses to older technology.
At the same time, I think it is also important to know your strengths and your weaknesses and to focus energy on the right places. As projects I’m involved in have gotten larger over the years, I've been learning more about delegating well and finding incredible collaborators.
LBB> Can you talk us through one or two of the most exciting recent productions that you’ve been involved in that you think had a really interesting innovation or technological aspect to them?
Carlo> I had a great time directing a recent series of spots for LG’s Signature series. We worked with Misty Copeland from the American Ballet Theatre Company (ABT) who is an incredible dancer and person. It was a crash course in choreography for me, working with James Whiteside, also from the ABT. I’d worked with James previously when I directed Fischerspooner’s last music video, and this was an opportunity for us to have some fun. At the same time, it was technically a challenge as we had a short amount of time to capture content for a number of different spots, and I wanted to capture the content in-camera with PlateLight synchronised to VRI and Sony cameras and some large motion control rigs tracking Misty.
One of the spots was for a pretty wild product - a thin rollable OLED TV, like a plane of illuminating glass that rolls out of a box. We used our lighting tech to create an environment of planes motivated by glass and reflections which Misty could interact with. These planes could change in response to her dance or the music.
This was during covid - I think it was one of the first large productions to shoot in New York after the lockdown, and everything seemed to happen at half speed, but I was blessed with an epic team and commissioned a terrific violinist, Ezinma, to score the music. I feel very lucky to have worked with these incredible people.
Another recent fun one was for Lexus. We wanted to bring a human, emotional experience to the spot, and both Lexus and Team One, which was the agency, let me have fun and try an experiment. We tracked three people through their experience of Lexus’s LS 500, monitoring biofeedback like heart rate, etc. via sensors on their body. The data from those sensors was used to drive the lighting, creating a colourful, visual depiction of their experience of the car.
LBB> Can you tell me more about your collaboration with Taika Waititi on Thor?
Carlo> We worked with Taika on Thor: Love and Thunder, which came out recently, and on Thor: Ragnarok, previously. Taika’s films are a lot of fun, and the fact that he and the whole Marvel family have been willing to take a chance on experimental filmmaking to push things forward is exciting and encouraging to me. I'm really proud of the scenes from both of those films - they sit within a visual language I’ve been developing for years, but at a scale I never could have imagined when I started this journey.
For Love and Thunder, we helped create looks based on in-camera lighting techniques for battle scenes involving a lot of light-emitting weapons, transitions between black and white and colour, and a way to change the angle of a constantly moving sun in post for editorial continuity, all using real in-camera footage. Doing these kinds of things in-camera can mean the footage looks more believable and it can ease the post process for films which could otherwise involve more CG. Our main focus on this film was the Moon of Shame scene, with Christian Bale as Gorr.
For Ragnarok we helped develop a look for the Valkyrie flashback scene which was filmed with DynamicLight. Because this scene was a recollection, time is out of whack, moving in a thick, almost soupy way, yet light continues to explore and reveal the scene in real time. This is a look and feel I had been working on for a long time, so getting to bring it to life with Taika, Cate Blanchett, and Tessa Thompson, and the rest of the team was an amazing experience.
Taika is courageous and innovative, and it's been a joy to collaborate with him and VFX supervisor Jake Morrison and so many others on these films. Plus, Stuart (who co-founded Satellite Lab with me) and I got to spend several months in Australia last year, which is close to the greatest place on earth, New Zealand, which is where we’re both from.
Disney's making of Love and Thunder explains PlateLight and how it was used in the film. Check it out here on Disney+. This episode of the podcast Befores and Afters also discusses it.
LBB> Quite often production involves trying to solve a problem that’s never been attempted before - and that can mean hacking existing technology or trying to find new technologies. When you get a project that has such technological challenges, how do you and your team like to approach them?
Carlo> I love projects that come with those sorts of problem-solving needs baked in - it’s partly why we built Satellite Lab as an R&D space with resources to problem solve, hack, and find new approaches. Our main focus has been in-camera solutions like camera and lighting tech, but as projects with different problems arise, we’ve used our hardware and coding skills in new areas like the blockchain or to develop new interactive and experiential work and embrace new platforms.
We also spend a fair amount of our time developing solutions in the lab that don't yet have a home. So, when a project comes to us, we have a kind of secret box of toolsets we can pull from. I think a good example of that would be our day/night technique, which I mentioned above as a way of filming a day and night look in one take; it’s a completely in-camera approach and looks 100% real - because it is. This is something I’m really excited about getting into.
LBB> More generally, what are the technologies that you have your eye on that either are having a big impact on how production is done - or have the potential to change things in a big way?
Carlo> In-camera tech is moving fast now, cameras are advancing fast, as is lighting, and VFX bottlenecks are disappearing - everything feels like it’s reaching a point of less friction, so I think we’ll see more integration of different approaches within single productions. As for things I have my eye on, I’m in a learning curve in understanding machine learning currently, but it seems like there is enormous new potential there.
