From Tupac to Tino: Rap’s Holographic Legacy Continued

Q> What’s it like working with 3D volumetric capture; is it an entirely new way of working?

Roland Lane, Director> It is and it isn’t. The manner of filming is different, and you are making a huge amount of the creative decisions in the post-production phase as opposed to on-set, but fundamentally it's still about capturing what you need and having an overall vision. I think creative freedom is the main takeaway for me, which is incredibly exciting. We add all camera and lighting in post, so  I really can just focus completely on the subject and the performance on the day, which is a really effective way of shooting. To then have total creative freedom and room to experiment when we begin working with the footage is very liberating and opens up an almost unlimited amount of possibilities. plus, you can do as many takes as you require, there's no limit in that regard, so it's a very clean way of working.

Andy McNamara, Prodigious Director of CGI and Innovation> Unlike traditional film-making, where you tend to shoot from one point of view at a time, volumetric capture is significantly different. You shoot the talent and environment from every angle simultaneously and the director then makes the decision where to place the camera later in post. It allows for much more creative freedom. Other than that, once the raw data has been processed, wrangled and prepped, it's pretty similar to existing CGI and VFX workflows.

Q> How did you balance maintaining Tino’s humanness in his digital recreation?

Roland Lane, Director> From the outset, I knew I wanted to experiment with Tino's appearance and create something eye-catching. We could have created all kinds of CG environments surrounding him, but I felt as long as the visuals were built around Tino himself we would preserve that human connection. His movements are very expressive when he performs and he is a constantly interesting presence on-screen, so I felt that would be enough. His tattoos not only provided great scope for innovative VFX, but they were also an important constant throughout - we recognise them as being him from the beginning, so even when his form changes to a creation made up of metallic tattoos, or he is filled with light, we still feel it's recognisably Tino. 

Q> Were there any challenges creating a music video in this way?

Andy McNamara, Prodigious Director of CGI and Innovation> As with all new innovative tech, there were a number of technical challenges. We had to do a fair bit of massaging of the VolCap data to allow us to isolate parts of Tino's body to apply different effects; such as the glowing tattoos, trousers, shoes etc. Many of the use-cases for volumetric are currently targeted towards real-time AR, where the performance is played back in real-time, often on fairly modest hardware such as mobile.  Creatively there are a number of technical restrictions of how far you can push things in this way, so it was great to use all the usual CGI and VFX tools and techniques to give us a wider palette of creative options by utilising a more traditional post-heavy workflow. We worked very closely with the director, Roland Lane, to come up with a range of vignette sections that in many cases would be very difficult or impossible to shoot for real and/or time-consuming and expensive to have built in CGI completely from scratch.

Q> What considerations were taken when working on this 3D volumetric shoot? 

Brian Carbin, Stone Dogs Co-Owner & Flame Artist> Initially, we had to prepare the processed volumetric capture so the team at Prodigious were able to work with it and apply the complex VFX we all wanted to showcase. We had to find a way to extract mattes from the unique UV's that were generated, and to then figure out a 2D workflow as opposed to the AR/VR path volumetric footage has most commonly been used to date. This was a considerable undertaking for their team, and required a lot of work. After the final video had been created - camera moves, CG, VFX, grade - the footage came back to Stone Dogs. We added some custom-built lens flares, particle and glow effects and overall enhancement (to the already incredible looking work that came from Prodigious). We also spent time finessing Tino's edges and finally added some realistic film grain and the occasional light bleed to add those tiny details that always elevate a piece of work. Finally, we spent time getting the little things like the glow around lights and flares just right before handing it back to the team at Prodigious.

Q> Where do you see 3D volumetric technology taking the creative industry in the future?

Simon Windsor, Dimension MD> It's a hugely exciting time for creatives. With the rise of AR and MR, we'll see web and digital content becoming 3D - moving off 2D screens and into our physical worlds. With VR we can increasingly explore rich new 3D worlds - putting us in the heart of the action, rather than simply watching or observing as we do with passive 2D mediums. These rich, new, interactive mediums mean the next generation of content will be three-dimensional and require volumetric virtual humans. It's an exciting paradigm shift, that ultimately means the creation of experiences that we can jump into and really be a part of with our own digital avatars. Right now we’re just scratching the surface.

Q> Are 3D volumetrically captured humans a chance for some of our passed-on favourites to come back and perform?

Simon Windsor, Dimension MD> Most probably. Volumetric video capture films a living performance. We recently filmed celebrity lookalikes, including Marilyn Monroe, Prince, Amy Winehouse and others for an entertainment production but this relied on having the impersonators to film. Without this, you would need to digitally recreate people, as was the approach in Rogue One with Carrie Fisher. There are many ethical considerations here of course but it's a technique we can expect to see more of in the pursuit of resurrecting past icons of the screen and stage.