The Difference Between XR and Virtual Production

Recently Studio Giggle has been talking a lot about XR studios and virtual events, but there is more to XR than just events. XR stages can also be used for virtual production or ‘VP’, which we believe is a massive development in production and will inform the future of the film and TV industry.

Virtual production is, as the name suggests, using technology derived from VR and gaming to produce shots for TV and film. It uses an identical setup as we use for our XR studio, but the virtual environments can be used for filming instead of events.

How does XR work?

XR stands for Extended Reality. It is a progression from AR and VR. We term XR to describe the process of extending the virtual set beyond the LED volume. In an XR studio, there is a volume, or cave, constructed from LED tiles. These screens show environments that have been created in real time generative software or gaming engines such as Notch or Unreal Engine. To create a whole world beyond just an LED backdrop, we combine camera tracking and real time rendering. This technology enables the content on these screens to be rendered dynamically from the point of view of the camera. So when the camera moves, so does our content.

Talent can be placed in the XR stage within a computer-generated world, creating the illusion of a real environment. Unlike with a green screen in which you can’t see your surroundings, the background can move with the camera, and the actors do not have to imagine the place they are in, they can actually experience it directly.

So what is Virtual Production?

Virtual production uses exactly the same process, but very often the entire production is captured in-camera, without the need for set extensions. VP workflows heavily rely on the art department- XR tends to use an LED floor in conjunction with the walls, whereas VP will most likely have the foreground or other parts of the background as an actual set and props. The art department and Unreal artists need to collaborate closely together to create the whole of the look to be captured in camera.

Benefits of Virtual production

The main benefit of Virtual Production is saved time. Often in effects-heavy production, the term ‘we’ll fix it in post’ is heard over and over again. The beauty of VP is that so much of the effect is created in the pre-vis stage. You are in essence doing the post-production in pre-production. You can plan your shots and see exactly what you want on the screen.

Your subject is lit naturally by the LED volume. In the hugely successful Disney production ‘The Mandalorian’, the eponymous character is by his very nature, shiny. A green or blue screen effect would be hellish in post. In the Virtual Production environment, the reflections on his armour are real reflections from the LED volume.

Next, there is the cost element. Reducing the number of locations needed in a production by filming using an LED volume in a studio saves on having to pay hefty booking fees at multiple locations. On that note, the reduced need to travel the world to find the perfect location, not only costs less but has huge environmental benefits. By using Virtual Production you could be in Paris in the morning, Las Vegas at lunch and Thailand in the afternoon. One of the biggest additional costs in traditional production is if a reshoot is needed as you would need to get back on location, set everything up and build it in exactly the same way. Using Virtual Production, this is not an issue, you can just load the same background onto the LED walls and then you are good to go again with no extra hassle.

One of the hardest parts of traditional production is making sure the setting is just right. This could mean having a 60-minute window whilst the sun sets to get the perfect take (magic hour), or relying on it being sunny on a rainy day – the chances are you will not get what you need. But with virtual production, you don’t need to worry about any of that. You control what is on screen at all times, so if you want it to be sunset for 3 hours, it can be. If you want it to be sunny on a winter day, it can be. This greatly reduces time being spent waiting for the perfect conditions.

Another benefit, in the time of social distancing and covid, is that being in a virtual environment helps enormously for keeping the cast and crew safe in a controlled environment. You can operate at a distance and also plan so much in advance collaborating online. The fewer people on set makes it easier to social distance and to covid-test employees.

History

Virtual production isn’t a new idea, but it is certainly rising in popularity now. Some elements of virtual production come from traditional rear screen projection. Rear screen projection was mainly used in the film industry in the 1930s-1960. An actor would stand in front of a screen while a projector positioned behind the screen casts a reversed image of the background. This allowed directors to film scenes of people driving whilst on a static set. The one downside, however, is that everything did have to be static, the camera could not move otherwise the illusion would be ruined. This limited the capabilities of rear-screen projection.

More recently virtual production has been used in a number of different ways. In 2019 Jon Favreau used elements of virtual production to help him film the live-action remake of The Lion King. He and his crew would wear VR headsets which allowed them to see the virtual world. Seeing the animated environment in front of them, like it was real, allowed them to better plan content and camera angles. This is something that was first done in 2009 by James Cameron for Avatar. By using a screen showing the animated world Cameron was able to watch this at the same time as the actors were performing live in front of him (Check out the video below of Jon Landau explaining how they used virtual production). This meant that he got an overview of how everything would look in post.

Nowadays every big ticket TV production has a previz team that can plan all the shots in advance of very expensive TV productions such as Game of Thrones or His Dark Materials.

These early combinations of animated world, live-action and AR/VR have all led to the creation of virtual production as we know it now.

So virtual production means that you can shoot scenes anywhere in the world, with any weather condition and move from one to the other, all whilst staying in the same studio. It means that production can be faster, cheaper and ultimately smarter than traditional production. You can be on a beach in the Maldives one minute and Mars the next. The possibilities of production are now endless.

We think that sounds pretty great to be honest!

We have loads of projects coming up using XR and Virtual Production and can’t wait to see how virtual production is used in the coming years and the effect it has on the film and TV industry.