Immersive Developments To Look Out For In 2021
The global pandemic has accelerated the adoption of immersive technology—virtual reality, augmented reality, and mixed reality—and this momentum shows no signs of slowing as we move into 2021.
There are plenty of developments ahead that will shape the future of the immersive industry and have an impact on how we live, work, and play. REWIND has compiled a list of what to look out for (in no particular order) in the new year. There’s a lot to look forward to!
The Release of Unreal Engine 5
Since it’s preview in June, we’ve all been waiting to get our hands on Epic Games’ Unreal Engine 5 because of it’s game-changing core technologies: Nanite Virtualised Geometry and Lumen real-time global illumination.
From the very beginning of commercial 3D gaming in the early ’90s, developers have been hampered by polycount restrictions (the maximum number of polygons — typically triangles — that an object or scene can have). As both hardware and software evolve, we have been able to push these limits to achieve remarkable levels of detail and quality, using per-pixel shading techniques, such as normal mapping to simulate higher resolution geometries, and building LOD (level of detail) systems to handle swapping between higher resolution and lower resolution models based on their size on the screen. UE5 is making this process easier with Nanite virtualised geometry.
Nanite virtualized geometry will allow artists to create as much geometric detail as the eye can see. It means that film-quality source art, comprising hundreds of millions or billions of polygons, can be imported directly into Unreal Engine and it just works. Nanite geometry is streamed and scaled in real-time, so there are no more polygon count budgets and there is no loss in quality. It will enable developers to focus on the content itself, and not the technical hurdles they have historically encountered.
Lumen is a fully dynamic, global illumination solution that immediately reacts to scene and light changes. Artists and designers can create more dynamic scenes using Lumen. For example, changing the sun angle for the time of day, turning on a flashlight, or blowing a hole in the ceiling, and indirect lighting will adapt accordingly. No more waiting for lightmap bakes to finish and to author lightmap UVs, which is a huge time saving for artists who can move a light inside the Unreal Editor and the lighting will look the same as when the game is run on a console.
Unreal Engine 5 will be available in preview in early 2021, and in full release late in 2021, supporting next-generation consoles, current-generation consoles, PC, Mac, iOS, and Android.
Ubisoft’s AAA franchises Splinter Cell and Assassin’s Creed Coming to Oculus
When the Oculus Quest 2 was announced in September this year, Oculus also revealed that Ubisoft was bringing two of it’s AAA franchises to Quest 2 with new entries in the Assassin’s Creed and Tom Clancy’s Splinter Cell universes. Both games are being developed from the ground up by Ubisoft’s Red Storm in collaboration with Ubisoft Reflections, Ubisoft Dusseldorf, and Ubisoft Mumbai.
“We are always exploring new ways we can leverage technology to help us create memorable gaming experiences,” said Elizabeth Loverso, Vice President of Product Development, Red Storm Entertainment. “Oculus’s state-of-the-art hardware will enable us to bring the worlds of Assassin’s Creed and Tom Clancy’s Splinter Cell to life for fans in a new way. These games will take full advantage of Oculus’s technology to deliver immersive, visceral experiences that players won’t be able to find anywhere else.”
The games are set to be released in 2021, and we’re looking forward to seeing how these games–built from the ground up for VR–perform. If they are anything like the quality of Half-Life Alyx, we are in for a treat!
Facebook’s Smart Glasses and Project Aria
Facebook announced its new research device called Project Aria at this year’s Facebook Connect event. The project aims to build an AR glasses interface that will “fundamentally change the way we interact with and experience the digital world — an interface that’s proactive rather than reactive, that’s intuitive, that understands our intent and acts almost before we know we need it. An interface that, for the first time, puts us at the centre of our technology, rather than requiring us to adapt to it.”
Project Aria is worn like a regular pair of glasses and will help build the software — including a live map of 3D spaces — and hardware necessary for future AR devices. Sensors on the Project Aria research device will capture the wearer’s video and audio, as well as their eye-tracking and location information. The glasses’ on-device computing power will then be used to encrypt and store information that, when uploaded to a separate, designated back-end storage space, will help researchers figure out how AR can work in the real world.
Facebook also announced that its first pair of consumer “smart glasses” will be released in 2021 as a branded Ray-Ban project. The glasses are part of a multi-year partnership with EssilorLuxottica (makers of eyewear from Oakley, Ray-Ban, Armani, and Versace) and will be designed to help better connect with friends and family. Importantly, they are not going to be classified as an AR device and will not have an integrated display of any kind, but will bring the company one step closer to a fully-fledged pair of AR glasses.
Super Nintendo World’s AR Mario Kart Ride
Universal Studios Japan’s Super Nintendo World opens in February 2021 with an AR Mario Kart-themed rollercoaster and Mario branded AR headset.
In a press statement, the company revealed the Mart Kart AR ride will combine several state-of-the-art video technologies, AR headsets, and projection mapping. The ride takes place in a recreation of Bowser’s castle and gives each rider an AR headset (it’s not clear whether it is an existing AR headset that’s been modified for this experience, or whether it’s a new piece of hardware entirely) and a steering wheel to race against other riders.
Thomas Galati, Universal Creative and Executive Producer said: “During the race, guests will throw the swords they got from the item box to repel the enemy and will compete for the first place with Mario and Princess Peach to advance the course. The experience changes each time you ride, such as winning at one time and losing at another time, and it is a completely new “Mario Kart” experience that everyone can enjoy.”
It is an ambitious launch date, considering the current state of travel and Osaka’s COVID-19 risk, but the park has addressed this issue early on by enforcing a new set of practices which includes reminders to socially distance, wear a mask, and use Osaka’s COVID-19 tracing app. It will be very interesting to see how location-based entertainment bounces back after the pandemic, and if the public will feel safe enough to return to it.
The End of An Era: Rift S Discontinued
With the arrival of the Oculus Quest 2 came the departure of the Oculus Rift S. Facebook revealed it would discontinue the line in 2021 and no longer pursue PC-only VR headsets. Although the company said that “the Rift platform isn’t going anywhere”, it’s clear that the Oculus Quest and Oculus Link hardware are the primary focus going forward.
Quest 2 product manager Prabhu Parthasarathy told The Verge that the Quest 2’s release was “the right moment for us to move to a single headset” and that “Quest 2 is the best of both worlds when it comes to VR experiences”.
It’s the end of an era for the Rift, which initially began as a Kickstarter project in 2012 and gained $2.5 million in support. The first developer kit — DK1 — was released in 2013, and it was the first piece of hardware that made REWIND pivot from high-end VFX, CGI, TV commercials, and indie feature films to immersive technology.
Oculus was then acquired by Facebook in 2014, and the Rift was turned into a high-end device that competed with the HTC Vive headset and Sony’s PSVR. The Rift S was released as an upgrade to the Rift and incorporated key Quest features like self-contained, inside-out camera tracking.
With the Quest line being the focus of Facebook’s future endeavours, we will be watching to see how they evolve the technology of standalone headsets to compete with PC tethered ones to provide the perfect all-in-one solution
Varjo’s New XR-3 and VR-3 Headsets
As much of the world continues to work remotely, demand for virtual and mixed reality use cases continues to rise. The latest offerings from Varjo promise to deliver the most natural mixed reality and virtual reality experiences ever constructed, providing the means for scaling VR/XR adoption within every industry.
Featuring the industry’s highest resolution (over 70 ppd), and field of view (115°), integrated hand tracking, XR-3 inside-out tracking, and photorealistic mixed reality depth awareness, professional users can perform at their highest level in a fully immersive environment.
Varjo’s XR-3 and VR-3 headsets will be the first that bring Ultraleap’s fifth-generation hand tracking, known as Gemini, to life. They also mark the first mixed reality headset with Ultraleap’s hand tracking.
Both the XR-3 and VR-3 headsets are available for order now or through one of the more than 35 authorized resellers in the company’s global reseller network. Shipments start in early 2021.
Nreal Light’s Debut in Germany and Spain
Initially launched in South Korea and Japan, the Nreal Light Mixed Reality glasses will be debuting in Germany and Spain in Spring 2021, with other European markets to follow. The glasses will utilise Vodafone’s high-speed, ultra-low latency Gigabit 5G network to deliver high quality augmented and mixed reality experiences.
Designed to be worn daily, Nreal Light glasses are lightweight and comfortable. They have an industry-leading widescreen, equivalent to viewing a portable IMAX screen. These are the first consumer smart glasses to offer an affordable, high-quality AR and MR experience with an advanced understanding of the surrounding environment, including six degrees of freedom tracking, plane detection, and image tracking.
Nreal CEO Chi Xu said: “European consumers will be able to experience the future through Nreal Light. They’ll enjoy a new way to see and interact with their favourite apps in a 3D mixed reality environment on Vodafone’s 5G network. We’re excited to be making Nreal Light available in Germany, Spain and other European countries in partnership with Vodafone as we continue our mission to make Mixed Reality accessible to consumers around the world.”
The Rise of Avatars
Rapid technological advancements, such as the evolution of AI tools and next-generation game engines, are poised to usher in a new era of hyper-realistic AR avatars. For example, in 2021 Sensorium Galaxy — a virtual reality media platform that provides users with access to an all-digital alternate universe — will go live with its state-of-the-art photo-realistic avatars.
Users will be able to create complex virtual representations of themselves without being limited to pre-set avatars. Not only will users be able to personalise their appearance but they will be able to set character traits too.
Brian Kean, Chief Communications Officer at Sensorium Corporation said in a press release: “By adding AI technology to the mix, the immersion of user-generated avatars in social and game mechanics goes further than ever before. Artificial intelligence allows the avatar to exist autonomously in virtual environments, maximizing the interaction with other users. At Sensorium, we call it the ‘digital evolution’ of the user. Naturally, this advancement will lead to ‘digital immortality’ — the next stage of this project.”
Signed Distance Fields
If you haven’t been following the developments with Signed Distance Fields (SDF), you need to be in 2021. ‘Signed Distance Fields’ is a fancy term for something relatively simple: an SDF is just a function which takes a position as an input, and outputs the distance from that position to the nearest part of a shape. You can have a play with SDF’s in this web application if you’re unclear about what they are.
This novel representation of 3D models offers a huge potential to change how 3D is made, as well as enable more people to create immersive content. This new exciting format burst into the public eye with Media Molecule’s game Sony Dreams, which allows creators to make beautiful works like this, without any complex 3D modelling software.
Andre Int is pushing the boundaries of what is possible in Unity with SDF through his plugin Clayxels. It allows for easy modeling inside the engine with awesome results:
Finally, in this category, one to watch is the team at Unbound who is using a proprietary engine they built based on SDFs to deliver 3D tools for designers and creatives. If you’re looking to create collaboratively without needing to break your creative flow, this could be for you. We’ve seen the Alpha and it’s looking promising:
The Auction of 5G Spectrum
Ofcom has confirmed that the auction of 5G spectrum will begin in January 2021. The auction will increase the total amount of airwaves, or “spectrum”, available for mobile in the UK by nearly a fifth (18%).
This will enable better and faster services for consumers and businesses. More spectrum tends to equal better mobile broadband speed and that’s going to be key for 5G.
Some say the future of immersive is reliant on 5G. Why? Both VR and AR require a cheaper, more substantial network with lower latency and more consistency if they are to continue to evolve. In short, 5G is a necessity.
Qualcomm 2021 flagship chipset — Snapdragon 888
Qualcomm has been a leader in mobile gaming technologies for over 20 years and is the only major supplier of generally-available high-performance mobile chips. The company has revealed the chip that will power the majority of 2021 Android flagship phones — Snapdragon 888, the successor to 865.
It features the new Kryo 680 CPU, Adreno 660 GPU, 6th generation AI Engine with the Hexagon 780 DSP, Spectra 580 ISP, Quick Charge 5, and the Snapdragon X60 modem-RF system.
What does all that mean? Qualcomm claims Snapdragon 888 will provide 35 percent faster graphics rendering while being 20 percent more power efficient when compared to its predecessor. This boost in graphics processing power enables the most advanced HDR features for mobile gaming and easily powers framerates up to 144 fps.
Whilst we don’t yet know the costs, reports have suggested this new chipset could be over 50% more than its predecessor. If this is true, it may be too pricey for incorporation into consumer VR devices.
We’re interested to see what the major hardware players do it with. Will Facebook launch an enterprise-focused HMD? Will HTC update the Vive Focus Plus from Qualcomm’s 2017 flagship (Snapdragon 835)? Watch this space.
