Dentsu Lab Tokyo's All Players Welcome Enables 'Physical' Expression in Metaverse

The ALL PLAYERS WELCOME project from Dentsu Lab Tokyo was established to develop tools that will create a world where anyone, including those with physical disabilities, can express themselves. With their first project, the team gave paralysed DJ, Masatane Muto, the ability to mix music through eye-movement technology. This Friday, they will be presenting their latest project at Ars Electronica Festival in Austria, making it possible for Masatane to operate an avatar using ‘Electromyography Signals’. ‘Project Humanity’ (Project #2) is a world first use of this technology and will allow the DJ to express himself ‘physically’ in real-time when he performs to his fans in the Metaverse or on a digital screen, and at the festival.

In partnership with ‘Nippon Telegraph and Telephone Corporation’ (NTT) and ‘WITH ALS’, an organisation established to support people living with the Amyotrophic Lateral Sclerosis (ALS) by Masatane Muto, ‘Project Humanity’ detects electrical signals transmitted from the brain to move muscles with electromyography sensors. Working with Masatane Muto to develop the prototype, and using NTT's technology to detect the signals which are converted into data to manipulate avatars in the Metaverse. Located in six areas on his neck, arms, and legs, signals can be controlled in each location (on/off) to perform actions such as raising hands, jumping, and walking.

Dentsu Lab Tokyo launched ALL PLAYERS WELCOME in 2022 with the aim of creating an environment where all people with disabilities can express themselves freely, based on a sense of the potential of creativity and social issues faced by people with disabilities. For its first project Dentsu Lab Tokyo developed three music tools that could be operated using only eye movements, the ALL PLAYERS TOOL LAB. ‘EYE XY PAD’ specialised in real-time performance, ‘SHOOTING PAD’ solves the issue of remote performance, and ‘EYE MIDI PAD’ focused on integration with the Ableton Live UI. Debuted at the Cannes Lions International Festival of Creativity 2022 in France, this was the first time in the world that someone with ALS created and performed live music.

Naoki Tanaka, creative director, Dentsu Lab Tokyo says “With the evolution of our technology, we are building a future in which people with physical disabilities will be able to operate avatars through their own will and senses in real time through the power of electromyography signals. Physical expression is an essential element to communication and performance and Project #2 makes this possible for the first time”

Masatane Muto, DJ and founder, WITH ALS added “I am very excited to free my body after a such long time due to ALS, to now be able to dance and clap to my music with audiences around the world. Through the power of technology and creativity I want to overturn the stereotype that ALS patients' bodies are immobile”.

Naoki Tanaka, from Dentsu Lab Tokyo, and Mariko Nakamura from NTT will be speaking at Ars Electronica on Friday 8th September, with DJ Masatane Muto performing a live set to showcase Project Humanity.