This AI-powered Service Allows Blind People to 'Watch' Music Videos

For the first time, this AI-powered service enables blind people to ‘watch’ any music video.

Just because the blind cannot see, doesn’t mean they cannot enjoy various forms of videos such as movies and videos on YouTube through music, dialogue and sound effects, which helps paint the picture in their minds. 

But there’s one thing that those who are blind have never experienced before, music videos, and since most music videos are mainly music and visuals that tell a story, for those who cannot see, watching a music video gives them no further enjoyment other than listening to the music track. Until now.

BBDO Bangkok, in partnership with Kunana, Facebook Thailand and TAB Foundation have launched Visible-Track.com, the first online service that enables blind individuals to enjoy millions of music videos that are available online. By simply entering the website with accessibility features, then copy and pasting any music video link from YouTube or Facebook Watch, the website uses AI to intelligently add sound effects to each scene automatically to help one paint the ‘scene’ in their mind. To do this, the service utilises as many as 6 Ais, e.g. scene splitting, scene classification, action classification and object classification that work together on different tasks to help music videos become visible and universally accessible to everyone.

Additionally, the service also uses image-captioning AI and text-to-speech AI to generate audio descriptions to describe what’s going on in the scene. Alternatively, blind individuals can also use the Visible Track service via a messenger bot on the TAB Foundation Facebook page (@TNIBinfo).

When asked about the campaign, Napapatch K. Creative Strategist at Facebook Creative Shop Thailand said, “We are proud that the Facebook platform is able to alter the world of the blind and change it for the better, making it more full of life.”

“The task of visualising music video for blind people is unprecedented”, shares Kunat Pipatanakul, chief technology officer at Kunana, “thus we do not have any proven solution as a reference. We need to come up with a novel solution that is generalised enough to handle any music video out there. As a result, we have developed a cluster of multiple AIs. Each AI is specialised in different subtask that together they can provide a complete solution.”

“Prior to this project, the art of music videos has never been accessible to the blind, but today’s technology enables us to reduce inequality and improve the quality of life for the blind,” commented BBDO Bangkok’s CCO Anuwat Nitipanont. “Visible Track was not created to help blind people see music videos in the same way a person with vison would. But it is designed to give them an enjoyment of music videos as any person deserves. Like music, music videos are an artform which everyone should be able to enjoy as equals”, he concluded.

Here are some examples from the website: New Light – John Mayer , See You Again – Wiz Khalifa

And you can visit the website here.