Your Shot: How Brazilian Museum Goers Are Conversing with Paintings Thanks to IBM Watson

According to a study by the Brazilian Institute of Economic Research, 70% of Brazilians have never been to a museum. Ogilvy Brazil and IMB Watson are looking to reverse that statistic with a beguiling project that’s allowing museum goers speak to pieces of art. In collaboration with the Pinacoteca de Sao Paulo Museum, ‘The Voice of Art’ involves visitors receiving a smartphone and headphones with a mobile app that’s synced with IBM Watson’s artificial intelligence technology. At certain pieces of art they will be able to ask questions which the computer will answer. 

The idea is that any visitor, no matter which social or cultural background that they are from, will be able to interpret the art and have every question they have answered. It shies away from the stuffy, over complicated information that audio guides are often filled with, in favour of a broader knowledge base. What’s more, it’s living proof of everyday people getting to grips with artificial intelligence. 

LBB’s Addison Capper spoke with Ogilvy Brazil chief creative officer Claudio Lima to find out more.

LBB> What were the starting steps for this campaign? Did you always plan to utilise IBM Watson? What was the initial brief?

CL> Actually the brief was from IBM, to launch Watson in Brazil, but they wanted to find a way to show Watson interacting with a larger audience and to show it could be beneficial for people from any background. The other part of the brief was that they wanted to focus on education, so we decided to connect Watson with the world of Art because in Brazil museums and art institutions don't have the audience they deserve.

LBB> What inspired the idea for conversation between the museum goers and the paintings?

CL> Our main goal was to change something that had been established for a long time, and the way people had visited museums had been the same for decades. So, Watson could play an amazing transformational role there, making it very interactive.

LBB> During research you discovered that a lot of Brazilians enjoy art but rarely go to museums. What was their reasoning for this? Were there any links between the demographics of the people that gave similar answers?

CL> The data came from a big piece of research done by a government agency, but this is something you can also see in everyday museum visits – they are empty, unless there's a huge exhibition going on. The main reason for this is education, or a lack of education actually, which is a widespread problem in Brazil. People can go to the museum, but have no way to interpret or understand what they are seeing.

LBB> The process of asking questions about the paintings is very simple, but the technology behind it less so! Can you explain how it works please?

CL> We spent months feeding Watson with information about Brazilian art with all kinds of sources (old newspapers, art critiques, books), and making questions and helping him find answers, so he could learn the process. We'd gotten almost 20,000 questions answered when we opened to the public, so he knew his way through almost any subject. But he is a self learning system, so now, when he doesn’t know an answer at that moment, there will be a new learning process and the next day he will know how to answer it.

LBB> It’s interesting because I love going to museums but, personally, I hate wearing audio guides. They’re kind of invasive and detach you from who you’re travelling with. Did revitalising this approach play a big role in your work?  

CL> Yes. Our main goal throughout the process was to ‘reinvent’ something and we kind of killed audio guides the way they are. Because most of the times audio guides are giving you information that they don’t really know you want. A child doesn’t want to know the style of the painting, they want to know why it is smiling and so on – so every person has a guide specific for them.

LBB> What were the biggest challenges that you had to overcome when developing this project? I can imagine working with machines in this way is a whole new experience… 

CL> The biggest challenge was to feed him with the ‘non-art’ information. We had art curators and educators researching and feeding Watson – they knew all about the ‘art’ part. But most of the questions people asked are not related to art – they want to know if the character played soccer, was friends with other animals, crazy stuff, so we had to get him current news and info to answer those types of questions.

LBB> Do you hope / see this technology being rolled out to other museums around the world? 

CL> Yes we do. There are already six museums interested in this after the video came out.

LBB> This is a real example of IBM Watson being used by real people - what are your predictions for the technology in the future? 

CL> That's the goal for Watson – not to be something far away, used by specialists and tech people, but to be used by anyone, even children. And we hope that this shows the potential.