Inspiration Without Judgement: How Pinterest Is Using AI to Celebrate the Beautiful Diversity of the Human Body

The promise of the internet is that we each have a world of inspiration at our fingertips, but when it comes to fashion and beauty, it can still be frustratingly difficult to find anything that doesn’t conform to the same old rigid norms - white skin, straight hair and impossibly elfin proportions. Not that there’s anything wrong with looking like Legolas, but when algorithms serve up the same biased images it can be impossible for people to zero in on the looks, fits and products that are relevant to them.

That’s something that Pinterest takes seriously. Over the past five years, the social platform and visual search engine has been making a stand against AI that mindlessly propagates bias. Instead, Annie Ta and her team have been figuring out how to use AI to platform more diverse images and to help people find the content that really works for them. This week, they’ve just revealed a new body type technology that they say is a ‘first-of-its-kind AI technology that uses shape, size, and form to identify various body types in over five billion images on the platform to increase body representation online’.

According to Annie, this proactive focus on inclusion goes back to Pinterest’s ethos. “You’ve probably heard that at Pinterest our mission is to bring everyone the inspiration to create a life they love. And we believe that you can’t feel inspiration unless you feel included, and that inspiration begins with inclusion. So, my team’s mission, specifically, is really focused on that ‘everyone’ piece. Our hope is to help everyone feel like Pinterest is a place for them,” she says.

The seeds of the first AI-driven inclusion idea sprouted in 2017, when a pair of interns at the end of their placements presented a brilliant idea to leadership. They suggested that pinners - what Pinterest calls their users - would appreciate seeing more people who might look like them and that the platform should allow them to search for inspiration based on skin tone. 

“I think that one of the most incredible things about Pinterest is that we really believe that great ideas come from anyone,” says Annie.

In order to bring that to life, Pinterest’s engineers built a skin tone AI which uses computer vision technology to identify visible skin tones in an image. That allowed Pinterest to get a sense of how diverse the content on the platform was in terms of the range of skin tones, and they could then group them into different ranges. From there, it allows pinners to search for things that are relevant to them. As an example, ‘wedding makeup ideas’ is a popular search on Pinterest, and now users can see how different looks or shades might appear on people with a similar skin tone to them. 

That tool launched in 2018 - but Pinterest’s quest didn’t end there.

Next up was an AI that identified hair patterns and texture in images. This idea came from the pinners themselves. They had sent Tweets and written to the customer support team to let them know that one real challenge particularly facing Black and Brown pinners was that they were struggling to find hair inspiration specific to their hair types.

“It was actually a really natural extension of skin tone, because with skin tone we were able to identify where a face was in an image. Hair is literally the natural extension around the face, and so our AI technology has kind of just built on top of each other,” says Annie.

Core to the development of these features is a methodology called ‘co-design’, whereby the Pinterest team ensure that pinners, experts and advocates are included in the design process and share their lived experiences and needs.

“We really think that, in order to build for the most marginalised groups, we need to build and design alongside them,” says Annie. We went through this process of bringing folks in, and understanding their needs…. Then a few weeks later came back to these people and showed them what we built out of it, which was a really interesting experience to talk to people, in person, about what they need, distill that, and come back to them and see what their response was - but it was also really positive.”

This leads on to the newly launched body type technology, an idea which also arose from the frustrations of pinners. “We hear from pinners all of the time that when they’re searching for things on Pinterest, they aren’t seeing a lot of diversity when it comes to body sizes. And that’s not just true of Pinterest, but all over the internet,” says Annie. “When you are looking for fashion inspiration, people oftentimes see folks who are very skinny - and that’s not representative of the greater population around the world when it comes to size.”

And so Annie’s team developed this new computer vision technology that uses shape, size and form to identify different body types in the billions of images that exist on Pinterest. Then by pairing that with the AIs they’ve already created around skin tone and hair pattern, Pinterest is now able to shape its algorithm to increase the representation of body sizes, shapes and skin tones across searches and on what they call ‘related pins’ on fashion and wedding-related content. 

In order to make sure that this new tool was meeting people’s needs, the team engaged with the National Association to Advance Fat Acceptance (NAFA) as well as Megan d’Alessio, inclusion and diversity manager at Pinterest. As well as being a seasoned DEI expert, Megan has been a guest lecturer at Stanford University where she taught a course on fat liberation specifically.

As different people have different relationships with their bodies, Annie emphasises that this new body type tech has been designed to increase the diversity and representation of images that the platform presents to viewers, increasing the likelihood that people will see images that reflect them. What they’re not doing is prying or presuming. The team has taken a sensitive and nuanced approach.

“We talked to them [NAFA] really about how people are thinking about bodies now, so that we can do our best, again, to create this very safe, personal experience that feels like a place where people can find inspiration for themselves, without the judgement of what their size might be,” says Annie. “Our signals actually just identify things on content within Pinterest right now. So for example, it's not going to be looking at your particular body size on your profile picture or anything, we're identifying body types across different images across Pinterest. And we hope that that really just really meets needs where our pinners were not able to find inspiration so far.”

Annie argues that this focus on inclusion strengthens the product experience for everyone. “We’ve actually found from pinners that when they’re able to see more diverse inspiration, that it’s more inspiring for everyone, no matter who they are, whatever size they are or however they identified.”

And it’s also proven to be a hit with brands and ad agencies. “I think that we’ve had a really great response from business, for a few reasons. One is that we hope to inspire everyone in our industry, as well as advertisers and businesses, to really think about what representation means online. I think that’s a really positive experience. The other is we’re just trying to create this positive corner of the internet. When it comes to our business, we’ve heard from advertisers again and again - and you see this all over the media - that they don’t want to show their ads alongside negative content. But showing this more diverse range of content on Pinterest, we actually believe, is not only better for our pinners, but better for businesses,” says Annie.

One of the biggest challenges that Annie’s team faces is that, when it comes to inclusion, there are so many facets and threads of identity and marginalisation that it’s impossible to do everything right away.

“The folks that I work with are the most dedicated people, the most empathetic people that I’ve ever worked with, because all these people really hope and desire to build products for people who are historically marginalised, which is incredible. But the biggest challenge we have is that there are so many approaches that we can take to inclusive design,” says Annie, 

That’s why Pinterest has decided to focus specifically on visual representation. “When we first started this team and thought about our strategy, we thought about what makes Pinterest stand out, and where are our greatest strengths. That's why we actually doubled down around this idea of representation, because as a visual search engine, we think we have an opportunity and a responsibility to increase representation online. And not only that, we have expertise in house and computer vision AI specifically,” says Annie, who explains that this computer vision tech means that the team can measure fairness in the images surfaced by their recommendation systems or can evaluate where the content itself might be lacking.

Indeed, as well as looking at the way AI can help improve representation, Pinterest is consciously making sure to train its AI with diverse data sets. And it’s not just about skin tone, body type or hair texture, these data sets include gender non-conforming and non-binary people, as well as people with different visible disabilities.

With all of this focus on very specific marginalised groups, Annie says that Pinterest is perhaps paradoxically becoming more broadly welcoming. It’s also, she says, filling a huge gap in the internet experience.

“There's a hunger out there in the world, not just in the fashion industry, but the world to see more diverse people. And I think that's one of the greatest takeaways from these things is, we had questions early on when we are building more inclusive products, if we are building for the most marginalised people, are we building for a smaller group? And the answer to me is: no,” says Annie, who points to the fact that dips in pavements that allow wheelchair users to mount sidewalks benefit a far greater number of pedestrians, including parents with baby buggies.

All of this work hasn’t just had an impact on Pinterest’s product, it’s also had a profound effect on Annie herself. The participatory, collaborative experience of co-design is something she believes will stay with her.

“One of the things that I found really interesting, in talking to folks, is just there's so much to learn,” she says. Probably the best lesson learned from this, that I hope to apply in my career for the rest of my life, is that when you are talking to people from diverse backgrounds, you can learn so much from them about their lived experience, about the way they approach different problems. I think that is one of the biggest takeaways the team has. Talking to people and understanding and just listening enables you to be a better practitioner and whatever role you may have on the team, whether you're a product person like myself, whether you're an engineer or a researcher.”