Framestore Uses Deep Fake Technology for Ground-breaking Anti-knife Campaign

Kiyan Prince, a football prodigy, tragically died after being stabbed at the age of 15 in May 2006 whilst trying to prevent another child from being bullied. Now, fifteen years later using their expertise in cutting-edge art and AI, Framestore has collaborated with marketing agency ENGINE Creative to support a ground-breaking effort to bring Kiyan Prince back to life virtually, and to ultimately help raise awareness of knife-crime.

The campaign called ‘Long Live the Prince’ will see Kiyan Prince sign for his former football club, Queens Park Rangers where he will wear the squad number ‘30’ to reflect the age he would have been today. In a world-first, Kiyan will also be introduced as a non-playable character in EA Sports FIFA21 – one of the world’s biggest video game franchises. Together with this, Kiyan will be a collectable footballer within the Match Attax card trading game which is being released by Topps on May 18th. Brands such as Adidas and JD Sports will be sponsoring Kiyan as a pro-footballer and proceeds will raise money for the Kiyan Prince Foundation.

Working pro bono Framestore’s co-founder and chief creative officer, Mike McGee; Project Lead, Karl Woolley; Creative Technologist, Johannes Saam; Global Head of Creatures, Markus Schmidt and Senior Designer, David Lochhead collaborated with ENGINE Creative and expert likeness artist Chris Scalf to lead the development of the assets needed for the campaign. To show the 30 year old likeness of Kiyan, the creative team had access to a leading expert on human ageing; Professor Hassan Ugail of the University of Bradford who used state-of-the art aging-projection software to create a scientifically accurate image of how Kiyan would look today. Once completed Framestore set about creating the suite of images to accompany the campaign.

“We've created many humans, digi doubles and superheroes in the past, but Kiyan was on a different level. Creating Kiyan as the 30 year old man he would have been today, was a sensitive and delicate process and we took our responsibility seriously,” said Karl Woolley. “Making sure we did Kiyan and his family justice by realising him as he would have been today, was the biggest challenge of this project and quite possibly, of all my projects to date.”

As the end sequence for the promo film wasn’t concepted until after the planning for Kiyan’s likeness had been completed, the creative team were left with no time to go down the traditional and more laborious digi double route. Alongside this there were no source images of Kiyan at 30 to train the Deep Fake on either.

To pull off the promo’s end sequence Framestore’s Creative Technologist, Johannes Saam partnered with Sydney based Matt Hermans, Founder of the creative company Electric Lens Co. to work on the promo video assets and began the challenging task of bringing life and movement to 30 year old Kiyan. Using a mixture of cutting-edge development and some secret sauce, the team were able to incorporate Scalf’s stills into the process to create synthetic data which they were able to train the Deep Fake on.

“Deep Fakes are often seen as gimmicks, to service a trend, but when we heard about the project and the potential to use Deep Fake for good, we knew we had to find a solution, employing new neural rendering based techniques.” said Johannes Saam. 

Based on a first pass of the geometry and painted reference frames, a virtual dataset was assembled with an automated pipeline rendering 6000 pictures as a base layer, which was then iteratively refined. 

“The first pass added facial animations to the training material. After that, the pipeline was fed a detailed lookdev pass with facial hair and additional texture detail to keep the machine in line with our 2D likeness,” said Saam. “Based on this dataset, we used a pre-trained neural network and the Kiyan specific images to focus the deep fake network on translating the human facial performance into a digital representation of Kiyan.” 

This network replaced the entire head frame by frame and was then composited over the photography, as well as removal of sponsor logos from the shirt using Nuke’s brand new CopyCat node.

All proceeds raised will go directly to Kiyan Prince Foundation, a charity run by Kiyan’s father, Dr Mark Prince OBE who’s been tirelessly campaigning to prevent knife-crime since his son’s tragic death. To find out more visit The Kiyan Prince Foundation.