Are the robots heading for edit suites – or will artificial intelligence hit the cutting room floor?
They can tweet pithy biblical verse, defeat Go champions and predict disease, but can AIs ever replace real craft? Adobe and Stanford University have their sights set on the craft of editing – in June a group of researchers from both organisations published a project that showed an artificial intelligence attempting to edit together a short scene. The programme organises the rushes and identifies different kinds of shots – and then cuts together a clip based on instructions or style templates from a human editor.
If you’re interested in the technology and process behind the experiment you can download the full paper here. The researchers concluded that this kind of technology will become increasingly prevalent in production, but we’ll let you decide if the final result really measures up to the human touch.
But what does this all mean for the craft of editing? LBB’s Laura Swinton caught up with experts from Whitehouse Post, Married to Giants and Envy to get their take.
Alaster Jordan, Partner/Editor at Whitehouse, NYC
“In the examples I saw, the human editor did an ok job structurally, a poor job performance-wise, and spent far too long cutting a 71 second clip. It looked like 15 minutes of work to me, not three hours. The AI edited scene was almost unwatchable. It missed the point in almost every way and omitted what little comedy or charm was in the source material entirely. The lack of touch, taste and humanity in the ‘idioms’ they plugged in resulted in a film with no touch, taste or humanity, just bits joined badly together with clipped words and no timing. But, 'you know what they say about enough monkeys on a typewriter…’ or as an AI editor would say “you say what they know about enough typewriters on a monkey…'”
Ryan Hunt, Editor at Married to Giants, Toronto
"From what I understand, the ‘AI Editor’ is more of an organiser than an editor. It assembles dialogue scenes based on idioms - "wide shot here, close up here” etc. The AI Editor doesn’t pick the best take, add the natural pauses in conversation, or find the unique moments that make dialogue spots believable and relatable.
"It’s pretty self-explanatory that editing is based on emotion. You need to scour the footage to find the subtle nuances, or change the pace by a fraction of a second based on what’s being said. And often times it’s actually the un-scripted moments that truly bring a piece to life. This is why it takes “three hours”. It’s also why you actually feel something after watching it.
"I think the AI Editor is great at assembling; not telling a story. Editing can’t be based on a formula or algorithm, even if it’s programmed to create different cuts that reflect the script. Realistically, when was the last time a script was followed?!"
Jai Cave, Head of Operations at ENVY, London
“For scripted work, this technology could certainly save time for sync-pulling and grouping. AI techniques for speech and facial recognition continue to gather pace and adoption. The first area this is likely to impact is transcription and logging, where currently teams are employed on larger projects to manually log action and speech.
“However, I am sceptical when it comes to automatically administering a style. Pre-programming a style into a set of rules by definition will lead to a lot of content being cut in a very similar way. I also don’t see an easy way of programming ‘flair’ or ‘humour’. Good drama, television and film looks unique not generic.”