Colour Experts on How AI Is - or Isn’t - Transforming Grading

I’ve been using AI in my colour workflow since early 2023 - essentially as soon as Photoshop Generative AI Fill came out. As a colourist, I've always loved to push the boundaries of what you can do in session, and to blur the lines between colour and VFX, especially with fixes such as paint-outs. Before Generative AI, the process for painting out an object in a shot was much more tedious. The options were essentially blurring the background (not ideal), having VFX create a clean plate for us to composite with (slow workflow), or using a clone stamp type tool (clunky tool, and the repeated pattern could become obvious). These fixes were often still doable, but they were not as clean, and took significantly more time. I now have a streamlined process for paint-outs, where I export a single still from my shot, open it in photoshop where I use Generative Fill, then export it back to my system. Once I have the painted still, I use it as a clean plate and composite it back onto my shot, all of which takes only a couple minutes. 

Being able to take these small fixes off the plate of online artists helps everyone; in circumstances where there are tight budgets or tight deadlines, it saves time and money, but it also allows the online artists to solely focus on the more complicated visual effects on their plate. We also use multiple softwares to create AI-generated mattes. Getting mattes made used to be prohibitively time consuming for a project with a quick turnaround, but now we can create them relatively quickly without having the shots leave the colour suite. I don't feel that AI has completely replaced any part of my job, but I do think that it's streamlined things immensely. There are definitely limitations to these tools though - the matte generation is only successful on very clear images with smooth or minimal movement, and the prompting for generative AI can be difficult to master. There are a lot of nerves around AI in post production, and in the film industry as a whole, but at its current state I just see it as a tool to enhance the work you're already doing. Every artist on nearly every project will lament ‘if only I had more time…’, and AI integrations can give us some of that time.

I began using a form of AI when DaVinci Resolve incorporated it within their tool set, about five years ago. 

I only use AI tools within my work software, I use AI to help with certain aspects of the edit and conform. I also use it to help with colour grading, especially for rotoscoping, beauty work, face refinement, object removal and sky replacement. Most of the projects I grade now require at least the use of two of the AI tools.

From experience, professional colourists and post production companies have not really incorporated any external AI into their colour grading workflow except for the AI tools already in their grading software. The same would apply to myself, but I do regularly use these tools while grading.

Presently AI colour grading seems to be used predominantly by video content creators and creatives needing a colour grade on their content without the budget to go to a professional colourist. With a few exceptions, the tools professional colourists use on a day-to-day basis haven't seen a revolutionary AI tool that has disrupted current workflows. Moving forward, AI and computer hardware will become more advanced, and colourists will get more involved in the development of AI grading. This will eventually lead to AI being an integral part of the professional colour grading workflow, as either standalone software, plugins or integrated into current professional grading software, mainly taking over the time-consuming aspects of the workflow. This will obviously have a big impact on the industry going forward as colour assisting is where future colourists learn and hone their craft.  So if AI is doing all the assistants’ work, where are the next generation of colourists coming from?  

However, I do think that high end colour grading will still need a human touch because at that level, aside from the technical and creative qualities of a grade, colour has an emotional impact which needs human understanding when applying the colour, which I believe AI will still lack.

Truthfully, I have not found many consistent uses for AI tools in my work as of yet! I think interesting tools may start appearing in the next year or two, but right now, most of the commonly available tools are more broad-stroke image generation and less tailored to specific roles and niches within the industry. Software developers have been keen to advertise AI or machine learning capabilities in their latest updates as it is this year’s biggest tech buzzword, but so far I have found those tools do not have the reliability that is required for doing the job day in and day out. The interactivity and manual control of AI is its current bottleneck. In a job where things are constantly changing in such subtle ways, it’s imperative that all the tools we use give us the flexibility to manually tweak the results. 

The most interesting AI-driven tool I’ve found so far is Color.io. It is a browser-based colour editor built on an engine of analogue imagery that allows each of the colour operators to make rather clean adjustments with a colour response similar to that of analogue film. It doesn’t integrate perfectly with Resolve, but it exports fairly robust LUTs that can offer a good starting point with the right conditions.

I think what all the colourists I know are waiting for is a proper tool that could automate creating and tracking windows around complex subjects. We would all welcome AI tools that could expedite the most laborious and time-consuming tasks. We’ve all been there in session when a client asks us to make a detailed outline to brighten something in colour that just was not lit the way they wanted on the day. It’s almost always possible to remedy these situations in post to a degree, but it’s a matter of how much time the task will take when there are still a number of other scenes to get through.

Unexpectedly, I’ve actually received a handful of references generated by AI, which is fascinating to me because to get favorable results, the creatives involved have to feed the algorithms the language of colour – something people so often describe having a hard time with in session. In some ways, I can foresee prompt generation forcing creatives to become more articulate and specific with their vision which is generally a good thing! However, I find most people are continuing to seek out existing artistic works for their references; they crave the humanity in human-made work, and AI just doesn’t have that…yet. 

Generally, I don't use AI - mainly due to environmental and moral issues with it, such as water waste and CO2 emissions, or deep-faking and stealing art.

Where possible, I avoid using any AI-based tools within Resolve, but some tools are necessary within the colour workflow. For example, Scene-Detect Cut for prepping ProRes conforms makes the job of the colour assist easier as they don't have to manually cut up timelines.

On the other hand, in my personal grading workflow I don't use any AI-based tools (as far as I know!) - for windows and masking, I draw and track them myself as I feel I have far more control over the shape, rather than using Magic Mask. Additionally, if I'm unable to select something myself, I'd prefer to ask for a roto matte, which can be generated by more junior operators and help them learn and advance within their field. 

That's not to say I haven't worked on jobs where AI was used - a lot of up-and-coming directors use it to generate content when their budget is restrictive. For example, a recent promo I did had shots where the background was extended using generative-AI tools. However, I do encourage using real artists as much as possible, because there is always a difference when something is made by a human with feelings! It also means that junior artists can learn new techniques if, again, the budget can't stretch to more senior ops - it's the best way to learn!

In the future, I can't really see how AI can take over as a colourist - at least not to the standard of a human. A lot of what we do is down to nuance, taste, and emotion. You could teach a machine to 'balance' shots, but there won't be any artistic flair to it. This is also true of all roles in our industry; nothing can replace genuine passion for creativity.

There are so many AI tools out there right now. And while there are AI tools for colour grading that show promise, I wouldn't rely on them for professional work just yet. That said, AI is showing a lot of promise in enhancing plugins, creating LUTs, and coding DCTLs. I'm already seeing impressive plugins improving processes like noise reduction, adding film grain, and achieving realistic film emulation. AI particularly excels in handling vast amounts of data, which can greatly benefit projects with diverse mixed media. It works well with metadata and is efficient in creating a base colour match between mixed media in these scenarios. It’s a great base to start from.

However, AI has its limitations. No matter how sophisticated the prompt is, AI cannot replicate the depth of human interaction that occurs during a live grading session between the DP, director, and the colourist. AI just cannot grasp the story's subtleties or the emotional and motivational nuances behind creative decisions. 

About a year ago, a friend introduced me to DALL-E, and that's when I began experimenting with AI. I've only started using AI for colour correction within the past month.

I use AI tools nearly every day for various tasks. I rely on ChatGPT to help me edit long emails, use ComfyUI to create generative videos, and utilize Eleven Labs to clone my voice, which makes updating video tutorials for my courses on Weble.io much easier. For colour correction, I use ColourLab AI.

ColourLab AI is an essential first step in my colour grading workflow. I use it to establish initial colour grades and styles, based on client preferences. Typically, clients send me reference images that capture the visual tone they envision for their videos, and I use ColourLab AI to match those looks. It’s an excellent way to get a project off to a strong start.

The AI tools have streamlined communication with clients, allowing them to visually demonstrate their preferences instead of trying to articulate them. ColourLab AI also speeds up shot matching and helps achieve consistency across scenes.

Currently, ColourLab AI lacks a monitor feature, so I can’t send a broadcast signal directly to my calibrated monitor, which limits accuracy during grading. Additionally, some features available in the standalone ColourLab AI app aren’t in the DaVinci Resolve plugin, which can be inconvenient. Still, this might be intentional rather than a bug.

I recently worked on a documentary for a prominent company launching a new AI product. After doing my usual colour and beauty corrections on interview subjects, I was asked to dial it back to avoid the impression that AI or artificial enhancements were used to ‘beautify’ the speakers. This highlighted how sensitive the public perception around AI has become.

AI colour grading tools are likely to significantly impact the industry, especially for mid to low-budget projects. While top-tier professionals will continue to use AI as an auxiliary tool, many current colourists may face challenges as editors, assistants, and even casual users adopt AI for colour correction in ways similar to using Instagram filters. I believe more people will start doing their own colour grading once the tools become easier to use. My advice to colourists is to learn AI tools aggressively, as it’s not AI that’s coming for your job - it’s someone using AI who is!

Like most people my first direct experience/interaction with AI will have been through chatbots and image creation AI tools like mid journey, it wasn’t until a little while later that features started popping up in my grading tools. 

As a colourist I’m using tools within my grading system. The AI masking tools in DaVinci Resolve are actually now very precise and also incredibly quick. For example, in the time it would have taken to merely draw a rough mask around a car in a commercial, I could have perfectly tracked the mask of the vehicle and already be adjusting the colour. I mention cars because they’re often the most difficult object to key and track due to reflections and shape and scale. 

I’d say for me that the biggest benefit to AI in almost every form is speed. For example being able to mask or get a depth map in seconds is only going to help especially in a room full of clients. I think it actually improves the creativity in my colour work because I’m more focussed on what the image actually looks like than the key-framing! 

Like most new product features it’s not perfect and I have had times where I’ve given up and gone back to using manual techniques lots of times. 

Our focus continues to be identifying the most impactful ways to connect creative craftspeople to content, meaning we aren't looking to replace those individuals; however, we are exploring the best ways to harness AI technologies to offset technical parts of the process, like prep or conform This strategy has actually allowed us to shift MORE of the available budgets towards our experienced creative professionals, as well as set our AEs up to be that much more nimble and efficient throughout our workflow.

The biggest limitation we've faced is ensuring that any technology is based on creative integrity-- if models are learning from certain data sets, what do the licensing agreements look like, for example. Ultimately, the technology itself is outpacing the legality, meaning often we forego harnessing a platform rather than expose a client to a tool that is anything but above reproach

In the immediate future, AI provides the largest potential within the pro-sumer and influencer tier of content creation-- individuals who are managing edits start to finish and need a relatively reliable means of elevating the production quality. As a production company and post house, this was typically work we weren't touching anyhow. Still, rather than find ourselves behind the curve in the coming months, we continue to research, evaluate and beta-test all the emerging platforms so, when the time comes, we already have a high-level of familiarity and mastery of the tools.

At Sweetshop, our journey into the world of AI began a little over a year ago. Like many, we started with tools like ChatGPT and Midjourney, curious but cautious. Then, everything changed last October when moving video AI made its debut - a moment that felt like diving headfirst into an ocean of new possibilities.

Since then, we’ve fully embraced AI, weaving it into the fabric of our creative process. Our sister company, The Gardening Club, showcases a roster of the world’s most visionary AI filmmakers, and alongside them is The Flower Factory, our hub for cutting-edge AI tech. Together, they amplify Sweetshop's capabilities, enhancing everything from commercials to long form projects in ways that were once unthinkable.

In our live-action commercials and films, traditional colour grading remains a cornerstone - requiring an artist’s eye, refined vision, and the subtle control that only hands-on work can provide. But in the world of AI filmmaking, grading merges directly with the creative prompt itself.  Whether it’s a vintage Kodachrome vibe, a pastel dream palette, or a moody noir tone, the aesthetic can be set at the click of a prompt. This capability gives unprecedented flexibility, letting artists achieve a unified visual language from the start without extensive post-production work.

Though professional colour grading may shift as AI filmmaking continues to evolve, it remains a vital piece of our creative toolkit - for now, and hopefully for years to come.

I started using AI ever since Resolve introduced AI tools such as Magic Mask, Face Refinement and Scene Cut detection. I use it when conforming, isolating subjects from the background and adding beauty. I use it most of the time!

It improved my workflow by making it more efficient. For example, scene cut detection is useful when receiving ProRes pre-conformed file, as it detects each shot change and makes a cut point in between. This saves me time from creating cuts, but I still need to check each of the cut points making sure it's correct. Most of the time, it's pretty accurate. Also, the magic mask is such a powerful tool when isolating subjects/objects from the background, it's faster than drawing your masks. 

However, the downside of it is that it's not as precise if the subject moves a lot or sometimes when changing frame size, it picks up different parts of the shot which flickers. When this happens, you spend more time trying to remove the flickers rather than drawing the magic masks. The face cut detection is also useful for beauty because the AI detects the face rather than keying or masking it.

I think AI has pros and cons. I think there is a fear from creatives whether it is a threat or an enhancement tool for our workflow. In my opinion, there is still far to go for AI to replace colourists and other creatives.  There are programs that you can download online such as Colourlab.AI or Color.io that offers basic colour grading such as copying the grade overall, exposure and balance adjustments. However, from doing a trial in one of these, I found that it doesn't do detailed colour grading. For example, drawing masks, keying, grade keyframes and you still need to manually check each grade to tweak with the exposure, contrast and saturation to match the overall look. 

Also, you may spend more time checking all these tools have worked correctly, rather than simply doing the grade manually from the beginning. AI plugins and tools can also cost extra money to download and integrate. 

Omar began exploring AI tools about a year ago when DaVinci Resolve introduced AI features. While he has yet to employ AI for actual colour grading work, he has integrated it into his day-to-day workflow, admitting it has absolutely enhanced efficiency with technical tasks.

One of the key benefits has been how AI assists with non-creative aspects of his work, freeing up more time for artistic decisions. Inguanzo specifically highlights Resolve’s MagicMask as a useful tool, saying "To just paint a single stroke…on somebody's suit, head, shoe, you name it, and quickly make a mask is pretty incredible." 

However, he acknowledges its limitations, noting that "unfortunately, it doesn't always exactly know what you want and isn’t reliably precise, so it's prone to make a lot of mistakes." When discussing AI's current shortcomings, Inguanzo is candid: "For being Intelligence, it can be stupid. It's not super intuitive and not 100% there yet."

The senior colourist emphasises that while AI is not fundamentally changing outcomes of any of his projects, it is streamlining workflow and “making my job easier and more efficient” in practical application.

Looking to the future, Inguanzo expresses concern about potential complacency in this branch of the industry: "The fear is that people will settle for whatever end result they get from it, and get complacent. I’d hate to lose the artistry of colour grading." He maintains that despite technological advances, the human eye and artistic judgment will remain essential in colour grading moving forward.