CODEX DRIVES COLOR PIPELINE INNOVATION
Most cinematic superheroes of today were born in the 1950s low-tech medium of comics. Today, the astonishing on-screen derring-do of these spandex-clad characters is made possible by the highest of high technology. And behind the scenes, the success of Marvel and others is driving change at an unprecedented rate.
Spider-Man: Far From Home is a perfect example. Codex, working closely with DIT extraordinaire Francesco Giardiello, developed and implemented a change in color-pipeline methodology on that project, and while it may seem like a subtle tweak to the process, the adjustment could save thousands of hours of work and many dollars going forward.
Essentially, Codex and Giardiello developed a much more efficient way to match footage across scenes and takes destined for the visual effects pipeline. This new method takes advantage of existing tools developed over the years by Codex.
In the standard procedure, the director of photography lights and shoots the scenes and the rushes go to the lab, where a dailies colorist makes a one-light grade, usually using ASC-CDL values that are baked into the images used for editorial. When on-set live grading tools are used, the DIT provides the ideal look to post-production, again mostly using ASC-CDLs. Both the DIT and the dailies colorist end up using the same tool for both purposes: to create a CDL designed to be the cinematographer’s artistic intent for the picture, and as a sort of matching mechanism used to balance shots.
Problems arise when the footage goes to VFX, where the dailies CDL grade is removed from the equation, and artists work on the raw files to maximize image quality and flexibility for the compositing process and CG integration. Because that CDL grade contains both the look along with the matching grade, removing it sends the footage back to its original state, which therefore won’t necessarily match. Perhaps the scene cuts between two shots that were done with different lenses or are otherwise inconsistent and will now require a “technical grade,” which will be performed by VFX prior to starting their work. This can cause delays and could generate new inconsistencies. It’s also inefficient and duplicates work.
“If you design your CG element to work with the color and texture properties of a master shot from a sequence, it’s not necessarily going to match the entire scene,” says Giardiello. “That forces VFX to pre-grade everything and the DI colorist to create masks and possibly add layers upon layers to balance the live action with the CG elements across all of the footage.”
Work expands exponentially. Creating a “double CDL” workflow was partly an early attempt to address this issue, but linking footage to a separate CDL file circumvented the problem without actually solving it, says Giardiello, and often came with its own issues. “The real solution was to find a way to provide well-balanced footage to visual effects,” he says.
After six years of testing and iterative improvement using Codex Production Suite and several software and hardware tools developed by Codex for the purpose, a solution was achieved. The new method takes advantage of the RAW capabilities of the cameras used on complex, effects-intensive projects like Spider-Man: Far From Home. Marvel, known for strict guidelines regarding workflow, required extensive testing before agreeing to the new procedure.
“These cameras offer the ability to modify the way their sensors see the scene,” Giardiello explains. “Using 3x3 matrices and transform equations, you can reach a sweet spot. That doesn’t necessarily work on the set, because it can take too much time. But by getting into the RAW footage and modifying the RAW metadata, we can interpret the footage so that when it comes out from the de-Bayer, it has balance already baked in. It’s much simpler than complex systems like two-layer CDLs, where things could go downhill once you leave the software framework or send it to a different vendor.”
The method uses Codex Production Suite in conjunction with Codex’s Review Module to adjust color temperature, tint and exposure index. Afterwards, a normal CDL can be applied.
“People think of those controls as a representation of an analog world, but we saw those as simply numbers that can be changed to adjust the footage,” says Giardiello. “When VFX gets the footage as an OpenEXR file, it’s already well balanced, and incorporates a CDL as a single look that’s applied to everything in that scene or that environment.”
In a way, the method enables a return to first principles for cinematographers, who can think as they did in the days of film emulsion, changing colors using lighting and gels on the set while depending on a consistent reaction in the camera – like that of a film stock. On Spider-Man: Far From Home, Giardiello and director of photography Matthew Lloyd, CSC designed the entire film’s color using just 48 CDLs – a number that often reaches thousands on complex projects. On the shoot, data managers Will Gardner and Andrea Michelon, working in a purpose-built truck, started balancing the footage almost immediately, guided by a master shot identified early in the work day. Later, Giardiello reviewed the work.
“The visual effects vendors are telling me that it’s going to save them hundreds of hours,” he says. “That’s essentially what it is – a time saver. It’s nothing you can’t achieve in postproduction, given enough time. It’s like having a dailies colorist on the set, but way more accurate and efficient, and you don’t have to time the rushes in three hours overnight. Instead you have a whole day to do so and the eyes of three people to judge the footage. Even the trailer’s color is remarkably close to our original on-set color correction.”
“This system is genius,” says Lloyd. “Francesco and Codex have cracked the code. I loved how the images looked, additional off-set dailies color correction was not needed. Francesco’s CDL work was right on the money, and had a lot to do with how smoothly it went. If there was a tricky shot, we could always go back to the ‘neg,’ put the LUT on, and look at the CDL from the set. You can see how the image ended up where it is. As far as the on-set work making its way into the DI, this was definitely one of the cleaner experiences I’ve had. It puts control back in our court. You don’t have to throw away all that work and start building the look from scratch in the DI. I really appreciate that there are people thinking about what the intention is, and how to preserve it from capture to finish.”
Camera Type: ALEXA Mini
Camera Rental by: ARRI Rental UK
Final Color/DI: Jill Bogdanowicz – Supervising Finishing Artist at CO3
Director: Jon Watts
Director of Photography: Matthew Lloyd, CSC
DIT: Francesco Giardiello
Codex related product and workflows
Images courtesy of their respective owners.
