Deep Fakes Were Fantastic for “Free Guy”

BY JULIAN MITCHELL

Ever since SkyNet became self-aware in 1997, the chilling result of computer vision in the film Terminator 2 – Judgement Day, we have held artificial intelligence at arm’s length. We love the face-swapping deep fake videos on YouTube but distrust AI’s inevitability and fear for the possibility of a real SkyNet. Now, this modern-day facial parlor trick has gone mainstream.

Ryan Reynolds’ new film Free Guy marks one of the first feature film uses of Digital Domain’s proprietary face-swapping tool, Charlatan. The film brilliantly mixes CG and live action as part of its “real life inside a video game” narrative. One of the main players of the game is called BadAss.

Late in the film, BadAss gives a speech with far-reaching implications. It’s vital to the plot and tone, and after seeing the finished results, the filmmakers decided that they needed to change the dialogue and add to it for the scene to have the necessary impact. Rather than schedule a new series of reshoots and force the actor back to read a handful of lines, various methods of animating the digital character were employed, with none producing a realistic and acceptable result. So Digital Domain decided to try Charlatan.

Artists created a new facial model of BadAss by hand, then used Charlatan to combine it with the original performance — Charlatan takes existing footage and analyzes the movements down to the minutia. Once the AI processing was able to link the two and replace the original animation, the results were a more realistic digital avatar that could then mimic the actor’s facial mannerisms and movements to mimic reading the new lines. The actor then later recorded the new dialogue in ADR.

Ian Failes from befores & afters digs a little deeper into the process: “The moment revolved around the game character version of BadAss, played by Channing Tatum. The actor performed that role, with Digital Domain making a ‘game-play’ digital double of the character. A speech made by BadAss was originally delivered, but then the filmmakers realized the scene needed changes to the dialogue.

“Digital Domain was tasked with initially trying to re-animate the mouth and face performance of the game-play digi-double. But something wasn’t quite right, so the VFX studio turned to a slightly unconventional use of their deep learning renderer called Charlatan (something they also refer to as a face-swapping tool) to handle the required face-swapping for BadAss here, which was essentially re-animated lines.”

READ MORE: VFX Futures: How a deep learning renderer was used to re-animate a character’s lines in ‘Free Guy’ (befores & afters)

Other recent uses of Charlatan included a Super Bowl digital return of NFL legendary coach Vince Lombardi. After receiving live-action plates of an actor cast for his physical similarities to the coach, Digital Domain turned to the difficult task of recreating a legendary face that not only looked right but would articulate believable emotions. Working with decades-old 2D images and video reference material while working remotely under a five-week deadline, Digital Domain delivered a high-definition version of Lombardi in his prime. Adding every detail possible, from wrinkles to eyebrow hair, the team of artists then utilized Charlatan to complete the effect.

READ MORE: NFL SUPER BOWL LV COMMERCIAL “AS ONE” (Digital Domain)

It’s reasonable to assume ccomputer vision techniques like Charlatan combined with real-time computer graphics will become ubiquitous within post production. Maybe the magic this particular technique creates will help with our general circumspection of AI while the specter of the uncanny valley faces becomes a thing of the past.

Want more? Halon Entertainment director of visualization Ryan McCoy and visualization supervisor Grant Olin dive into the visualization process for Free Guy and how Unreal Engine played an integral part in making the film’s blockbuster metaverse a cinematic reality: