How AI Is Already Democratizing Production and Post

Watch the full NAB Show 2023 session, “Generative AI: Coming Now to a Workflow Near You?”

Generative AI is changing not only the economics and logistics of film production, but also the entire creative process as a whole.

In an insightful discussion held at NAB Show with Pinar Seyhan Demirdag, the co-founder of Seyhan Lee, and Yves Bergquist, director of the AI and blockchain and media project at the Entertainment Technology Center at USC (the think tank funded by all of the major Hollywood studios and tech companies like Google, Microsoft and Amazon) consider the implications. Watch the full session, “Generative AI: Coming Now to a Workflow Near You?” in the video at the top of the page.

Demirdag kicked off the conversation by talking about Cuebric, the AI filmmaking tool developed by Seyhan Lee. “It is to my knowledge, the first virtual production tool that runs on generative AI,” she explains. “It’s a tool that enables filmmakers to go from concept to camera in minutes. Other services that our company offers involve include a generative AI VFX for films and advertising films.”

In developing Cuebric, Dermirdag explained that Seyhan Lee spotted several gaps in the current film and TV workflow that could use an AI boost:

“For example, on a film set, it feels always like everybody’s hurrying up… and then there’s always this constant waiting, everybody waits. Then, in the VFX process, there’s a bunch of repetitive tasks like rotoscoping. I don’t know how many of you rotoscoped in your life but if ever I offer you a solution to push a button, none of you will regret not rotoscoping.

“The third benefit of AI is a barrier with virtual production workflow — it’s a completely new technology and directors of photography especially are terrified of it. 3D real-time engines require an expert to operate them and it’s quite costly.”

There’s another problem that generative AI could help solve, too: the cost of reshoots. Seyhan Lee’s research found that reshoots, even on “healthy movies,” can cost anywhere from 5% to 20% of its budget. “For a medium-size movie, it costs the production $375,000. And for a big-budget movie, it costs up to $24 million for reshoots alone,” Demirdag added.

“How about we make a tool where we can save the production even a fraction of that and bring the mental health of the producers back?”

Bergquist explained that AI is making immediate inroads into enabling faster and cheaper pre-production. The use of generative AI for storyboarding is among the most accessible use cases for filmmakers.

“Down the road a lot of visual production is going to be extremely disrupted [by AI]. The Adobes of this world are already integrating AI into their software,” he said.

Bergquist predicted that many VFX companies will integrate generative AI into their pipelines. Not only would this make content faster, and cheaper, he said, “but it opens up a lot of opportunities for new creatives, riskier content and riskier content formats to be produced.”

READ IT ON AMPLIFY: Traditional Media Companies Aren’t “Computable,” and That’s Where AI Actually Poses a Threat

Forecasting what might lie beyond automating workflows, he commented that AI would be liberating as well as disruptive for all aspects of moving image storytelling.

“The macro trend is giving individuals extremely powerful tools that used to be reserved for institutions. Information used to be in the monopoly of institutions, now in the hands of everybody with the internet. This is really putting the tools of large-scale high value production into the hands of everybody,” he said.

“We’re heading into a world where everyone including TikTok influencers and Instagram influencers and all sort micro content creators are going to have the tools to make a full anime series, for example. This is not something the entertainment industry is ready for.”

As a neat example, Dermirdag pointed out that that bullet time sequence developed painstakingly from research into photogrammetry by Paul Debevec in 1998, and employed in The Matrix a few years later, “was a megalithic invention” at the time but that now “you can basically have a cheap iPhone, and you get a similar effect as The Matrix.”

Bergquist continued the theme, theorizing that taking the tools of production out of the hands of large, expensive organizations and putting them in the hands of everybody means “we’re at the cusp of just an explosion of creativity. And that’s really, really exciting.”

However, both were in agreement that just having generative AI doesn’t mean it will produce a great story. “If you look at the history of independent cinema — most of independent cinema is just garbage,” Bergquist said.

“There are very, very few independent filmmakers who are very, very talented. So is AI going to take creativity to a completely new level of quality in general, in terms of how deep we reach into the human condition and tell stories that never been told before? Or is it just going to be painting over a lack of talent?”

Dermirdag dubbed this “the danger of normalization of mediocrity.”

The pair then held an interesting discussion about the merits and possibility of AI as a creative force in its own right. Dermirdag argued that much of the fear about using generative AI in the creative arts is because we don’t call it what it is: a tool.

“This tool is great, this tool doesn’t work. This tool will help me, this tool will ameliorate my workflow, this tool is going to help me make more money. This tool is complicated too to understand, but I’m going to read some books and understand it,” she said.

“Unfortunately, our collective subconsciousness is tainted by [negative] stories. 2001: A Space Odyssey Robocop, Terminator, Blade Runner, like we’re all entering forcefully, very fast into a zone that does not serve the elation of humankind. But it is our responsibility as every single human being to research into what AI does do. It’s actually quite simple. There’s a data set and then there’s an algorithm and it produces results in order to serve your creativity.”

Bergquist made the distinction between the craft of production and the decision making that leads to the craft. While AI could vastly improve the efficiency of production, it would allow for humans to make decisions about what to make for audiences.

“As all the crafting part of creativity becomes a lot more automated, a lot faster, a lot more optimized, my question is this. Does the quality of creativity lie in knowing about the craft, or does it just lie in the kind of creative decisions that you’re making?” Bergquist asked.

“Are there elements of knowing about the craft of being trained in the craft versus the decisions that are material to good creativity? How much of quality of how much of knowing about the craft of what aspects of Visual Arts just disappears. If production becomes just the push of a button what impact would this have on art or does telling the algorithm what to make pack all the creative decision making up front? Does knowing about the craft make you a better decision maker?”

These are great questions which will horrify editors and cinematographers and production designers.

Many, including cinematographers and directors, enjoy the creative discussion up front with a script before principal photography as the most inspiring part of the process. It’s where many believe the show is designed and created (editors would disagree), but surely all great filmed art is the outcome of a multiplicity of talented people and of the happenstance of coming together with technology to fulfill a vision.

Demirdag is in no doubt: “Generative AI has nothing to do with creativity. It has everything to do with being your parallel processing, never tiring, just your assistant constantly giving you options for you to curate, review, and select.”

You could probably program an AI to factor in artifacts and deviations from an original idea but would this produce, say, Flowers of the Killer Moon?

It may not be long until we find out.

VISIT OUR DEDICATED 2023 NAB SHOW MICROSITE.