Skip to content
  • Stories
    • Create
    • Connect
    • Capitalize
    • Intelligent Content
  • Events
  • Sign Up
  • Sign In
September 1, 2022
Posted September 1, 2022

Watch This: “The Crow” Beautifully Employs Text-to-Video Generation

author
Adrian Pennington
Computer artist Glenn Marshall’s AI-driven short film “The Crow” paves the way for a future where entire feature films are produced by text-to-video systems. Cr: Glenn Marshall
Computer artist Glenn Marshall’s AI-driven short film “The Crow” paves the way for a future where entire feature films are produced by text-to-video systems. Cr: Glenn Marshall

READ MORE: How an award-winning AI film was brought to life by text-to-video generation (The Next Web)

Sooner or later an AI, or several of them, is going to make an entire narrative film from script to screen. A step closer to that inevitable day has been provided by computer artist Glenn Marshall.

Marshall’s works are entirely created through programming and code art. In 2008 he won the prestigious Prix Ars Electronica for a music video he created for Peter Gabriel — unique in that it was created entirely out of programming and algorithms. He also created an AI-generated Daft Punk video.

READ MORE: This AI-generated Daft Punk video is the perfect tribute to the electro pioneers (The Next Web)

The Crow is a finalist for The Lumen Prize, considered to be two of the most prestigious digital arts awards in the world, and is also eligible for submission to the BAFTA Awards.

Computer artist Glenn Marshall’s AI-driven short film “The Crow” paves the way for a future where entire feature films are produced by text-to-video systems. Cr: Glenn Marshall
Computer artist Glenn Marshall’s AI-driven short film “The Crow”

“I had been heavily getting into the idea of AI style transfer using video footage as a source,” Marshall told The Next Web. “So every day I would be looking for something on YouTube or stock video sites, and trying to make an interesting video by abstracting it or transforming it into something different using my techniques. 

“It was during this time I discovered Painted on YouTube — a short live-action dance film — which would become the basis of The Crow.”

  • Cr: Glenn Marshall
    Cr: Glenn Marshall
  • Cr: Glenn Marshall
    Cr: Glenn Marshall
  • Cr: Glenn Marshall
    Cr: Glenn Marshall


ALSO ON NAB AMPLIFY:

One of cinema’s most chilling villains is the artificial intelligence HAL 9000 from “2001: A Space Odyssey,” 1968. Cr: MGM

When Your Kubrick AI Isn’t HAL… It’s an AI Kubrick




EXPLORING ARTIFICIAL INTELLIGENCE:

With nearly half of all media and media tech companies incorporating Artificial Intelligence into their operations or product lines, AI and machine learning tools are rapidly transforming content creation, delivery and consumption. Find out what you need to know with these essential insights curated from the NAB Amplify archives:
  • This Will Be Your 2032: Quantum Sensors, AI With Feeling, and Life Beyond Glass
  • Learn How Data, AI and Automation Will Shape Your Future
  • Where Are We With AI and ML in M&E?
  • How Creativity and Data Are a Match Made in Hollywood/Heaven
  • How to Process the Difference Between AI and Machine Learning

Marshall fed the video frames of Painted to CLIP, a neural network created by OpenAI.

He then prompted the system to generate a video of “a painting of a crow in a desolate landscape.”

Marshall says the outputs required little cherry-picking. He attributes this to the similarity between the prompt and underlying video, which depicts a dancer in a black shawl mimicking the movements of a crow.

Although 'video styling' is nothing new in the AI world, it's beginning to creep into the mainstream. I think 'The Crow' is playing its part. I'm getting accelerated interest from high profile bloggers, journalists and TV stations,citing it as a glimpse into the future of cinema. pic.twitter.com/gYwJkJDCyC

— Glenn Marshall (@GlennIsZen) August 30, 2022

“It’s this that makes the film work so well, as the AI is trying to make every live action frame look like a painting with a crow in it. I’m meeting it half way, and the film becomes kind of a battle between the human and the AI — with all the suggestive symbolism.”

  • Cr: Glenn Marshall
    Cr: Glenn Marshall
  • Cr: Glenn Marshall
    Cr: Glenn Marshall
  • Cr: Glenn Marshall
    Cr: Glenn Marshall

Marshall says he’s exploring CLIP-guided video generation, which can add detailed text-based directions, such as specific camera movements.

That could lead to entire feature films produced by text-to-video systems. Yet Marshall believes even his current techniques could attract mainstream recognition.

Deep learning is not coming to Hollywood. It is already here.

See for yourself in the video below:

Are you interested in contributing ideas, suggestions or opinions? We’d love to hear from you. Email us here.

  • Content Creation
  • Intelligent Content
  • Acquisition and Production
  • Management and Systems
  • Media Content
  • Post Production
  • Al / Machine Learning
  • Short / Form Programming

Subscribe

for more content like this sent directly to your inbox:

Sign Up
  • Main Pages
  • Homepage
  • Stories
  • Events
  • Companies
  • Products
  • Policy
  • Privacy Policy
  • Terms of Use
  • Code of Conduct
  • Cookie Policy
  • Quick Links
  • Advertising & Thought Leadership
  • NAB Amplify Press
  • FAQs
  • Technical Difficulties
  • Contact
  • Cookie Preferences
  • RSS Feed
The Angle Newsletter

Weekly editorial newsletter covering the latest content, events and more taking place on NAB Amplify.

Subscribe

© 2023 National Association of Broadcasters. All Rights Reserved.