Meta has introduced Movie Gen, a new suite of AI models designed to transform video creation on Instagram. 

Set for release in 2025, Movie Gen aims to empower users with advanced video and audio generation capabilities.

The system comprises four main components: a 30-billion-parameter video generation model, a 13-billion-parameter audio generation model, a personalized video model, and an editing model. 

These tools allow users to create high-definition videos up to 16 seconds long at 1080p resolution, along with synchronized 48kHz audio.

AI Training

Meta’s research team trained the video model on a dataset of 100 million videos and 1 billion images, enabling it to understand complex visual concepts such as motion, interactions, and camera dynamics. 

The audio model, trained on over a million hours of audio, generates sound effects and background music that align with visual elements.

The company employs a novel technique called Flow Matching in the training process, which Meta claims offers advantages over traditional diffusion models. Flow Matching provides robustness against noise schedules and improved efficiency in both training and inference.

How Movie Gen Works

Meta positions Movie Gen as a tool for casual Instagram users and professional content creators. The system allows for text-prompted video generation, personalized content creation, and precise video editing. 

Users can modify specific elements within a video, such as objects or colors, and make global changes like background swaps using text instructions.

Meta CEO Mark Zuckerberg demonstrated Movie Gen’s capabilities in a video posted on his personal Instagram account. The clip shows Zuckerberg using a leg press machine that transforms into various themed versions, including cyberpunk and Ancient Roman styles.

According to Meta’s technical paper, Movie Gen outperforms competing models from companies like Runway, Luma, OpenAI, and Kling on several audience-rated attributes, including consistency and natural motion.

Movie Gen’s release follows earlier Meta projects in generative AI, such as the Make-A-Scene and Llama Image models.

As Movie Gen enters its research phase, Meta plans to collaborate with creators and filmmakers to refine the system’s features. 

“We’re sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them,” Meta wrote in a blog post.

The company acknowledges that while Movie Gen offers significant potential, it is intended to enhance rather than replace the work of professional artists and animators.

Concerns Due to AI

Meta has not specified whether the training data for Movie Gen was licensed, in the public domain, or scraped from the internet. 

This lack of clarity may lead to criticism from artists and video creators, similar to concerns about other AI model providers.

A recent investigation revealed that major tech companies have used content from thousands of YouTube videos to train AI models without the creators’ knowledge or consent, resulting in lawsuits against OpenAI for alleged unauthorized use of video content.

The Creators’ Rights Alliance (CRA), a coalition representing over 500,000 professional creators in the UK’s creative industries, recently issued a letter to AI developers addressing the unauthorized use of copyrighted works in AI model training.

In the letter, the CRA explicitly stated that its members do not authorize the use of their copyrighted works for AI training, development, or operation without specific licensing arrangements.