Meta has announced the launch of its new model, “Movie Gen,” which aims to produce high-definition videos with sound. This announcement comes after several months of OpenAI revealing its similar model known as Sora.
The “Movie Gen” model relies on textual inputs to automatically generate new video clips, in addition to the ability to edit existing footage or still images. According to The New York Times, the sounds added to the videos are also generated using artificial intelligence, including noise, sound effects, and appropriate background music. The model can also create videos in different dimensions.
In addition to generating new video clips, Meta stated that the “Movie Gen” model can also create customized videos from images or modify existing videos by changing various elements. For example, the company showcased a still image of a man and demonstrated how the model created a video of him standing in a lab conducting a chemistry experiment, along with other diverse examples.
The “Movie Gen” model can also be used to edit existing footage, change styles and transitions, or add new elements that were not present before. In one example provided by Meta, the model successfully edited a simple video of a runner in various ways using artificial intelligence, such as changing the scene background or the runner’s appearance, using only text commands. The company presented other examples of editing processes performed by the model.
Chris Cox, Meta’s product director, explained in a post on the Threads platform that the company “is not yet ready to launch this tool as a commercial product soon,” noting that the high cost and the duration of video generation are major challenges.
Although OpenAI announced the Sora model earlier this year, the tool based on it has not yet been made available for public use. Recently, one of the officials responsible for developing the model resigned from OpenAI and moved to Google.
AI-generated image and video tools raise concerns about ownership and potential harmful uses. Reports have indicated that some startups in the field of artificial intelligence train their tools and models using thousands of videos extracted from YouTube, which is considered a violation of the platform’s terms of use, according to YouTube’s CEO, Neal Mohan.
Meta confirmed in a blog post that it trained “Movie Gen” “with a mix of licensed and publicly available datasets,” but did not specify the exact sources of those datasets.
