Adobe recently announced the integration of Firefly, its AI-powered generative model, into Premiere Pro, enabling users to generate videos from text and image prompts, and even extend existing clips. This new functionality is currently being rolled out to Premiere Pro subscribers.
Firefly’s Video Generation Capabilities in Premiere Pro
Several new tools within Premiere Pro and the Firefly web app showcase the video generation feature. One notable addition is Generative Extend in Premiere Pro. This tool allows users to add up to two seconds of AI-generated footage to the beginning or end of a clip. Furthermore, it enables mid-shot adjustments to camera positioning, tracking, and even the subjects within the shot.
An example of the Firefly video generator.
The generated video is output at either 720p or 1080p resolution at 24 frames per second (fps). Generative Extend can also extend a clip’s sound effects and ambient noise by up to 10 seconds, although it currently doesn’t support extending spoken dialogue or musical scores.
Firefly Web App Enhancements
The Firefly web app also receives two new AI-powered tools: Text-to-Video and Image-to-Video. Currently in limited public beta, these tools offer functionalities as their names suggest. Text-to-Video creates short video clips in various artistic styles, allowing creators to refine the output using the web app’s camera controls. Image-to-Video utilizes both a text prompt and reference images to generate videos that closely align with the creator’s vision, often requiring fewer iterations. Both web app features generate videos up to five seconds in length at 720p resolution and 24 fps, with a generation time of approximately 90 seconds. Interested users can join the waitlist to access these beta features.
Firefly’s Competitive Edge: Commercially Safe Content
While these video generation features are similar to those offered by competitors like Runway’s Gen-3, Meta’s Movie Gen, and OpenAI’s upcoming Sora, Firefly offers a distinct advantage: commercially safe outputs. Adobe trained Firefly on Adobe Stock images, openly licensed content, and public domain content. This training approach minimizes the risk of copyright infringement claims arising from generated content, a concern that has been raised with competitors like Runway, Meta, and Nvidia.
Conclusion
Adobe Firefly’s integration into Premiere Pro and its new web app features represent a significant step forward in AI-powered video generation. The focus on commercially safe content provides a valuable advantage for creators concerned about copyright issues. While the features are comparable to other AI video generators, Firefly’s commitment to legally sourced training data positions it as a strong contender in this rapidly evolving space. The accessibility of these tools directly within Premiere Pro streamlines the creative workflow for video editors, offering powerful new options for content creation.
