Adobe revealed its intention to integrate AI video synthesis tools into Premiere Pro and After Effects on Monday. This announcement follows last week’s revelation that Premiere Pro will feature AI-powered text-based video editing in May.
Certain special effects or editing tasks can currently be quite labor-intensive. However, Adobe hopes to streamline the process by leveraging generative AI assistance.
In a demo reel showcased on its blog, Adobe demonstrated the capabilities of “Firefly for Video,” which included generating a background soundtrack, generating sound effects, changing the visual season in a video, creating a visual effect on the word “molten,” locating and brightening a person’s face and generating audio transcripts via text commands.
Adobe’s promotional blog mentions several features that users can expect from its new AI tools, including the ability to generate sound effects and music. Additionally, these tools will offer text-based color enhancements, text and logo animations, automatic storyboard creation, and “co-pilots” that can assist users in mastering editing skills.
It’s worth noting that these text-based video editing tools may seem familiar, as Runway teased similar capabilities back in September. However, Runway has yet to deliver on those promises to the degree showcased in its teaser. As such, it’s probably wise to approach Adobe’s marketing video with a degree of skepticism until the product is officially released.
Lastly, only time will tell if Adobe Firefly can ever produce a video of Will Smith eating spaghetti that can rival the masterpiece created by ModelScope, an open-source video synthesis model.
In a related announcement last week, Adobe shared that it incorporates text-based video editing into Adobe Premiere Pro. This feature will enable editors to trim and rearrange video using automatically detected speech transcripts extracted from video clips. Adobe explains the feature as follows:
Premiere Pro is the only professional editing software to incorporate Text-Based Editing, revolutionizing the way creators approach their craft by making video editing as simple as copying and pasting text. Powered by Adobe Sensei, Text-Based-Editing automatically analyzes and transcribes clips so editors, producers and assistants can simply copy and paste sentences into any order they want and instantly see them appear on the timeline. Transcripts can be searched in the transcript window to identify exact words and phrases.
Back in October, Adobe previewed a comparable text-based video editing capability as part of “Project Blink.” However, Adobe now claims these features are part of its “Adobe Sensei” framework. It’s unclear whether this difference is technical or simply a branding shift. It’s worth mentioning that Descript, a company, provides a similar feature for video and podcast editing.
Adobe plans to begin rolling out Firefly for Video features “later this year.” Currently, Firefly is in closed beta testing, but individuals can register to try it out. In the meantime, Adobe has promised that text-based video editing will be available in Adobe Premiere Pro in May.
Related Stories: