Adobe is giving filmmakers new generative AI audio tools that can quickly add thematically appropriate backing tracks and narration to videos. Generate Soundtrack and Generate Speech are being introduced to a redesigned Adobe Firefly AI app, while Adobe is also developing a new web-based video production tool that combines multiple AI features with a simple editing timeline.
The Generate Soundtrack tool is launching in public beta in the Firefly app, and works by assessing an uploaded video and then generating a selection of instrumental audio clips that automatically synchronize to the footage. Users can direct the style of the music by selecting from provided presets like lofi, hip-hop, classical, EDM, and more, or describe the desired vibe in the provided text prompt interface — asking it to be more sentimental, aggressive, and so on. The tool will also suggest a prompt based on the uploaded video footage that can be used as a starting point.
“We wanna help users prompt music. It’s a new muscle we need to develop, so in order to make that easier and more accessible, if you give us your clip, we will predict what type of music goes with that clip,” Adobe’s generative AI head Alexandru Costin told The Verge. “But we’re also offering you this Mad Libs approach where you can pick the vibe, the style, the objective of your clip.”
Generate Soundtrack will provide four distinct audio clip variations per prompt, with each clip having a maximum length of five minutes. The Firefly AI model that Generate Soundtrack is built around was trained on licensed content, meaning it won’t place creators at risk of having videos taken down by copyright strikes. “We purchased music and voice from IP owners, that’s why we have the confidence to offer it as commercially safe,” said Costin.
That promise gives Adobe’s AI music-making efforts a leg up over competing companies like Suno and Udio, which have been targeted with copyright infringement lawsuits and admitted to training their own AI models on protected materials. Generate Soundtrack is only designed for background audio, but Adobe is also working on other tools that could provide a more comprehensive AI music production experience — one that aims to save creators the headache of navigating IP laws.
Firefly’s Generate Speech tool is also launching in public beta, and can be used to create voiceovers for video projects from text. It provides more than 50 voices powered by either Adobe’s Firefly Speech Model or ElevenLabs, with support for more than 20 languages. Users can fine-tune things like speed, pitch, and emotion, and manually correct pronunciation for names or words that may have regional variations.
Another in-development filmmaking tool is the Firefly video editor, which Adobe describes as a “multitrack timeline editor for generating, organizing, trimming, and sequencing clips.” It combines Adobe’s various tools for generating voiceovers, soundtracks, and titles into a single web-based app, alongside frame-by-frame editing features and style presets. The Firefly video editor will start rolling out in private beta next month, with prospective users required to sign up to a waitlist for early access.
