Adobe has released the latest version of Firefly that now leans heavily on AI for nearly every facet of video and image post-production. The updated app can now use AI to generate narration, music, images and video clips, while even helping you to brainstorm ideas and piece together clips. Many creators may find it distasteful to lean on AI for nearly every aspect of production, but Adobe calls it “a tool for, not a replacement of, human creativity.”
Firefly has mostly been a content generation tool until now, but Adobe has now introduced the Firefly video editor into private beta. It’s a web-based multitrack timeline editor, not unlike Adobe Premiere Pro, that lets you generate, organize, trim and arrange clips, with tools to add voiceovers, soundtracks and titles. You can organize existing Firefly content or generate new ones inside the editor (with presets like claymation, anime and 2D), and combine that with captured media. All that can be edited with “frame-by-frame precision or through a built-in transcript,” Adobe said.
On top of video, Firefly eliminates the need for humans to make voiceovers and music, too. Adobe’s new Generate Soundtrack (public beta) is a Firefly Audio Model-powered AI music generator that lets you select a style or comes up with one to match any clip you upload. It then syncs and times it precisely with that footage.
Generate Speech, meanwhile, does the same thing for voiceovers. It gives you a choice between Firefly’s Speech Model and one for ElevenLabs, letting you generate “lifelike voices in multiple languages, and fine-tune emotion, pacing and emphasis for natural, expressive delivery.”
Adobe is also expanding access to its Fire!y Creative Production tool directly in the Fire!y app as a private beta to start with. It’s a complete AI-powered batch image editing system that lets creators piece together clips, automatically replace backgrounds, apply uniform color grading and crop in via a prompt-driven, no code interface.
Then there’s Fire!y Boards, an “AI-powered ideation surface” to brainstorm new concepts. A feature called “Rotate Object” helps you convert 2D images into 3D so you can position objects and people in different poses and rotate them to new perspectives. Two others, PDF exporting and bulk image downloading, speed the the process of sharing visual concepts across projects.
Finally, Prompt to Edit (available now on Firefly) is a conversational editing interface that allows you to use everyday language to describe the edits you want to make to an image, much as you’d use text-to-image tools like Midjourney to create new images. It’s available with Adobe’s latest Fire!y Image Model 5 AI, along with partner models from Black Forest Labs, Google and OpenAI.
With Firefly’s AI now able to handle every aspect of production, you may be wondering if this will result in a wave of unwatchable AI “slop” appearing on YouTube and elsewhere. The answer is “probably,” but it won’t necessarily be cheap. Standalone Firefly subscriptions are $10/month for the basic plan (20 five-second videos), $20/month for the the Pro plan (40 five-second videos) and $199 for the Premium plan (unlimited videos). However, Adobe is throwing in free image and video generation (with some restrictions) for all Firefly and Creative Cloud Pro customers until December 1st. All the new tools are now available either as part of the update, in public beta or in private beta as mentioned above.
This article originally appeared on Engadget at https://www.engadget.com/apps/adobes-firefly-can-now-use-ai-to-generate-soundtracks-speech-and-video-120018593.html?src=rss
