Adobe Firefly adds AI sound & video tools to boost creativity

Adobe Firefly has introduced a suite of new tools and partner models to streamline AI-driven video production and editing, emphasising flexibility, precision, and creative control for content creators.

New sound effects generation

The new Generate Sound Effects (beta) feature allows users to create bespoke sound effects either by entering a text prompt or using their voice. This tool is designed to enable creators to easily add custom audio elements that fit the emotional and atmospheric requirements of their videos. According to the company, Firefly listens to the timing and energy of the user's voice input to match the action in the corresponding video, providing more cinematic alignment between audio and visuals.

"Sound is a powerful storytelling tool that adds emotion and depth to your videos. Generate Sound Effects (beta) makes it easy to create custom sounds, like a lion's roar or ambient nature sounds, that enhance your visuals. And like our other Firefly generative AI models, Generate Sound Effects (beta) is commercially safe, so you can create with confidence. Just type a simple text prompt to generate the sound effect you need. Want even more control? Use your voice to guide the timing and intensity of the sound. Firefly listens to the energy and rhythm of your voice to place sound effects precisely where they belong - matching the action in your video with cinematic timing," the company said.

Expanded partner ecosystem

Firefly has broadened its partner ecosystem to include additional generative AI models. Users now have access to Moonvalley's Marey, Google's Veo 3 (with audio), and Runway's Gen-4 Video, allowing a greater range of creative options in video style and production without requiring users to move between different applications or workflows. Additional models from Topaz Labs and Luma AI are scheduled to become available soon in Firefly Boards and Generate Video.

The company stated, "Creatives enjoy experimenting with different styles, so we're continuously expanding the models we offer inside the Firefly app. Recently, we added Runway's Gen-4 Video and Google Veo3 with Audio to Firefly Boards and Veo3 with Audio in Generate Video. And there are more partner models coming soon to the Firefly app: Topaz Labs' Image and Video Upscalers and Moonvalley's Marey will be launching soon in Firefly Boards. Luma AI's Ray 2 and Pika 2.2, which are already available in Boards, will soon be added to Generate Video."

Enhanced video controls

Firefly has released advanced video controls that give users the ability to direct specific aspects of composition, pacing and style on a frame-by-frame basis. The app now supports flexible aspect ratio selection - vertical, horizontal, or square - streamlining the creation process for multiple formats such as mobile, widescreen, or social content.

Among the new tools is Composition Reference for Video, which enables creators to upload a reference video and a description, from which Firefly generates new content that maintains the original's visual structure. This is particularly helpful for repurposing content or maintaining consistency across scenes. The Style Presets tool allows users to apply visual styles such as claymation, anime, or line art with a single click, expediting the setting of tone for pitches, briefs, or final productions.

Keyframe Cropping provides an intuitive solution for managing framing transitions. Users set the initial and final frames and the intended crop, and Firefly handles the video generation to fit the format, aiming to make the entire process efficient without exiting the creative workflow.

Composition Reference, Style Presets, and Keyframe Cropping are built to give you more control, more speed, and more creative freedom. And they're just the beginning. Even more enhancements are on the way to help you push your storytelling further.

Text to avatar and prompt enhancement

Firefly has also launched Text to Avatar (beta), which enables creators to generate avatar-led videos from scripts with a few clicks. The tool offers a library of avatars, customisable backgrounds, and a selection of accents to match the desired tone or audience.

The company said, "With Text to avatar (beta), you can turn your scripts into engaging, avatar-led videos in just a few clicks. Choose from a diverse library of avatars, customise your background with a colour, image, or video, and select the accent that best fits your video. Firefly handles the rest." The tool is being used for video lessons, transforming written content for social media, and creating internal training materials with a virtual presenter.

Recognising the challenges some users face in crafting prompts, the new Enhance Prompt feature in Generate Video takes user input and augments it for greater clarity and direction, reducing the time spent refining prompt language.

Commercial safety and creative rights

Adobe maintains that all generative AI models within Firefly are trained only on assets for which it holds the appropriate permissions. The company emphasises that user-generated content within Firefly will not be used for further model training, aiming to respect and protect creator rights.

A user guide and best practice tutorials are available to help users get started quickly and to assist with optimising their creative processes within the Firefly platform.