Adobe is bringing new features to its Premiere Pro video editor, including caption translations and the ability to search videos by describing them.
Premiere Pro can now recognize objects, locations, camera angles, and more in videos but doesn’t identify or label people (yet). You can simply describe the footage to find videos in your library via a new Search panel. For example, you can find matching items with simple search keywords like “California” or use natural language for more complex queries, such as “a person skating with a lens flare” or “a person skating with a lens flare,” the company said. The feature will find matching sections of a clip or the relevant range.
If you edit many videos in Premiere Pro, this feature could save you time compared to manually reviewing videos to find the right footage for your project. It takes advantage of an AI-powered computer vision algorithm Adobe calls Media Intelligence. The company says Media Intelligence currently only identifies visuals, not sound. It also does not use optical character recognition, so you can’t find a specific text in a frame.
The app can also find spoken words if the video file has a transcript attached. And beacuse Media Intelligence can analyze metadata, too, you’re able to find videos based on GPS locations, camera types, capture dates, and other information embeded in the metadata.
Adobe’s FAQ pagesays privacy is guaranteed because all processing for Media Intelligence is performed on-device. Nothing is uploaded to the cloud, and no internet connectivity is required for Media Intelligence to work. Your content is not used to train Adobe’s models. Premiere Pro can also now automatically translate captions into 17 languages, with multiple caption tracks visible at the same time. Video is transcribed in real-time using AI voice-to-text technology, and you can then edit the video directly from the transcript.
Adobe isn’t alone in bringing AI capabilities to video editing.Apple’s Final Cut Pro software recently gainedthe ability to automatically create closed captions using a large language model that transcribes spoken audio.Microsoft’s Clipchamp video editor in Windows has a similar feature, where you edit videos by using text transcripts instead of the usual timeline.
Adobe also announced improvements for After Effects and Frame.io, its cloud-based video collaboration platform which now supports Canon’s C80 and C400 cameras. After Effects has a new caching system that should eliminate the need to pause playback to cache or render on slower computers. The app now supports HDR monitoring for PQ and HLG video via a calibrated reference monitor. you’re able to try these improvements in theAfter Effects Beta.
