Something to look forward to: Fast real-time translation is one key selling point that tech giants have used to push generative AI, and Microsoft will soon bake it into its Edge browser. While YouTube has been developing translation services for some time, Microsoft aims to extend AI video translations to many other sites and services.
Microsoft unveiled this week an AI-based video translation feature that will soon soon come to Edge. The feature will automatically generate foreign language transcriptions and dubs for videos on an expanding range of sites, including YouTube, LinkedIn (Microsoft owned), Bloomberg, Reuters, and Coursera, among others.
Microsoft posted a brief clip to demonstrate the functionality. The video shows an Edge user clicking on a translation icon near the top of a LinkedIn video of a Microsoft Ignite keynote from CEO Satya Nadella. A menu appears, displaying options for audio or subtitle translations from English into several other languages.
After selecting English and Spanish as the input and output languages from drop-down menus, the user clicks “Translate.” The video briefly pauses, and a Spanish dub replaces the original audio.
Upon the feature’s debut, Edge will support translations from Spanish to English and from English into Spanish, Italian, Russian, and Hindi. Presumably, other languages will follow. Furthermore, real-time interpretation for news sites like Reuters, CNBC, and Bloomberg is planned.
Edge’s translator proposes to offer a browser-wide take on a feature that YouTube has been testing for years. Independent YouTubers and companies have delivered human-translated and dubbed videos on the site for some time, but YouTube announced official support for this trend last year. YouTubers have been able to upload videos with multiple audio tracks since last year, and a small industry of startups helps some of the most popular content creators dub their videos in numerous languages.
Although some of these translations use AI tools, they are combined with human authorship to ensure accuracy. Since Microsoft’s solution will support any given video on multiple sites, it is likely fully automatic, which could raise reliability concerns. Google Translate’s accuracy is known to vary widely depending on the language, and generative AI still hasn’t solved its hallucination problem.
AI-generated summaries have also appeared on YouTube, both from Google and Microsoft. YouTube began experimenting with video summaries last year to complement official descriptions, and Edge can currently use AI to summarize official video transcripts.