How to Automate Media Localization Workflows with AI

This content surge has created an urgent need for more efficient media localization processes and workflows, including the all-essential subtitles and captions (text on screen for language access and accessibility), so that diverse audiences worldwide can be more quickly and easily reached.

Traditional localization methods often include task-based workflows that rely on human intervention. They can be time-consuming, expensive, and create the ideal situation for more errors. Localizing massive amounts of AV content means that scalability is a must. And a logical way to get there (without incurring astronomical costs) is by increasingly automating and streamlining the different components mentioned above.

Media localization has several stages. It begins with transcription when no source language scripts are available, converting spoken language into text. Once a transcript is reviewed to ensure its accuracy and linguistic correctness, it undergoes translation and adaptation (transcreation) to ensure the content fits the target language linguistically and culturally, and that the general length of speech is preserved.

Translated and adapted text can in turn be used for interlingual subtitles and closed captions, or for dubbing (for a new recording of dialogues and narration in the target language). Each component starts with a precise transcript, which plays a critical role in making target AV content accessible and true to the original while being locally relevant.

The Media Localization Process, Step by Step

From transcription to final testing of subtitled, captioned, and/or dubbed AV products, Artificial Intelligence (AI) can now help human experts reduce time, costs, and risks at every stage of the language conversion process, compared to traditional localization models.

AI has a key role in Media localization at all the stages mentioned above. AI-enabled transcription, for example, can quickly and accurately convert spoken language into text. What used to take hours for humans to transcribe, is now automated by core technologies like Automatic Speech Recognition (ASR) and machine learning. ASR plus AI makes a tremendous difference vs manual transcription methods because the process is vastly better in speed and quality.

AI-enabled transcription tools use advanced algorithms and machine learning to analyze audio recordings and generate transcripts in real-time or with little latency. Some of the features that help to greatly accelerate timelines at this stage include automatic speaker identification (aka “speaker diarization” for live broadcasting), which labels each speaker's dialogue, and timecoding to mark specific points in the audio, making it easier to navigate and reference the transcript.

Although accuracy in AI transcription continuously improves, errors can still occur, particularly with accents, dialects, or technical jargon. As a result, human review and quality checks are still necessary for ensuring the final transcript is accurate and free of errors before it is translated.

Machine translation (MT) via a combination of neural machine translation, large language models-enabled translation and even translation memories can also be highly automated. This can dramatically shorten text-based tasks in AV localization and reduce the need for more review cycles.

Something to keep in mind when it comes to MT in creative content is that it might not capture correctly the nuances and cultural context of the target language. For this reason, the intervention of an expert linguist is a must. This person should understand not just the target language and general culture, but also the subject matter at hand.

Another aspect of AV localization that AI can assist with is quality assurance (QA). There are multiple AI-enabled tools that automatically check for errors and inconsistencies as content is machine translated. And based on how the full process is set up, for example, if glossaries are loaded onto a machine translation platform or data like translation memories are annotated before text is translated, the QA process may need a higher or lower degree of intervention by a linguist to ensure quality.

How Subtitles and Closed Captions are Automated

Once clean, high-quality translations are available, AI can be leveraged to generate subtitles and closed captions (specifically tailored for the deaf and hard of hearing communities). Captions can go beyond dialogues or narration, incorporating descriptions of non-speech elements like sound effects, music cues, and speaker identification, all elements essential for viewers with hearing impairments to fully understand the content.

According to a report by the World Intellectual Property Organization (WIPO), AI-enabled subtitling tools are becoming increasingly sophisticated, leveraging machine learning algorithms to improve accuracy and adapt to different accents and languages. These tools can also be trained to recognize specific terminology and jargon, which makes them suitable for specialized AV content, like educational videos.

AI not only increases efficiency and cost-effectiveness in subtitling and captioning, but it also broadens opportunities to add machine-translated/human approved languages and dialects. Certain use cases dispense with the need for a human reviewer, though. One of those is the automatic captioning feature in YouTube, allowing creators to add automatic captions to their videos in order to increase engagement.

Subtitles and captions for authoritative content, though, such as documentaries and online courses, should always be reviewed and edited by a human for accuracy.

How Is AI Used in Dubbing?

A critical technology that has also existed for many years and now enjoys AI enhancements is Text-to-Speech (TTS), which converts written text into natural-sounding speech. TTS is playing an increasing role in dubbing for certain types of AV materials, such as short content creator videos, where AI-generated voices can replace the original audio, maintaining the tone and style of the original.

Advancements in TTS with the integration of AI have also led to the emergence of voice cloning and voice synthesis, which allow the creation of custom voices to mimic actors or narrators. This lends an element of authenticity to dubbed content, even if that sounds like a contradiction when referring to artificially generated voices in other languages.

TTS can also be combined with AI-enabled lip-syncing, a technology that improves the automated dubbing process by synchronizing the generated speech with the on-screen lip movements.

New Technologies, New Challenges

One of the unavoidable elements of implementing new technologies, including AI, is that there could be as many ways of doing things as there are users of the technology. This is not necessarily negative, but unless those who implement AV localization have a deep understanding of the implications of low quality multilingual subtitles and captions, they could face less than desirable outcomes.

The success of an international brand, for example, depends greatly on how it is perceived across languages and geographies. If a video commercial is not correctly translated, adapted, and dubbed, the expense of the marketing campaign riding on it may end up being wasted money. AI is great at accelerating overall localization workflows and time to market for AV materials, but the process still requires a solid set of quality standards.

The standards that can be adapted to the new AI-enabled localization process are not new. The technology is. So the best way to think about quality assurance in an AI-enabled AV localization process is to have expert reviewers ensure the following:

• Training of large language models and MT engines with terminology and locale-specific annotated content

• Review of source transcripts for completeness and accuracy

• Adaptation and review of machine translated content for cultural and subject-specific nuance

• Testing localized subtitles and captions for quality standards applicable to them, including timing, synchronization, readability, and special requirements (such as censored words in a particular country)

As the localization industry evolves and relies on both technology and human experts for different types of quality checks, workflows will continue changing. At the same time, quality standards will change. As AI improves, those who localize or use captions for accessibility will be able to trust its results more and more. Naturally, with improvements all around, end-user expectations will also be higher.