AWS’ transcription platform is now powered by generative AI

AWS added new languages to its Amazon Transcribe product, offering speech foundation model-based transcription for 100 languages and a slew of new AI capabilities for customers. 

Announced during the AWS re: Invent event, Amazon Transcribe can now recognize more spoken languages and spin up a call transcription. AWS customers use Transcribe to add speech-to-text capabilities to their apps on the AWS Cloud. 

The company said in a blog post that Transcribe trained on “millions of hours of unlabeled audio data from over 100 languages” and uses self-supervised algorithms to learn patterns of human speech in different languages and accents. AWS said it ensured that some languages were not overrepresented in the training data to ensure that lesser-used languages could be as accurate as more frequently spoken ones. 

In late 2023, Amazon Transcribe supported 39 languages.

Amazon Transcribe improves accuracy by 20 to 50 percent over its previous version across many languages, according to AWS. It also offers automatic punctuation, custom vocabulary, automatic language identification, and custom vocabulary filters. It can recognize speech in audio and video formats and noisy environments. 

The Verge reached out to AWS for information on which foundation models it used for Amazon Transcribe.

With better language recognition, AWS said advances with Amazon Transcribe also bleed into better accuracy with its Call Analytics platform, which its contact center customers often use. Amazon Transcribe Call Analytics, now also powered by generative AI models, summarizes interactions between an agent and a customer. AWS said this cuts down on after-call work creating reports, and managers can quickly read information without needing to go through the entire transcript. 

Of course, AWS is not the only company offering AI-powered transcription services. Otter has been providing AI transcriptions to consumers and enterprises for a while and released a summarization tool in June. While not exactly the same, Meta announced it is working on a generative AI-powered translation model that recognizes nearly 100 spoken languages.

AWS also announced additional capabilities to its Amazon Personalization product, which allows clients to offer products or show recommendations to customers, like how streaming services can suggest new shows based on previous activity. AWS added Content Generation, which will write titles or email subject lines to thematically connect recommendation lists. 

Correction, November 28 3:20 PM PT: Corrected to reflect the number of languages supported by Transcribe in 2023 and clarified accuracy information.