Neural lingual cure-all: Unbabel open sources Machine Translation (MT) tool
Babel is a term long associated with language and many of us have read the Tower of Babel pages on Wikipedia and elsewhere.
When doing this, if nothing else, you can get some bible study in and also learn that babel comes from the Hebrew word for Babylon.
In the question for language clarity and comprehension in digital terms, San Francisco headquartered Unbabel has detailed its open source work to build an AI-powered human-refined translation platform.
Why bother?
Because the company is trying to help enable multilingual customer service at scale.
The project has now seen the release of COMET (Crosslingual Optimized Metric for Evaluation of Translation), an open source neural framework and metric for Machine Translation (MT) evaluation.
So although Unababel is working to engineer human-refined AI, it is also working to make COMET a technology that reduces the need for human review, enabling (it hopes) the rapid assessment and deployment of accurate machine translation models.
To date, MT evaluation solutions correlate poorly with human judgements of translation quality. An MT translation might misfire with regards to syntax, grammar or other important linguistic elements, leading to miscommunication and, in the worst case, offensive communication.
Neural lingual
COMET captures the meaning similarity between texts with enough granularity to predict human experts’ translation quality judgments. It takes advantage of recent breakthroughs in large-scale cross-lingual neural language modeling, resulting in multilingual and adaptable MT evaluation models of unprecedented accuracy.
“We are launching COMET as an open source, ready-to-use, trained model because it can greatly help drive and accelerate MT research and development to levels of accuracy not seen before. We believe that COMET should be adopted as a new standard measure for assessing the quality of MT systems across multiple languages,” said Alon Lavie, vice president of language technologies at Unbabel, co-creator of the company’s previous product in this space (Metric for Evaluation of Translation With Explicit ORdering) METEOR and consulting professor at Carnegie Mellon University.
Unbabel processes high volumes of translations using highly specialised AI models for customer service solutions. Applied in a range of customer domains, the company’s MT engines are continually retrained to ensure that the highest levels of translation quality and robustness.
COMET’s open source project is hosted on GitHub here.
Approved image use: Unababel