Evaluating & Implementing AI
Quality is one of the most important aspects to consider when choosing an AI provider. There are tried and tested methods for determining the quality of machine translation content, and they don’t include asking translators open-ended questions! In this module, we will provide an overview of different ways to evaluate the quality of machine translation, including error analysis (which is later used to customise systems and/or inform post-editors through post-editing guidelines).
Concepts.
Is automatic translation the same for all languages and content types?
Languages - from the most suitable to the most complex
Content - required characteristics for suitable content
Automatic metrics (from Bleu to Comet)
Human evaluation
- Quality evaluation - AB testing
Most common errors
Gross mistranslations
Post-edit distance
Associated
Exercises
The proposed exercise at the end of this module will be to make a selection of two or three potential AI providers using the systematic method explained during the session.
Latest Resources
Stay on top of industry trends, best practices, and practical tips to make your job easier
