Jonathan DownieThere's a fashion for people using automated metrics to evaluate the quality of #interpreting. In this research article, Joss Moorkens and I explain why that doesn't work and what to do instead.
https://arxiv.org/abs/2601.05864
What do the metrics mean? A critical analysis of the use of Automated Evaluation Metrics in Interpreting
With the growth of interpreting technologies, from remote interpreting and Computer-Aided Interpreting to automated speech translation and interpreting avatars, there is now a high demand for ways to quickly and efficiently measure the quality of any interpreting delivered. A range of approaches to fulfil the need for quick and efficient quality measurement have been proposed, each involving some measure of automation. This article examines these recently-proposed quality measurement methods and will discuss their suitability for measuring the quality of authentic interpreting practice, whether delivered by humans or machines, concluding that automatic metrics as currently proposed cannot take into account the communicative context and thus are not viable measures of the quality of any interpreting provision when used on their own. Across all attempts to measure or even categorise quality in Interpreting Studies, the contexts in which interpreting takes place have become fundamental to the final analysis.
CUHK_CTT
FID Anglo-American Culture
avldigital.de
Kevin Lossner
Sarah Del Grosso
Péter Király