WIT3

Web Inventory of Transcribed and Translated Talks

Home 2012-03Human evaluation data

The complete release of the IWSLT 2012 human evaluation data is available here. 

Human evaluation was carried out on all primary runs submitted by participants to the following tasks:

For each task, systems were evaluated on an evaluation set composed of 400 sentences randomly taken from the test set used for automatic evaluation.

The IWSLT 2012 human evaluation focused on System Ranking, which aims at producing a complete ordering of the systems participating in a given task. The ranking evaluation was carried out with the following characteristics:

For further information see:

Marcello Federico, Mauro Cettolo, Luisa Bentivogli, Michael Paul, Sebastian Stüker.
Overview of the IWSLT 2012 evaluation campaign.
In Proceedings of the International Workshop on Spoken Language Translation (IWSLT), Hong Kong, 6-7 December 2012.