Baselines


Blind Test Set

Subtask1

The baseline WERs for Subtask1 blind set are as follows:

Language (% WER)
Hindi 37.20
Marathi 29.04
Odiya 38.46
Tamil 34.09
Telugu 31.44
Gujarati 26.15
Average 32.73

Subtask2

The baseline WERs for Subtask2 blind set are as follows:

Language (% WER) (% Transliterated WER)
Hindi - English 25.53 23.80
Bengali - English 32.81 31.70
Average 29.17 27.75

Test Set

Baselines are built using Kaldi (Hybrid) and ESPNet (End-to-End). Please go to the link for instructions on how to replicate the baselines.

The Word Error Rates of the baseline systems for Sub-task 1 are below:

Hybrid - Kaldi Based System
Language GMM-HMM (% WER) TDNN (% WER)
Hindi 69.03 40.41
Marathi 33.22 22.44
Odiya 55.78 39.06
Tamil 48.81 33.35
Telugu 47.27 30.62
Gujarati 28.33 19.27
Average 46.88 30.73

The Word Error Rates of the baseline systems for Sub-task 2 are below:

Hybrid - Kaldi Based System End-to-End ESPnet Based System
Language GMM-HMM (% WER) TDNN (% WER) (% WER)
Hindi - English 44.30 36.94 27.7
Bengali - English 39.19 34.31 37.2
Average 41.75 35.63 32.45