Baselines
Blind Test Set
Subtask1
The baseline WERs for Subtask1 blind set are as follows:
Language | (% WER) |
---|---|
Hindi | 37.20 |
Marathi | 29.04 |
Odiya | 38.46 |
Tamil | 34.09 |
Telugu | 31.44 |
Gujarati | 26.15 |
Average | 32.73 |
Subtask2
The baseline WERs for Subtask2 blind set are as follows:
Language | (% WER) | (% Transliterated WER) |
---|---|---|
Hindi - English | 25.53 | 23.80 |
Bengali - English | 32.81 | 31.70 |
Average | 29.17 | 27.75 |
Test Set
Baselines are built using Kaldi (Hybrid) and ESPNet (End-to-End). Please go to the link for instructions on how to replicate the baselines.
The Word Error Rates of the baseline systems for Sub-task 1 are below:
Hybrid - Kaldi Based System | |||
---|---|---|---|
Language | GMM-HMM (% WER) | TDNN (% WER) | |
Hindi | 69.03 | 40.41 | |
Marathi | 33.22 | 22.44 | |
Odiya | 55.78 | 39.06 | |
Tamil | 48.81 | 33.35 | |
Telugu | 47.27 | 30.62 | |
Gujarati | 28.33 | 19.27 | |
Average | 46.88 | 30.73 |
The Word Error Rates of the baseline systems for Sub-task 2 are below:
Hybrid - Kaldi Based System | End-to-End ESPnet Based System | ||
---|---|---|---|
Language | GMM-HMM (% WER) | TDNN (% WER) | (% WER) |
Hindi - English | 44.30 | 36.94 | 27.7 |
Bengali - English | 39.19 | 34.31 | 37.2 |
Average | 41.75 | 35.63 | 32.45 |