Baselines
Blind Test Set
Subtask1
The baseline WERs for Subtask1 blind set are as follows:
| Language | (% WER) |
|---|---|
| Hindi | 37.20 |
| Marathi | 29.04 |
| Odiya | 38.46 |
| Tamil | 34.09 |
| Telugu | 31.44 |
| Gujarati | 26.15 |
| Average | 32.73 |
Subtask2
The baseline WERs for Subtask2 blind set are as follows:
| Language | (% WER) | (% Transliterated WER) |
|---|---|---|
| Hindi - English | 25.53 | 23.80 |
| Bengali - English | 32.81 | 31.70 |
| Average | 29.17 | 27.75 |
Test Set
Baselines are built using Kaldi (Hybrid) and ESPNet (End-to-End). Please go to the link for instructions on how to replicate the baselines.
The Word Error Rates of the baseline systems for Sub-task 1 are below:
| Hybrid - Kaldi Based System | |||
|---|---|---|---|
| Language | GMM-HMM (% WER) | TDNN (% WER) | |
| Hindi | 69.03 | 40.41 | |
| Marathi | 33.22 | 22.44 | |
| Odiya | 55.78 | 39.06 | |
| Tamil | 48.81 | 33.35 | |
| Telugu | 47.27 | 30.62 | |
| Gujarati | 28.33 | 19.27 | |
| Average | 46.88 | 30.73 | |
The Word Error Rates of the baseline systems for Sub-task 2 are below:
| Hybrid - Kaldi Based System | End-to-End ESPnet Based System | ||
|---|---|---|---|
| Language | GMM-HMM (% WER) | TDNN (% WER) | (% WER) |
| Hindi - English | 44.30 | 36.94 | 27.7 |
| Bengali - English | 39.19 | 34.31 | 37.2 |
| Average | 41.75 | 35.63 | 32.45 |