</style>
Leaderboard (Final)
Subtask1
Leaderboard1 for Subtask1 is based on evaluation on all the blind test samples (both channel matched and mismatched conditions during data collection between blind test and non-blind test). The best submission (out of the 5) of every team is considered. Ranks are calculated on the basis of Average WER.
# | Team Name | Hindi (% WER) | Marathi (% WER) | Oriya (% WER) | Tamil (% WER) | Telugu (% WER) | Gujarati (% WER) | Average (% WER) |
---|---|---|---|---|---|---|---|---|
1 | CSTR | 14.33 | 15.79 | 25.34 | 23.16 | 21.88 | 20.59 | 20.18 |
2 | Bytedance-SA | 16.59 | 15.65 | 17.81 | 28.59 | 25.37 | 21.3 | 20.89 |
3 | EthereumMiner | 17.54 | 20.15 | 19.99 | 28.52 | 26.08 | 20.11 | 22.06 |
4 | Uniphore | 22.79 | 14.9 | 29.55 | 18.8 | 28.69 | 22.79 | 22.92 |
5 | GOT-HIM | 17.18 | 18.48 | 29.99 | 29.66 | 28.74 | 21.79 | 24.31 |
6 | GoVivace | 21.77 | 25.73 | 29.05 | 28.92 | 26.5 | 21.22 | 25.53 |
7 | Ekstep | 12.24 | 39.74 | 27.1 | 27.2 | 22.43 | 30.65 | 26.56 |
8 | TUTU | 19.93 | 26.52 | 34.18 | 27.69 | 30.25 | 25.34 | 27.32 |
9 | TCS-SpeechNLP | 19.77 | 37.45 | 35.21 | 26.26 | 26.82 | 28.53 | 29.0 |
10 | Lottery | 17.81 | 58.78 | 17.74 | 30.69 | 27.67 | 23.62 | 29.39 |
11 | IIITHSPL | 31.11 | 33.8 | 37.19 | 35.03 | 17.0 | 26.94 | 30.18 |
12 | ScribeTech | 27.78 | 33.05 | 34.57 | 33.01 | 30.08 | 28.22 | 31.12 |
13 | Dialpad | 21.49 | 46.41 | 32.13 | 28.6 | 28.03 | 34.57 | 31.87 |
14 | Baseline | 37.2 | 29.04 | 38.46 | 34.09 | 31.44 | 26.15 | 32.73 |
15 | Sayint | 28.72 | 36.49 | 36 | 36.41 | 32.89 | 28.29 | 33.13 |
16 | Jio Speech | 35.53 | 36.33 | 38.55 | 33.69 | 31.14 | 24.79 | 33.34 |
17 | Nuronics | 38.02 | 39.12 | 48.4 | 34.89 | 33.11 | 29.68 | 37.23 |
18 | IITM-SMT-Lab | 19.51 | 85.92 | 37.13 | 32.01 | 30.34 | 32.94 | 39.64 |
19 | SRI-B | 32.47 | 76.27 | 47.72 | 27.97 | 29.13 | 30.17 | 40.62 |
20 | Dheeyantra | 35.97 | 27.8 | 32.85 | 73.77 | 43.78 | 39.75 | 42 .32 |
21 | HAL101 | 20.74 | 96.75 | 36.07 | 37.95 | 34.79 | 34.08 | 43.4 |
22 | IITM Speech Lab | 23.79 | 105.61 | 37.95 | 52.27 | 43.98 | 41.86 | 50.91 |
23 | MIDAS | 20.84 | 68.59 | 28.73 | 72.34 | 71.91 | 49.53 | 51.99 |
24 | INDIGO-IITG | 54.39 | 111.45 | 46.85 | 76.26 | 80.94 | 69.54 | 73.24 |
Leaderboard 2 for Subtask1 has 5 languages (all except Marathi) and their average. Leaderboard 2 is based on evaluation of the audio files belonging to all languages except Marathi. The best submission (out of the 5) of every team is considered. The ranks are calculated on the basis of average WER.
# | Team Name | Hindi (% WER) | Oriya (% WER) | Tamil (% WER) | Telugu (% WER) | Gujarati (% WER) | Average (% WER) |
---|---|---|---|---|---|---|---|
1 | CSTR | 14.33 | 25.34 | 23.16 | 21.88 | 20.59 | 21.06 |
2 | Bytedance-SA | 16.59 | 17.81 | 28.59 | 25.37 | 21.3 | 21.93 |
3 | EthereumMiner | 17.54 | 19.99 | 28.52 | 26.08 | 20.11 | 22.45 |
4 | Lottery | 17.81 | 17.74 | 30.69 | 27.67 | 23.62 | 23.51 |
5 | Ekstep | 12.24 | 27.1 | 27.2 | 22.43 | 30.65 | 23.92 |
6 | Uniphore | 22.79 | 29.55 | 18.8 | 28.69 | 22.79 | 24.52 |
7 | GOT-HIM | 17.72 | 29.14 | 27.94 | 26.36 | 22.62 | 24.76 |
8 | GoVivace | 21.77 | 29.05 | 28.92 | 26.5 | 21.22 | 25.49 |
9 | IITM-SMT-Lab | 17.8 | 32.21 | 27.12 | 28.11 | 29.8 | 27.01 |
10 | TCS-SpeechNLP | 19.77 | 35.21 | 26.26 | 26.82 | 28.53 | 27.32 |
11 | TUTU | 19.93 | 34.18 | 27.69 | 30.25 | 25.34 | 27.48 |
12 | Dialpad | 21.49 | 32.13 | 28.6 | 28.03 | 34.57 | 28.96 |
13 | IIITHSPL | 31.11 | 37.19 | 35.03 | 17.0 | 26.94 | 29.45 |
14 | ScribeTech | 27.78 | 34.57 | 33.01 | 30.08 | 28.22 | 30.73 |
15 | Sayint | 28.01 | 35.21 | 35.76 | 32.14 | 28.09 | 31.84 |
16 | HAL101 | 21.42 | 34.66 | 37.92 | 33.92 | 34.37 | 32.46 |
17 | SRI-B | 30.84 | 49.8 | 26.07 | 28.34 | 27.61 | 32.53 |
18 | Jio Speech | 35.53 | 38.55 | 33.69 | 31.14 | 24.79 | 32.74 |
19 | Baseline | 37.2 | 38.46 | 34.09 | 31.44 | 26.15 | 33.47 |
20 | Nuronics | 38.02 | 48.4 | 34.89 | 33.11 | 29.68 | 36.82 |
21 | IITM Speech Lab | 23.79 | 37.95 | 52.27 | 43.98 | 41.86 | 39.97 |
22 | Dheeyantra | 36.88 | 40.31 | 64.02 | 40.62 | 33.6 | 43.09 |
23 | MIDAS | 20.84 | 28.73 | 72.34 | 71.91 | 49.53 | 48.67 |
24 | INDIGO-IITG | 54.39 | 46.85 | 76.26 | 80.94 | 69.54 | 65.60 |
Subtask2
Ranks are determined on the basis of average WER.:
# | Team Name | Hindi-English (% WER) | Bengali-English (% WER) | Average (% WER) |
---|---|---|---|---|
1 | JHU-CLSP/GoVivace | 16.43 | 26.39 | 21.41 |
2 | CSTR | 20.41 | 24.43 | 22.42 |
3 | Sayint | 20.85 | 26.08 | 23.46 |
4 | KARI | 22.54 | 25.33 | 23.93 |
5 | Bytedance-SA | 21.56 | 26.68 | 24.12 |
6 | IITM-SMT-Lab | 22.06 | 27.8 | 24.93 |
7 | Ekstep | 21.77 | 28.27 | 25.02 |
8 | TUTU | 23.96 | 29.37 | 26.67 |
9 | MCSASR | 24.52 | 29.8 | 27.16 |
10 | Baseline | 25.53 | 32.81 | 29.17 |
11 | Jio Speech | 27.69 | 32.59 | 30.14 |
12 | INDIGO-IITG | 27.92 | 33.55 | 30.73 |
13 | ScribeTech | 28.25 | 34.21 | 31.23 |
14 | Deterministic Algorithms Lab | 32.45 | 31.0 | 31.72 |
15 | SRI-B | 28.77 | 35.19 | 31.98 |
16 | MIDAS | 29.81 | 39.6 | 34.71 |
Ranks are determined on the basis of average Transliterated WER.:
# | Team Name | Hindi-English (% WER) | Bengali-English (% WER) | Average (% WER) |
---|---|---|---|---|
1 | CSTR | 15.64 | 22.61 | 19.12 |
2 | JHU-CLSP/GoVivace | 15.51 | 24.67 | 20.09 |
3 | Sayint | 18.78 | 24.34 | 21.56 |
4 | KARI | 20.49 | 24.07 | 22.28 |
5 | Bytedance-SA | 19.65 | 25.29 | 22.47 |
6 | IITM-SMT-Lab | 20.97 | 26.69 | 23.83 |
7 | Ekstep | 20.75 | 26.96 | 23.85 |
8 | TUTU | 22.3 | 28.04 | 25.17 |
9 | MCSASR | 22.54 | 28.57 | 25.55 |
10 | Jio Speech | 23.83 | 30.15 | 26.99 |
11 | INDIGO-IITG | 23.78 | 31.2 | 27.49 |
12 | Baseline | 23.8 | 31.7 | 27.75 |
13 | ScribeTech | 25.1 | 32.48 | 28.79 |
14 | Deterministic Algorithms Lab | 31.15 | 29.82 | 30.48 |
15 | SRI-B | 28.77 | 35.19 | 31.98 |
16 | MIDAS | 28.37 | 38.17 | 33.27 |