Hugging Face's logo

KhushiDS
/
wav2vec2-large-xlsr-53-oria

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
Oriya
wav2vec2
Eval Results
Inference Endpoints
Model card Files Metrics Community
KhushiDS commited on
Commit
7a19d26
·
verified ·
1 Parent(s): 2b2d536

Upload folder using huggingface_hub

Browse files
Files changed (10) hide show
  1. README.md +112 -3
  2. added_tokens.json +4 -0
  3. config.json +117 -0
  4. model.safetensors +3 -0
  5. preprocessor_config.json +9 -0
  6. runs/Sep12_19-14-44_speech2text-large/events.out.tfevents.1726168574.speech2text-large.433138.0 +3 -0
  7. special_tokens_map.json +6 -0
  8. tokenizer_config.json +47 -0
  9. training_args.bin +3 -0
  10. vocab.json +132 -0
README.md CHANGED
@@ -1,3 +1,112 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: facebook/wav2vec2-large-xlsr-53
3
+ datasets:
4
+ - google/fleurs
5
+ library_name: transformers
6
+ license: apache-2.0
7
+ metrics:
8
+ - wer
9
+ model-index:
10
+ - name: wav2vec2-large-xlsr-53-oria-Version2
11
+ results:
12
+ - task:
13
+ type: automatic-speech-recognition
14
+ name: Automatic Speech Recognition
15
+ dataset:
16
+ name: fleurs
17
+ type: fleurs
18
+ config: or_in
19
+ split: None
20
+ args: or_in
21
+ metrics:
22
+ - type: wer
23
+ value: 0.43801320700545504
24
+ name: Wer
25
+ language:
26
+ - or
27
+ pipeline_tag: automatic-speech-recognition
28
+ ---
29
+
30
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
31
+ should probably proofread and complete it, then remove this comment. -->
32
+
33
+ # wav2vec2-large-xlsr-53-oria-Version2
34
+
35
+ This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the fleurs dataset.
36
+ It achieves the following results on the evaluation set:
37
+ - Loss: 0.7388
38
+ - Wer: 0.4380
39
+
40
+ ## Model description
41
+
42
+ More information needed
43
+
44
+ ## Intended uses & limitations
45
+
46
+ More information needed
47
+
48
+ ## Training and evaluation data
49
+
50
+ More information needed
51
+
52
+ ## Training procedure
53
+
54
+ ### Training hyperparameters
55
+
56
+ The following hyperparameters were used during training:
57
+ - learning_rate: 0.0003
58
+ - train_batch_size: 16
59
+ - eval_batch_size: 8
60
+ - seed: 42
61
+ - gradient_accumulation_steps: 2
62
+ - total_train_batch_size: 32
63
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
64
+ - lr_scheduler_type: linear
65
+ - lr_scheduler_warmup_steps: 500
66
+ - num_epochs: 70
67
+ - mixed_precision_training: Native AMP
68
+
69
+ ### Training results
70
+
71
+ | Training Loss | Epoch | Step | Validation Loss | Wer |
72
+ |:-------------:|:-------:|:----:|:---------------:|:------:|
73
+ | 7.1886 | 2.1505 | 100 | 5.8077 | 1.0 |
74
+ | 3.4942 | 4.3011 | 200 | 3.4903 | 1.0 |
75
+ | 3.4535 | 6.4516 | 300 | 3.4622 | 1.0 |
76
+ | 3.4533 | 8.6022 | 400 | 3.4484 | 1.0 |
77
+ | 3.5108 | 10.7527 | 500 | 3.4461 | 1.0 |
78
+ | 3.2951 | 12.9032 | 600 | 3.2130 | 1.0 |
79
+ | 1.1127 | 15.0538 | 700 | 1.0820 | 0.8127 |
80
+ | 0.6285 | 17.2043 | 800 | 0.7477 | 0.6510 |
81
+ | 0.5072 | 19.3548 | 900 | 0.6540 | 0.5758 |
82
+ | 0.3865 | 21.5054 | 1000 | 0.6212 | 0.5475 |
83
+ | 0.2889 | 23.6559 | 1100 | 0.6020 | 0.5204 |
84
+ | 0.2316 | 25.8065 | 1200 | 0.6334 | 0.5086 |
85
+ | 0.2105 | 27.9570 | 1300 | 0.6443 | 0.5007 |
86
+ | 0.2124 | 30.1075 | 1400 | 0.6318 | 0.4970 |
87
+ | 0.1916 | 32.2581 | 1500 | 0.6638 | 0.4918 |
88
+ | 0.1487 | 34.4086 | 1600 | 0.6562 | 0.4784 |
89
+ | 0.1742 | 36.5591 | 1700 | 0.6583 | 0.4808 |
90
+ | 0.1542 | 38.7097 | 1800 | 0.6618 | 0.4662 |
91
+ | 0.1015 | 40.8602 | 1900 | 0.6770 | 0.4652 |
92
+ | 0.1173 | 43.0108 | 2000 | 0.6982 | 0.4617 |
93
+ | 0.1162 | 45.1613 | 2100 | 0.7020 | 0.4575 |
94
+ | 0.1077 | 47.3118 | 2200 | 0.7189 | 0.4551 |
95
+ | 0.0914 | 49.4624 | 2300 | 0.7335 | 0.4536 |
96
+ | 0.1038 | 51.6129 | 2400 | 0.7180 | 0.4554 |
97
+ | 0.0814 | 53.7634 | 2500 | 0.7276 | 0.4524 |
98
+ | 0.0782 | 55.9140 | 2600 | 0.7411 | 0.4486 |
99
+ | 0.1001 | 58.0645 | 2700 | 0.7250 | 0.4474 |
100
+ | 0.0781 | 60.2151 | 2800 | 0.7226 | 0.4475 |
101
+ | 0.0677 | 62.3656 | 2900 | 0.7344 | 0.4426 |
102
+ | 0.0757 | 64.5161 | 3000 | 0.7341 | 0.4405 |
103
+ | 0.0636 | 66.6667 | 3100 | 0.7394 | 0.4390 |
104
+ | 0.0762 | 68.8172 | 3200 | 0.7388 | 0.4380 |
105
+
106
+
107
+ ### Framework versions
108
+
109
+ - Transformers 4.45.0.dev0
110
+ - Pytorch 2.4.1+cu121
111
+ - Datasets 2.21.0
112
+ - Tokenizers 0.19.1
added_tokens.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "</s>": 131,
3
+ "<s>": 130
4
+ }
config.json ADDED
@@ -0,0 +1,117 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "facebook/wav2vec2-large-xlsr-53",
3
+ "activation_dropout": 0.0,
4
+ "adapter_attn_dim": null,
5
+ "adapter_kernel_size": 3,
6
+ "adapter_stride": 2,
7
+ "add_adapter": false,
8
+ "apply_spec_augment": true,
9
+ "architectures": [
10
+ "Wav2Vec2ForCTC"
11
+ ],
12
+ "attention_dropout": 0.1,
13
+ "bos_token_id": 1,
14
+ "classifier_proj_size": 256,
15
+ "codevector_dim": 768,
16
+ "contrastive_logits_temperature": 0.1,
17
+ "conv_bias": true,
18
+ "conv_dim": [
19
+ 512,
20
+ 512,
21
+ 512,
22
+ 512,
23
+ 512,
24
+ 512,
25
+ 512
26
+ ],
27
+ "conv_kernel": [
28
+ 10,
29
+ 3,
30
+ 3,
31
+ 3,
32
+ 3,
33
+ 2,
34
+ 2
35
+ ],
36
+ "conv_stride": [
37
+ 5,
38
+ 2,
39
+ 2,
40
+ 2,
41
+ 2,
42
+ 2,
43
+ 2
44
+ ],
45
+ "ctc_loss_reduction": "mean",
46
+ "ctc_zero_infinity": false,
47
+ "diversity_loss_weight": 0.1,
48
+ "do_stable_layer_norm": true,
49
+ "eos_token_id": 2,
50
+ "feat_extract_activation": "gelu",
51
+ "feat_extract_dropout": 0.0,
52
+ "feat_extract_norm": "layer",
53
+ "feat_proj_dropout": 0.0,
54
+ "feat_quantizer_dropout": 0.0,
55
+ "final_dropout": 0.0,
56
+ "gradient_checkpointing": false,
57
+ "hidden_act": "gelu",
58
+ "hidden_dropout": 0.1,
59
+ "hidden_size": 1024,
60
+ "initializer_range": 0.02,
61
+ "intermediate_size": 4096,
62
+ "layer_norm_eps": 1e-05,
63
+ "layerdrop": 0.1,
64
+ "mask_channel_length": 10,
65
+ "mask_channel_min_space": 1,
66
+ "mask_channel_other": 0.0,
67
+ "mask_channel_prob": 0.0,
68
+ "mask_channel_selection": "static",
69
+ "mask_feature_length": 10,
70
+ "mask_feature_min_masks": 0,
71
+ "mask_feature_prob": 0.0,
72
+ "mask_time_length": 10,
73
+ "mask_time_min_masks": 2,
74
+ "mask_time_min_space": 1,
75
+ "mask_time_other": 0.0,
76
+ "mask_time_prob": 0.05,
77
+ "mask_time_selection": "static",
78
+ "model_type": "wav2vec2",
79
+ "num_adapter_layers": 3,
80
+ "num_attention_heads": 16,
81
+ "num_codevector_groups": 2,
82
+ "num_codevectors_per_group": 320,
83
+ "num_conv_pos_embedding_groups": 16,
84
+ "num_conv_pos_embeddings": 128,
85
+ "num_feat_extract_layers": 7,
86
+ "num_hidden_layers": 24,
87
+ "num_negatives": 100,
88
+ "output_hidden_size": 1024,
89
+ "pad_token_id": 129,
90
+ "proj_codevector_dim": 768,
91
+ "tdnn_dilation": [
92
+ 1,
93
+ 2,
94
+ 3,
95
+ 1,
96
+ 1
97
+ ],
98
+ "tdnn_dim": [
99
+ 512,
100
+ 512,
101
+ 512,
102
+ 512,
103
+ 1500
104
+ ],
105
+ "tdnn_kernel": [
106
+ 5,
107
+ 3,
108
+ 3,
109
+ 1,
110
+ 1
111
+ ],
112
+ "torch_dtype": "float32",
113
+ "transformers_version": "4.45.0.dev0",
114
+ "use_weighted_layer_sum": false,
115
+ "vocab_size": 132,
116
+ "xvector_output_dim": 512
117
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcb26c8d6cb649c534a3fa0d64231c218f079833d9b3f2ec478b06cad39814d8
3
+ size 1262348680
preprocessor_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "feature_extractor_type": "Wav2Vec2FeatureExtractor",
4
+ "feature_size": 1,
5
+ "padding_side": "right",
6
+ "padding_value": 0.0,
7
+ "return_attention_mask": true,
8
+ "sampling_rate": 16000
9
+ }
runs/Sep12_19-14-44_speech2text-large/events.out.tfevents.1726168574.speech2text-large.433138.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f5ddfc4815632010d758cdf86a22a37f4c6b9f829ee37a90d7eec0846c3c5ae
3
+ size 85099
special_tokens_map.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<s>",
3
+ "eos_token": "</s>",
4
+ "pad_token": "[PAD]",
5
+ "unk_token": "[UNK]"
6
+ }
tokenizer_config.json ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "128": {
4
+ "content": "[UNK]",
5
+ "lstrip": true,
6
+ "normalized": false,
7
+ "rstrip": true,
8
+ "single_word": false,
9
+ "special": false
10
+ },
11
+ "129": {
12
+ "content": "[PAD]",
13
+ "lstrip": true,
14
+ "normalized": false,
15
+ "rstrip": true,
16
+ "single_word": false,
17
+ "special": false
18
+ },
19
+ "130": {
20
+ "content": "<s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "131": {
28
+ "content": "</s>",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ }
35
+ },
36
+ "bos_token": "<s>",
37
+ "clean_up_tokenization_spaces": true,
38
+ "do_lower_case": false,
39
+ "eos_token": "</s>",
40
+ "model_max_length": 1000000000000000019884624838656,
41
+ "pad_token": "[PAD]",
42
+ "replace_word_delimiter_char": " ",
43
+ "target_lang": null,
44
+ "tokenizer_class": "Wav2Vec2CTCTokenizer",
45
+ "unk_token": "[UNK]",
46
+ "word_delimiter_token": "|"
47
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c16be36d4ca13ed225f2c54c415b55b73234401ea6c7471f85a0149ca1cc5d88
3
+ size 5240
vocab.json ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "$": 38,
3
+ "'": 103,
4
+ "*": 73,
5
+ "+": 27,
6
+ "/": 26,
7
+ "0": 111,
8
+ "1": 31,
9
+ "2": 100,
10
+ "3": 85,
11
+ "4": 123,
12
+ "5": 15,
13
+ "6": 84,
14
+ "7": 72,
15
+ "8": 20,
16
+ "9": 108,
17
+ "[": 125,
18
+ "[PAD]": 129,
19
+ "[UNK]": 128,
20
+ "]": 22,
21
+ "a": 98,
22
+ "b": 14,
23
+ "c": 29,
24
+ "d": 58,
25
+ "e": 53,
26
+ "f": 0,
27
+ "g": 124,
28
+ "h": 45,
29
+ "i": 122,
30
+ "j": 94,
31
+ "k": 57,
32
+ "l": 41,
33
+ "m": 67,
34
+ "n": 50,
35
+ "o": 48,
36
+ "p": 119,
37
+ "q": 62,
38
+ "r": 32,
39
+ "s": 60,
40
+ "t": 59,
41
+ "u": 16,
42
+ "v": 91,
43
+ "w": 74,
44
+ "x": 112,
45
+ "y": 78,
46
+ "z": 86,
47
+ "|": 105,
48
+ "£": 61,
49
+ "¥": 93,
50
+ "°": 75,
51
+ "½": 19,
52
+ "¾": 80,
53
+ "õ": 107,
54
+ "।": 102,
55
+ "ଁ": 2,
56
+ "ଂ": 7,
57
+ "ଃ": 115,
58
+ "ଅ": 118,
59
+ "ଆ": 49,
60
+ "ଇ": 13,
61
+ "ଈ": 116,
62
+ "ଉ": 69,
63
+ "ଊ": 87,
64
+ "ଋ": 52,
65
+ "ଏ": 104,
66
+ "ଐ": 90,
67
+ "ଓ": 65,
68
+ "ଔ": 34,
69
+ "କ": 21,
70
+ "ଖ": 30,
71
+ "ଗ": 92,
72
+ "ଘ": 5,
73
+ "ଙ": 101,
74
+ "ଚ": 4,
75
+ "ଛ": 71,
76
+ "ଜ": 39,
77
+ "ଝ": 33,
78
+ "ଞ": 55,
79
+ "ଟ": 1,
80
+ "ଠ": 54,
81
+ "ଡ": 12,
82
+ "ଢ": 9,
83
+ "ଣ": 70,
84
+ "ତ": 76,
85
+ "ଥ": 120,
86
+ "ଦ": 64,
87
+ "ଧ": 23,
88
+ "ନ": 79,
89
+ "ପ": 6,
90
+ "ଫ": 37,
91
+ "ବ": 109,
92
+ "ଭ": 89,
93
+ "ମ": 10,
94
+ "ଯ": 47,
95
+ "ର": 43,
96
+ "ଲ": 56,
97
+ "ଳ": 97,
98
+ "ଵ": 106,
99
+ "ଶ": 44,
100
+ "ଷ": 95,
101
+ "ସ": 96,
102
+ "ହ": 11,
103
+ "଼": 66,
104
+ "ା": 99,
105
+ "ି": 81,
106
+ "ୀ": 121,
107
+ "ୁ": 35,
108
+ "ୂ": 25,
109
+ "ୃ": 18,
110
+ "ୄ": 117,
111
+ "େ": 36,
112
+ "ୈ": 83,
113
+ "ୋ": 77,
114
+ "ୌ": 82,
115
+ "୍": 28,
116
+ "ୗ": 114,
117
+ "ୟ": 88,
118
+ "୦": 127,
119
+ "୧": 68,
120
+ "୨": 17,
121
+ "୩": 46,
122
+ "୪": 110,
123
+ "୬": 24,
124
+ "୭": 51,
125
+ "ୱ": 63,
126
+ "​": 40,
127
+ "‌": 126,
128
+ "‍": 113,
129
+ "–": 42,
130
+ "—": 8,
131
+ "’": 3
132
+ }