gonzalez-agirre commited on
Commit
9bb0c8c
·
1 Parent(s): ff189ac

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -15
README.md CHANGED
@@ -116,22 +116,9 @@ The training corpus consists of several corpora gathered from web crawling and p
116
  ### Training procedure
117
 
118
  The training corpus has been tokenized using a byte version of [Byte-Pair Encoding (BPE)](https://github.com/openai/gpt-2)
119
- used in the original [RoBERTA](https://github.com/p
120
-
121
- ### Author
122
- Text Mining Unit (TeMU) at the Barcelona Supercomputing Center ([email protected])
123
-
124
- ### Contact information
125
- For further information, send an email to <[email protected]>
126
-
127
- ### Copyright
128
- Copyright by the Spanish State Secretariat for Digitalization and Artificial Intelligence (SEDIA) (2022)
129
-
130
- ### Licensing informationytorch/fairseq/tree/master/examples/roberta) model with a vocabulary size of 50,262 tokens.
131
  The RoBERTa-ca-v2 pretraining consists of a masked language model training that follows the approach employed for the RoBERTa base model
132
- with the same hyperparameters as in the original work.
133
- The training lasted a total of 96 hours with 16 NVIDIA V100 GPUs of 16GB DDRAM.
134
-
135
 
136
  ## Evaluation
137
 
 
116
  ### Training procedure
117
 
118
  The training corpus has been tokenized using a byte version of [Byte-Pair Encoding (BPE)](https://github.com/openai/gpt-2)
119
+ used in the original [RoBERTA](https://github.com/pytorch/fairseq/tree/master/examples/roberta) model with a vocabulary size of 50,262 tokens.
 
 
 
 
 
 
 
 
 
 
 
120
  The RoBERTa-ca-v2 pretraining consists of a masked language model training that follows the approach employed for the RoBERTa base model
121
+ with the same hyperparameters as in the original work. The training lasted a total of 96 hours with 16 NVIDIA V100 GPUs of 16GB DDRAM.
 
 
122
 
123
  ## Evaluation
124