Training in progress, step 239, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: EleutherAI/pythia-1b
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,41 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "EleutherAI/pythia-1b",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "embed_tokens",
+    "lm_head"
+  ],
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "dense",
+    "k_proj",
+    "gate_proj",
+    "v_proj",
+    "dense_h_to_4h",
+    "down_proj",
+    "dense_4h_to_h",
+    "o_proj",
+    "up_proj",
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2e8037fd1742aafeefe9394eb6eb3056e379cec6904dd227fdd37e6f723d139
+size 67126760

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c117807c6d7ec4eab79023545c24abd813201ccbb0d506a573fa5dc435b019a
+size 134325882

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b355518e25eb3f8f0a23ea4bbd5bf062b989e7cbba231f8b4b77cf35bb4106d
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0bd0e7decbf7c91ab9fd757f5ce5ae4cb006710e4bd76818c20a5be991c1f90
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,223 @@
+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "unk_token": "<|endoftext|>"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1706 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.7076239822353811,
+  "eval_steps": 500,
+  "global_step": 239,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0029607698001480384,
+      "grad_norm": 1.2330008745193481,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 2.1766,
+      "step": 1
+    },
+    {
+      "epoch": 0.005921539600296077,
+      "grad_norm": 1.2820323705673218,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 2.9334,
+      "step": 2
+    },
+    {
+      "epoch": 0.008882309400444115,
+      "grad_norm": 1.2554841041564941,
+      "learning_rate": 3e-06,
+      "loss": 2.3881,
+      "step": 3
+    },
+    {
+      "epoch": 0.011843079200592153,
+      "grad_norm": 1.1881266832351685,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 2.0848,
+      "step": 4
+    },
+    {
+      "epoch": 0.014803849000740192,
+      "grad_norm": 1.6150553226470947,
+      "learning_rate": 5e-06,
+      "loss": 2.9626,
+      "step": 5
+    },
+    {
+      "epoch": 0.01776461880088823,
+      "grad_norm": 1.2984336614608765,
+      "learning_rate": 6e-06,
+      "loss": 2.1568,
+      "step": 6
+    },
+    {
+      "epoch": 0.02072538860103627,
+      "grad_norm": 1.2844996452331543,
+      "learning_rate": 7.000000000000001e-06,
+      "loss": 3.2131,
+      "step": 7
+    },
+    {
+      "epoch": 0.023686158401184307,
+      "grad_norm": 1.2315980195999146,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 2.054,
+      "step": 8
+    },
+    {
+      "epoch": 0.026646928201332347,
+      "grad_norm": 1.3966083526611328,
+      "learning_rate": 9e-06,
+      "loss": 2.4242,
+      "step": 9
+    },
+    {
+      "epoch": 0.029607698001480384,
+      "grad_norm": 1.4393157958984375,
+      "learning_rate": 1e-05,
+      "loss": 2.1222,
+      "step": 10
+    },
+    {
+      "epoch": 0.03256846780162842,
+      "grad_norm": 1.2350786924362183,
+      "learning_rate": 1.1000000000000001e-05,
+      "loss": 1.8152,
+      "step": 11
+    },
+    {
+      "epoch": 0.03552923760177646,
+      "grad_norm": 1.1927133798599243,
+      "learning_rate": 1.2e-05,
+      "loss": 2.1521,
+      "step": 12
+    },
+    {
+      "epoch": 0.0384900074019245,
+      "grad_norm": 1.1297754049301147,
+      "learning_rate": 1.3000000000000001e-05,
+      "loss": 2.4425,
+      "step": 13
+    },
+    {
+      "epoch": 0.04145077720207254,
+      "grad_norm": 1.0867418050765991,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 2.423,
+      "step": 14
+    },
+    {
+      "epoch": 0.04441154700222058,
+      "grad_norm": 1.200924277305603,
+      "learning_rate": 1.5e-05,
+      "loss": 2.1261,
+      "step": 15
+    },
+    {
+      "epoch": 0.047372316802368614,
+      "grad_norm": 1.159411907196045,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 1.6653,
+      "step": 16
+    },
+    {
+      "epoch": 0.050333086602516654,
+      "grad_norm": 1.0632994174957275,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 2.0275,
+      "step": 17
+    },
+    {
+      "epoch": 0.053293856402664694,
+      "grad_norm": 1.0881314277648926,
+      "learning_rate": 1.8e-05,
+      "loss": 2.7033,
+      "step": 18
+    },
+    {
+      "epoch": 0.056254626202812734,
+      "grad_norm": 0.9275356531143188,
+      "learning_rate": 1.9e-05,
+      "loss": 2.1658,
+      "step": 19
+    },
+    {
+      "epoch": 0.05921539600296077,
+      "grad_norm": 0.9293656349182129,
+      "learning_rate": 2e-05,
+      "loss": 1.9394,
+      "step": 20
+    },
+    {
+      "epoch": 0.06217616580310881,
+      "grad_norm": 0.8644701838493347,
+      "learning_rate": 2.1e-05,
+      "loss": 2.2393,
+      "step": 21
+    },
+    {
+      "epoch": 0.06513693560325684,
+      "grad_norm": 0.8931481242179871,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 1.729,
+      "step": 22
+    },
+    {
+      "epoch": 0.06809770540340489,
+      "grad_norm": 0.784940242767334,
+      "learning_rate": 2.3000000000000003e-05,
+      "loss": 1.4568,
+      "step": 23
+    },
+    {
+      "epoch": 0.07105847520355292,
+      "grad_norm": 0.9451686143875122,
+      "learning_rate": 2.4e-05,
+      "loss": 2.0144,
+      "step": 24
+    },
+    {
+      "epoch": 0.07401924500370097,
+      "grad_norm": 0.945660412311554,
+      "learning_rate": 2.5e-05,
+      "loss": 2.2391,
+      "step": 25
+    },
+    {
+      "epoch": 0.076980014803849,
+      "grad_norm": 1.0585007667541504,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 2.3989,
+      "step": 26
+    },
+    {
+      "epoch": 0.07994078460399703,
+      "grad_norm": 0.8671739101409912,
+      "learning_rate": 2.7000000000000002e-05,
+      "loss": 1.9778,
+      "step": 27
+    },
+    {
+      "epoch": 0.08290155440414508,
+      "grad_norm": 1.2988766431808472,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 2.9086,
+      "step": 28
+    },
+    {
+      "epoch": 0.08586232420429311,
+      "grad_norm": 1.096251368522644,
+      "learning_rate": 2.9e-05,
+      "loss": 2.6151,
+      "step": 29
+    },
+    {
+      "epoch": 0.08882309400444116,
+      "grad_norm": 0.843379020690918,
+      "learning_rate": 3e-05,
+      "loss": 1.5709,
+      "step": 30
+    },
+    {
+      "epoch": 0.0917838638045892,
+      "grad_norm": 1.0188798904418945,
+      "learning_rate": 3.1e-05,
+      "loss": 2.1272,
+      "step": 31
+    },
+    {
+      "epoch": 0.09474463360473723,
+      "grad_norm": 0.9005125164985657,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 2.2583,
+      "step": 32
+    },
+    {
+      "epoch": 0.09770540340488527,
+      "grad_norm": 0.9418131709098816,
+      "learning_rate": 3.3e-05,
+      "loss": 2.042,
+      "step": 33
+    },
+    {
+      "epoch": 0.10066617320503331,
+      "grad_norm": 0.9076238870620728,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 1.9507,
+      "step": 34
+    },
+    {
+      "epoch": 0.10362694300518134,
+      "grad_norm": 0.8957440853118896,
+      "learning_rate": 3.5e-05,
+      "loss": 1.8002,
+      "step": 35
+    },
+    {
+      "epoch": 0.10658771280532939,
+      "grad_norm": 0.8622126579284668,
+      "learning_rate": 3.6e-05,
+      "loss": 2.0028,
+      "step": 36
+    },
+    {
+      "epoch": 0.10954848260547742,
+      "grad_norm": 0.8732911944389343,
+      "learning_rate": 3.7e-05,
+      "loss": 1.6368,
+      "step": 37
+    },
+    {
+      "epoch": 0.11250925240562547,
+      "grad_norm": 0.8856744170188904,
+      "learning_rate": 3.8e-05,
+      "loss": 1.8424,
+      "step": 38
+    },
+    {
+      "epoch": 0.1154700222057735,
+      "grad_norm": 0.7756940722465515,
+      "learning_rate": 3.9000000000000006e-05,
+      "loss": 1.6186,
+      "step": 39
+    },
+    {
+      "epoch": 0.11843079200592153,
+      "grad_norm": 0.7404859662055969,
+      "learning_rate": 4e-05,
+      "loss": 0.7831,
+      "step": 40
+    },
+    {
+      "epoch": 0.12139156180606958,
+      "grad_norm": 0.7246972322463989,
+      "learning_rate": 4.1e-05,
+      "loss": 0.7162,
+      "step": 41
+    },
+    {
+      "epoch": 0.12435233160621761,
+      "grad_norm": 0.7735653519630432,
+      "learning_rate": 4.2e-05,
+      "loss": 0.649,
+      "step": 42
+    },
+    {
+      "epoch": 0.12731310140636565,
+      "grad_norm": 0.6949583888053894,
+      "learning_rate": 4.3e-05,
+      "loss": 0.579,
+      "step": 43
+    },
+    {
+      "epoch": 0.13027387120651368,
+      "grad_norm": 0.8814601898193359,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 0.5323,
+      "step": 44
+    },
+    {
+      "epoch": 0.13323464100666174,
+      "grad_norm": 0.7667685747146606,
+      "learning_rate": 4.5e-05,
+      "loss": 0.3718,
+      "step": 45
+    },
+    {
+      "epoch": 0.13619541080680977,
+      "grad_norm": 0.7219610810279846,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 0.2537,
+      "step": 46
+    },
+    {
+      "epoch": 0.1391561806069578,
+      "grad_norm": 0.8067429661750793,
+      "learning_rate": 4.7e-05,
+      "loss": 0.2967,
+      "step": 47
+    },
+    {
+      "epoch": 0.14211695040710584,
+      "grad_norm": 0.7931633591651917,
+      "learning_rate": 4.8e-05,
+      "loss": 0.3117,
+      "step": 48
+    },
+    {
+      "epoch": 0.14507772020725387,
+      "grad_norm": 1.1760865449905396,
+      "learning_rate": 4.9e-05,
+      "loss": 0.3627,
+      "step": 49
+    },
+    {
+      "epoch": 0.14803849000740193,
+      "grad_norm": 2.106400489807129,
+      "learning_rate": 5e-05,
+      "loss": 0.5829,
+      "step": 50
+    },
+    {
+      "epoch": 0.15099925980754997,
+      "grad_norm": 1.7357358932495117,
+      "learning_rate": 5.1000000000000006e-05,
+      "loss": 2.1019,
+      "step": 51
+    },
+    {
+      "epoch": 0.153960029607698,
+      "grad_norm": 1.7898359298706055,
+      "learning_rate": 5.2000000000000004e-05,
+      "loss": 2.0167,
+      "step": 52
+    },
+    {
+      "epoch": 0.15692079940784603,
+      "grad_norm": 1.5378262996673584,
+      "learning_rate": 5.300000000000001e-05,
+      "loss": 1.7384,
+      "step": 53
+    },
+    {
+      "epoch": 0.15988156920799407,
+      "grad_norm": 1.3437457084655762,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 1.5327,
+      "step": 54
+    },
+    {
+      "epoch": 0.16284233900814213,
+      "grad_norm": 1.632769227027893,
+      "learning_rate": 5.500000000000001e-05,
+      "loss": 2.0959,
+      "step": 55
+    },
+    {
+      "epoch": 0.16580310880829016,
+      "grad_norm": 1.125044822692871,
+      "learning_rate": 5.6000000000000006e-05,
+      "loss": 1.8964,
+      "step": 56
+    },
+    {
+      "epoch": 0.1687638786084382,
+      "grad_norm": 1.159507155418396,
+      "learning_rate": 5.6999999999999996e-05,
+      "loss": 2.0412,
+      "step": 57
+    },
+    {
+      "epoch": 0.17172464840858623,
+      "grad_norm": 1.1605857610702515,
+      "learning_rate": 5.8e-05,
+      "loss": 2.3957,
+      "step": 58
+    },
+    {
+      "epoch": 0.17468541820873426,
+      "grad_norm": 0.8025280833244324,
+      "learning_rate": 5.9e-05,
+      "loss": 1.4531,
+      "step": 59
+    },
+    {
+      "epoch": 0.17764618800888232,
+      "grad_norm": 1.0899537801742554,
+      "learning_rate": 6e-05,
+      "loss": 1.9804,
+      "step": 60
+    },
+    {
+      "epoch": 0.18060695780903036,
+      "grad_norm": 0.9055036306381226,
+      "learning_rate": 6.1e-05,
+      "loss": 1.6996,
+      "step": 61
+    },
+    {
+      "epoch": 0.1835677276091784,
+      "grad_norm": 0.9412537217140198,
+      "learning_rate": 6.2e-05,
+      "loss": 2.0901,
+      "step": 62
+    },
+    {
+      "epoch": 0.18652849740932642,
+      "grad_norm": 0.8289198875427246,
+      "learning_rate": 6.3e-05,
+      "loss": 1.727,
+      "step": 63
+    },
+    {
+      "epoch": 0.18948926720947445,
+      "grad_norm": 0.9061427116394043,
+      "learning_rate": 6.400000000000001e-05,
+      "loss": 2.0229,
+      "step": 64
+    },
+    {
+      "epoch": 0.19245003700962252,
+      "grad_norm": 0.8487738370895386,
+      "learning_rate": 6.500000000000001e-05,
+      "loss": 1.8997,
+      "step": 65
+    },
+    {
+      "epoch": 0.19541080680977055,
+      "grad_norm": 0.8029152750968933,
+      "learning_rate": 6.6e-05,
+      "loss": 1.7052,
+      "step": 66
+    },
+    {
+      "epoch": 0.19837157660991858,
+      "grad_norm": 0.8886892199516296,
+      "learning_rate": 6.7e-05,
+      "loss": 1.7248,
+      "step": 67
+    },
+    {
+      "epoch": 0.20133234641006661,
+      "grad_norm": 0.8168774843215942,
+      "learning_rate": 6.800000000000001e-05,
+      "loss": 1.6882,
+      "step": 68
+    },
+    {
+      "epoch": 0.20429311621021465,
+      "grad_norm": 0.85191410779953,
+      "learning_rate": 6.9e-05,
+      "loss": 1.6616,
+      "step": 69
+    },
+    {
+      "epoch": 0.20725388601036268,
+      "grad_norm": 0.8954592347145081,
+      "learning_rate": 7e-05,
+      "loss": 2.148,
+      "step": 70
+    },
+    {
+      "epoch": 0.21021465581051074,
+      "grad_norm": 0.9161210060119629,
+      "learning_rate": 7.1e-05,
+      "loss": 2.1652,
+      "step": 71
+    },
+    {
+      "epoch": 0.21317542561065878,
+      "grad_norm": 0.8220468759536743,
+      "learning_rate": 7.2e-05,
+      "loss": 1.8472,
+      "step": 72
+    },
+    {
+      "epoch": 0.2161361954108068,
+      "grad_norm": 0.9098365902900696,
+      "learning_rate": 7.3e-05,
+      "loss": 1.914,
+      "step": 73
+    },
+    {
+      "epoch": 0.21909696521095484,
+      "grad_norm": 0.9032228589057922,
+      "learning_rate": 7.4e-05,
+      "loss": 1.8387,
+      "step": 74
+    },
+    {
+      "epoch": 0.22205773501110287,
+      "grad_norm": 0.9047945141792297,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 1.9642,
+      "step": 75
+    },
+    {
+      "epoch": 0.22501850481125094,
+      "grad_norm": 0.8859308958053589,
+      "learning_rate": 7.6e-05,
+      "loss": 1.7764,
+      "step": 76
+    },
+    {
+      "epoch": 0.22797927461139897,
+      "grad_norm": 0.8051456809043884,
+      "learning_rate": 7.7e-05,
+      "loss": 1.3194,
+      "step": 77
+    },
+    {
+      "epoch": 0.230940044411547,
+      "grad_norm": 0.8149760961532593,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 1.4126,
+      "step": 78
+    },
+    {
+      "epoch": 0.23390081421169504,
+      "grad_norm": 0.9568467736244202,
+      "learning_rate": 7.900000000000001e-05,
+      "loss": 1.9886,
+      "step": 79
+    },
+    {
+      "epoch": 0.23686158401184307,
+      "grad_norm": 0.7884660363197327,
+      "learning_rate": 8e-05,
+      "loss": 1.4901,
+      "step": 80
+    },
+    {
+      "epoch": 0.23982235381199113,
+      "grad_norm": 0.9412587881088257,
+      "learning_rate": 8.1e-05,
+      "loss": 1.7959,
+      "step": 81
+    },
+    {
+      "epoch": 0.24278312361213916,
+      "grad_norm": 0.9693312644958496,
+      "learning_rate": 8.2e-05,
+      "loss": 1.8617,
+      "step": 82
+    },
+    {
+      "epoch": 0.2457438934122872,
+      "grad_norm": 1.0750447511672974,
+      "learning_rate": 8.3e-05,
+      "loss": 1.71,
+      "step": 83
+    },
+    {
+      "epoch": 0.24870466321243523,
+      "grad_norm": 0.8832487463951111,
+      "learning_rate": 8.4e-05,
+      "loss": 1.8103,
+      "step": 84
+    },
+    {
+      "epoch": 0.25166543301258326,
+      "grad_norm": 0.9104954600334167,
+      "learning_rate": 8.5e-05,
+      "loss": 1.6251,
+      "step": 85
+    },
+    {
+      "epoch": 0.2546262028127313,
+      "grad_norm": 1.0365192890167236,
+      "learning_rate": 8.6e-05,
+      "loss": 1.989,
+      "step": 86
+    },
+    {
+      "epoch": 0.25758697261287933,
+      "grad_norm": 0.8882848620414734,
+      "learning_rate": 8.7e-05,
+      "loss": 1.6533,
+      "step": 87
+    },
+    {
+      "epoch": 0.26054774241302736,
+      "grad_norm": 0.9500681757926941,
+      "learning_rate": 8.800000000000001e-05,
+      "loss": 2.0243,
+      "step": 88
+    },
+    {
+      "epoch": 0.26350851221317545,
+      "grad_norm": 1.2127057313919067,
+      "learning_rate": 8.900000000000001e-05,
+      "loss": 2.7702,
+      "step": 89
+    },
+    {
+      "epoch": 0.2664692820133235,
+      "grad_norm": 0.7713281512260437,
+      "learning_rate": 9e-05,
+      "loss": 0.9571,
+      "step": 90
+    },
+    {
+      "epoch": 0.2694300518134715,
+      "grad_norm": 0.6999084949493408,
+      "learning_rate": 9.1e-05,
+      "loss": 0.6233,
+      "step": 91
+    },
+    {
+      "epoch": 0.27239082161361955,
+      "grad_norm": 0.7298312187194824,
+      "learning_rate": 9.200000000000001e-05,
+      "loss": 0.5504,
+      "step": 92
+    },
+    {
+      "epoch": 0.2753515914137676,
+      "grad_norm": 0.7176688313484192,
+      "learning_rate": 9.300000000000001e-05,
+      "loss": 0.5124,
+      "step": 93
+    },
+    {
+      "epoch": 0.2783123612139156,
+      "grad_norm": 0.5131226778030396,
+      "learning_rate": 9.4e-05,
+      "loss": 0.27,
+      "step": 94
+    },
+    {
+      "epoch": 0.28127313101406365,
+      "grad_norm": 0.6256267428398132,
+      "learning_rate": 9.5e-05,
+      "loss": 0.3631,
+      "step": 95
+    },
+    {
+      "epoch": 0.2842339008142117,
+      "grad_norm": 0.7403827905654907,
+      "learning_rate": 9.6e-05,
+      "loss": 0.2167,
+      "step": 96
+    },
+    {
+      "epoch": 0.2871946706143597,
+      "grad_norm": 0.6680935621261597,
+      "learning_rate": 9.7e-05,
+      "loss": 0.189,
+      "step": 97
+    },
+    {
+      "epoch": 0.29015544041450775,
+      "grad_norm": 0.8044579029083252,
+      "learning_rate": 9.8e-05,
+      "loss": 0.152,
+      "step": 98
+    },
+    {
+      "epoch": 0.29311621021465584,
+      "grad_norm": 1.1157574653625488,
+      "learning_rate": 9.900000000000001e-05,
+      "loss": 0.4304,
+      "step": 99
+    },
+    {
+      "epoch": 0.29607698001480387,
+      "grad_norm": 2.423006772994995,
+      "learning_rate": 0.0001,
+      "loss": 0.7202,
+      "step": 100
+    },
+    {
+      "epoch": 0.2990377498149519,
+      "grad_norm": 1.0296176671981812,
+      "learning_rate": 9.999560724782174e-05,
+      "loss": 1.2209,
+      "step": 101
+    },
+    {
+      "epoch": 0.30199851961509994,
+      "grad_norm": 1.4382774829864502,
+      "learning_rate": 9.998242976313776e-05,
+      "loss": 1.9754,
+      "step": 102
+    },
+    {
+      "epoch": 0.30495928941524797,
+      "grad_norm": 1.3792943954467773,
+      "learning_rate": 9.996046986136509e-05,
+      "loss": 2.0047,
+      "step": 103
+    },
+    {
+      "epoch": 0.307920059215396,
+      "grad_norm": 1.2752324342727661,
+      "learning_rate": 9.992973140107997e-05,
+      "loss": 2.0791,
+      "step": 104
+    },
+    {
+      "epoch": 0.31088082901554404,
+      "grad_norm": 1.1318740844726562,
+      "learning_rate": 9.989021978333995e-05,
+      "loss": 2.1738,
+      "step": 105
+    },
+    {
+      "epoch": 0.31384159881569207,
+      "grad_norm": 1.031514286994934,
+      "learning_rate": 9.98419419507348e-05,
+      "loss": 1.6898,
+      "step": 106
+    },
+    {
+      "epoch": 0.3168023686158401,
+      "grad_norm": 0.9624975919723511,
+      "learning_rate": 9.97849063861667e-05,
+      "loss": 1.9637,
+      "step": 107
+    },
+    {
+      "epoch": 0.31976313841598814,
+      "grad_norm": 1.0019279718399048,
+      "learning_rate": 9.971912311135967e-05,
+      "loss": 1.8575,
+      "step": 108
+    },
+    {
+      "epoch": 0.32272390821613617,
+      "grad_norm": 0.8692964315414429,
+      "learning_rate": 9.964460368509867e-05,
+      "loss": 1.3659,
+      "step": 109
+    },
+    {
+      "epoch": 0.32568467801628426,
+      "grad_norm": 0.8809174299240112,
+      "learning_rate": 9.956136120119858e-05,
+      "loss": 1.3946,
+      "step": 110
+    },
+    {
+      "epoch": 0.3286454478164323,
+      "grad_norm": 0.960116446018219,
+      "learning_rate": 9.94694102862035e-05,
+      "loss": 1.8712,
+      "step": 111
+    },
+    {
+      "epoch": 0.3316062176165803,
+      "grad_norm": 0.9009029269218445,
+      "learning_rate": 9.936876709681668e-05,
+      "loss": 1.687,
+      "step": 112
+    },
+    {
+      "epoch": 0.33456698741672836,
+      "grad_norm": 0.8052501082420349,
+      "learning_rate": 9.925944931706173e-05,
+      "loss": 1.3864,
+      "step": 113
+    },
+    {
+      "epoch": 0.3375277572168764,
+      "grad_norm": 1.0476393699645996,
+      "learning_rate": 9.914147615517526e-05,
+      "loss": 2.2286,
+      "step": 114
+    },
+    {
+      "epoch": 0.3404885270170244,
+      "grad_norm": 1.0064458847045898,
+      "learning_rate": 9.901486834023182e-05,
+      "loss": 2.1631,
+      "step": 115
+    },
+    {
+      "epoch": 0.34344929681717246,
+      "grad_norm": 0.9857942461967468,
+      "learning_rate": 9.887964811850159e-05,
+      "loss": 1.7814,
+      "step": 116
+    },
+    {
+      "epoch": 0.3464100666173205,
+      "grad_norm": 0.9749800562858582,
+      "learning_rate": 9.873583924954152e-05,
+      "loss": 2.0465,
+      "step": 117
+    },
+    {
+      "epoch": 0.3493708364174685,
+      "grad_norm": 1.1256530284881592,
+      "learning_rate": 9.85834670020205e-05,
+      "loss": 2.3322,
+      "step": 118
+    },
+    {
+      "epoch": 0.35233160621761656,
+      "grad_norm": 0.9596055150032043,
+      "learning_rate": 9.842255814927944e-05,
+      "loss": 2.009,
+      "step": 119
+    },
+    {
+      "epoch": 0.35529237601776464,
+      "grad_norm": 0.9318323135375977,
+      "learning_rate": 9.825314096462685e-05,
+      "loss": 1.5815,
+      "step": 120
+    },
+    {
+      "epoch": 0.3582531458179127,
+      "grad_norm": 0.8369913101196289,
+      "learning_rate": 9.807524521637102e-05,
+      "loss": 1.4723,
+      "step": 121
+    },
+    {
+      "epoch": 0.3612139156180607,
+      "grad_norm": 0.90513676404953,
+      "learning_rate": 9.788890216258939e-05,
+      "loss": 1.6677,
+      "step": 122
+    },
+    {
+      "epoch": 0.36417468541820874,
+      "grad_norm": 0.9872686862945557,
+      "learning_rate": 9.769414454563615e-05,
+      "loss": 2.1453,
+      "step": 123
+    },
+    {
+      "epoch": 0.3671354552183568,
+      "grad_norm": 1.007397174835205,
+      "learning_rate": 9.749100658638914e-05,
+      "loss": 1.7057,
+      "step": 124
+    },
+    {
+      "epoch": 0.3700962250185048,
+      "grad_norm": 0.8887436985969543,
+      "learning_rate": 9.72795239782369e-05,
+      "loss": 1.7311,
+      "step": 125
+    },
+    {
+      "epoch": 0.37305699481865284,
+      "grad_norm": 1.100716471672058,
+      "learning_rate": 9.705973388080693e-05,
+      "loss": 2.1422,
+      "step": 126
+    },
+    {
+      "epoch": 0.3760177646188009,
+      "grad_norm": 0.9766169190406799,
+      "learning_rate": 9.68316749134364e-05,
+      "loss": 1.742,
+      "step": 127
+    },
+    {
+      "epoch": 0.3789785344189489,
+      "grad_norm": 1.0313608646392822,
+      "learning_rate": 9.659538714838634e-05,
+      "loss": 2.2097,
+      "step": 128
+    },
+    {
+      "epoch": 0.38193930421909694,
+      "grad_norm": 1.1221710443496704,
+      "learning_rate": 9.635091210380051e-05,
+      "loss": 1.7041,
+      "step": 129
+    },
+    {
+      "epoch": 0.38490007401924503,
+      "grad_norm": 1.0365699529647827,
+      "learning_rate": 9.609829273641034e-05,
+      "loss": 2.5044,
+      "step": 130
+    },
+    {
+      "epoch": 0.38786084381939306,
+      "grad_norm": 1.0537230968475342,
+      "learning_rate": 9.583757343398684e-05,
+      "loss": 1.8424,
+      "step": 131
+    },
+    {
+      "epoch": 0.3908216136195411,
+      "grad_norm": 1.0906157493591309,
+      "learning_rate": 9.55688000075414e-05,
+      "loss": 2.0675,
+      "step": 132
+    },
+    {
+      "epoch": 0.39378238341968913,
+      "grad_norm": 1.0615696907043457,
+      "learning_rate": 9.529201968327616e-05,
+      "loss": 2.3767,
+      "step": 133
+    },
+    {
+      "epoch": 0.39674315321983716,
+      "grad_norm": 1.1298242807388306,
+      "learning_rate": 9.500728109428604e-05,
+      "loss": 1.9942,
+      "step": 134
+    },
+    {
+      "epoch": 0.3997039230199852,
+      "grad_norm": 0.9128508567810059,
+      "learning_rate": 9.47146342720133e-05,
+      "loss": 1.71,
+      "step": 135
+    },
+    {
+      "epoch": 0.40266469282013323,
+      "grad_norm": 0.9869701266288757,
+      "learning_rate": 9.44141306374566e-05,
+      "loss": 1.7858,
+      "step": 136
+    },
+    {
+      "epoch": 0.40562546262028126,
+      "grad_norm": 0.9508257508277893,
+      "learning_rate": 9.410582299213573e-05,
+      "loss": 1.6292,
+      "step": 137
+    },
+    {
+      "epoch": 0.4085862324204293,
+      "grad_norm": 0.868828535079956,
+      "learning_rate": 9.378976550881394e-05,
+      "loss": 1.2036,
+      "step": 138
+    },
+    {
+      "epoch": 0.41154700222057733,
+      "grad_norm": 0.9286262392997742,
+      "learning_rate": 9.346601372197914e-05,
+      "loss": 1.5908,
+      "step": 139
+    },
+    {
+      "epoch": 0.41450777202072536,
+      "grad_norm": 0.8766072988510132,
+      "learning_rate": 9.3134624518086e-05,
+      "loss": 0.782,
+      "step": 140
+    },
+    {
+      "epoch": 0.41746854182087345,
+      "grad_norm": 0.8357656598091125,
+      "learning_rate": 9.279565612556043e-05,
+      "loss": 0.4964,
+      "step": 141
+    },
+    {
+      "epoch": 0.4204293116210215,
+      "grad_norm": 0.5693497657775879,
+      "learning_rate": 9.244916810456821e-05,
+      "loss": 0.2504,
+      "step": 142
+    },
+    {
+      "epoch": 0.4233900814211695,
+      "grad_norm": 0.5122264623641968,
+      "learning_rate": 9.209522133654969e-05,
+      "loss": 0.3069,
+      "step": 143
+    },
+    {
+      "epoch": 0.42635085122131755,
+      "grad_norm": 0.5361589193344116,
+      "learning_rate": 9.173387801352232e-05,
+      "loss": 0.2766,
+      "step": 144
+    },
+    {
+      "epoch": 0.4293116210214656,
+      "grad_norm": 0.6745699644088745,
+      "learning_rate": 9.136520162715287e-05,
+      "loss": 0.4646,
+      "step": 145
+    },
+    {
+      "epoch": 0.4322723908216136,
+      "grad_norm": 0.5727047920227051,
+      "learning_rate": 9.098925695760132e-05,
+      "loss": 0.1874,
+      "step": 146
+    },
+    {
+      "epoch": 0.43523316062176165,
+      "grad_norm": 0.5952191948890686,
+      "learning_rate": 9.060611006213832e-05,
+      "loss": 0.1782,
+      "step": 147
+    },
+    {
+      "epoch": 0.4381939304219097,
+      "grad_norm": 0.6233859062194824,
+      "learning_rate": 9.021582826353824e-05,
+      "loss": 0.1705,
+      "step": 148
+    },
+    {
+      "epoch": 0.4411547002220577,
+      "grad_norm": 0.6710546016693115,
+      "learning_rate": 8.981848013824993e-05,
+      "loss": 0.1251,
+      "step": 149
+    },
+    {
+      "epoch": 0.44411547002220575,
+      "grad_norm": 1.7101366519927979,
+      "learning_rate": 8.94141355043471e-05,
+      "loss": 0.4686,
+      "step": 150
+    },
+    {
+      "epoch": 0.44707623982235384,
+      "grad_norm": 1.2731906175613403,
+      "learning_rate": 8.900286540926061e-05,
+      "loss": 1.1952,
+      "step": 151
+    },
+    {
+      "epoch": 0.45003700962250187,
+      "grad_norm": 1.3987884521484375,
+      "learning_rate": 8.858474211729469e-05,
+      "loss": 1.7733,
+      "step": 152
+    },
+    {
+      "epoch": 0.4529977794226499,
+      "grad_norm": 1.2485826015472412,
+      "learning_rate": 8.815983909692943e-05,
+      "loss": 2.0824,
+      "step": 153
+    },
+    {
+      "epoch": 0.45595854922279794,
+      "grad_norm": 1.1472845077514648,
+      "learning_rate": 8.772823100791151e-05,
+      "loss": 1.7076,
+      "step": 154
+    },
+    {
+      "epoch": 0.45891931902294597,
+      "grad_norm": 0.9043837189674377,
+      "learning_rate": 8.728999368813591e-05,
+      "loss": 1.5772,
+      "step": 155
+    },
+    {
+      "epoch": 0.461880088823094,
+      "grad_norm": 1.1139400005340576,
+      "learning_rate": 8.684520414032023e-05,
+      "loss": 1.7797,
+      "step": 156
+    },
+    {
+      "epoch": 0.46484085862324204,
+      "grad_norm": 0.8343661427497864,
+      "learning_rate": 8.639394051847472e-05,
+      "loss": 1.6327,
+      "step": 157
+    },
+    {
+      "epoch": 0.46780162842339007,
+      "grad_norm": 0.9228811860084534,
+      "learning_rate": 8.593628211416964e-05,
+      "loss": 1.3159,
+      "step": 158
+    },
+    {
+      "epoch": 0.4707623982235381,
+      "grad_norm": 1.0521384477615356,
+      "learning_rate": 8.547230934260311e-05,
+      "loss": 1.937,
+      "step": 159
+    },
+    {
+      "epoch": 0.47372316802368614,
+      "grad_norm": 1.0546250343322754,
+      "learning_rate": 8.500210372847127e-05,
+      "loss": 1.5866,
+      "step": 160
+    },
+    {
+      "epoch": 0.47668393782383417,
+      "grad_norm": 0.9808539152145386,
+      "learning_rate": 8.452574789164351e-05,
+      "loss": 1.3465,
+      "step": 161
+    },
+    {
+      "epoch": 0.47964470762398226,
+      "grad_norm": 1.0605146884918213,
+      "learning_rate": 8.404332553264547e-05,
+      "loss": 1.9493,
+      "step": 162
+    },
+    {
+      "epoch": 0.4826054774241303,
+      "grad_norm": 1.044192910194397,
+      "learning_rate": 8.355492141795185e-05,
+      "loss": 1.7087,
+      "step": 163
+    },
+    {
+      "epoch": 0.4855662472242783,
+      "grad_norm": 1.0825512409210205,
+      "learning_rate": 8.30606213650922e-05,
+      "loss": 1.7466,
+      "step": 164
+    },
+    {
+      "epoch": 0.48852701702442636,
+      "grad_norm": 1.1549116373062134,
+      "learning_rate": 8.256051222757188e-05,
+      "loss": 1.9754,
+      "step": 165
+    },
+    {
+      "epoch": 0.4914877868245744,
+      "grad_norm": 0.9423612952232361,
+      "learning_rate": 8.2054681879611e-05,
+      "loss": 1.8689,
+      "step": 166
+    },
+    {
+      "epoch": 0.4944485566247224,
+      "grad_norm": 0.9141284823417664,
+      "learning_rate": 8.154321920070414e-05,
+      "loss": 1.3927,
+      "step": 167
+    },
+    {
+      "epoch": 0.49740932642487046,
+      "grad_norm": 1.1286426782608032,
+      "learning_rate": 8.102621406000309e-05,
+      "loss": 1.9011,
+      "step": 168
+    },
+    {
+      "epoch": 0.5003700962250185,
+      "grad_norm": 1.188552737236023,
+      "learning_rate": 8.050375730052621e-05,
+      "loss": 2.5164,
+      "step": 169
+    },
+    {
+      "epoch": 0.5033308660251665,
+      "grad_norm": 0.8950062990188599,
+      "learning_rate": 7.997594072319625e-05,
+      "loss": 1.2163,
+      "step": 170
+    },
+    {
+      "epoch": 0.5062916358253146,
+      "grad_norm": 0.8495607972145081,
+      "learning_rate": 7.944285707070998e-05,
+      "loss": 1.3757,
+      "step": 171
+    },
+    {
+      "epoch": 0.5092524056254626,
+      "grad_norm": 0.9846630096435547,
+      "learning_rate": 7.890460001124242e-05,
+      "loss": 1.3396,
+      "step": 172
+    },
+    {
+      "epoch": 0.5122131754256106,
+      "grad_norm": 1.158858299255371,
+      "learning_rate": 7.83612641219884e-05,
+      "loss": 1.8067,
+      "step": 173
+    },
+    {
+      "epoch": 0.5151739452257587,
+      "grad_norm": 0.9049079418182373,
+      "learning_rate": 7.781294487254436e-05,
+      "loss": 1.3725,
+      "step": 174
+    },
+    {
+      "epoch": 0.5181347150259067,
+      "grad_norm": 1.169640064239502,
+      "learning_rate": 7.725973860813338e-05,
+      "loss": 2.5245,
+      "step": 175
+    },
+    {
+      "epoch": 0.5210954848260547,
+      "grad_norm": 1.0044087171554565,
+      "learning_rate": 7.670174253267641e-05,
+      "loss": 1.7303,
+      "step": 176
+    },
+    {
+      "epoch": 0.5240562546262029,
+      "grad_norm": 1.306775689125061,
+      "learning_rate": 7.613905469171246e-05,
+      "loss": 2.2293,
+      "step": 177
+    },
+    {
+      "epoch": 0.5270170244263509,
+      "grad_norm": 0.8953151702880859,
+      "learning_rate": 7.557177395517112e-05,
+      "loss": 1.4446,
+      "step": 178
+    },
+    {
+      "epoch": 0.5299777942264989,
+      "grad_norm": 0.8270052671432495,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 1.2409,
+      "step": 179
+    },
+    {
+      "epoch": 0.532938564026647,
+      "grad_norm": 1.06266450881958,
+      "learning_rate": 7.442383329265062e-05,
+      "loss": 1.726,
+      "step": 180
+    },
+    {
+      "epoch": 0.535899333826795,
+      "grad_norm": 1.1915879249572754,
+      "learning_rate": 7.384337507142531e-05,
+      "loss": 1.9542,
+      "step": 181
+    },
+    {
+      "epoch": 0.538860103626943,
+      "grad_norm": 1.1052125692367554,
+      "learning_rate": 7.325872732868869e-05,
+      "loss": 1.804,
+      "step": 182
+    },
+    {
+      "epoch": 0.5418208734270911,
+      "grad_norm": 0.9792408347129822,
+      "learning_rate": 7.26699927929466e-05,
+      "loss": 1.3709,
+      "step": 183
+    },
+    {
+      "epoch": 0.5447816432272391,
+      "grad_norm": 1.0242524147033691,
+      "learning_rate": 7.20772749107956e-05,
+      "loss": 1.4006,
+      "step": 184
+    },
+    {
+      "epoch": 0.5477424130273871,
+      "grad_norm": 0.9607504606246948,
+      "learning_rate": 7.14806778287464e-05,
+      "loss": 1.6155,
+      "step": 185
+    },
+    {
+      "epoch": 0.5507031828275352,
+      "grad_norm": 0.9280562400817871,
+      "learning_rate": 7.088030637492429e-05,
+      "loss": 1.7849,
+      "step": 186
+    },
+    {
+      "epoch": 0.5536639526276832,
+      "grad_norm": 1.276000738143921,
+      "learning_rate": 7.027626604064969e-05,
+      "loss": 2.5948,
+      "step": 187
+    },
+    {
+      "epoch": 0.5566247224278312,
+      "grad_norm": 1.0355356931686401,
+      "learning_rate": 6.966866296190241e-05,
+      "loss": 1.9808,
+      "step": 188
+    },
+    {
+      "epoch": 0.5595854922279793,
+      "grad_norm": 1.275265097618103,
+      "learning_rate": 6.905760390067235e-05,
+      "loss": 2.0494,
+      "step": 189
+    },
+    {
+      "epoch": 0.5625462620281273,
+      "grad_norm": 0.6706751585006714,
+      "learning_rate": 6.844319622620039e-05,
+      "loss": 0.603,
+      "step": 190
+    },
+    {
+      "epoch": 0.5655070318282753,
+      "grad_norm": 0.6546670794487,
+      "learning_rate": 6.782554789611255e-05,
+      "loss": 0.4406,
+      "step": 191
+    },
+    {
+      "epoch": 0.5684678016284234,
+      "grad_norm": 0.6711502075195312,
+      "learning_rate": 6.720476743745072e-05,
+      "loss": 0.6604,
+      "step": 192
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.6742061972618103,
+      "learning_rate": 6.65809639276034e-05,
+      "loss": 0.4936,
+      "step": 193
+    },
+    {
+      "epoch": 0.5743893412287194,
+      "grad_norm": 0.6132559776306152,
+      "learning_rate": 6.595424697513964e-05,
+      "loss": 0.4007,
+      "step": 194
+    },
+    {
+      "epoch": 0.5773501110288675,
+      "grad_norm": 0.5519651174545288,
+      "learning_rate": 6.532472670054974e-05,
+      "loss": 0.3362,
+      "step": 195
+    },
+    {
+      "epoch": 0.5803108808290155,
+      "grad_norm": 0.6012782454490662,
+      "learning_rate": 6.469251371689606e-05,
+      "loss": 0.2836,
+      "step": 196
+    },
+    {
+      "epoch": 0.5832716506291635,
+      "grad_norm": 1.4975916147232056,
+      "learning_rate": 6.405771911037699e-05,
+      "loss": 0.1957,
+      "step": 197
+    },
+    {
+      "epoch": 0.5862324204293117,
+      "grad_norm": 1.4344160556793213,
+      "learning_rate": 6.342045442080818e-05,
+      "loss": 0.3677,
+      "step": 198
+    },
+    {
+      "epoch": 0.5891931902294597,
+      "grad_norm": 0.9004579782485962,
+      "learning_rate": 6.278083162202375e-05,
+      "loss": 0.122,
+      "step": 199
+    },
+    {
+      "epoch": 0.5921539600296077,
+      "grad_norm": 1.5413613319396973,
+      "learning_rate": 6.213896310220139e-05,
+      "loss": 0.3552,
+      "step": 200
+    },
+    {
+      "epoch": 0.5951147298297558,
+      "grad_norm": 1.3081581592559814,
+      "learning_rate": 6.149496164411468e-05,
+      "loss": 2.3571,
+      "step": 201
+    },
+    {
+      "epoch": 0.5980754996299038,
+      "grad_norm": 1.5233243703842163,
+      "learning_rate": 6.08489404053159e-05,
+      "loss": 1.7568,
+      "step": 202
+    },
+    {
+      "epoch": 0.6010362694300518,
+      "grad_norm": 1.0305086374282837,
+      "learning_rate": 6.020101289825324e-05,
+      "loss": 1.2343,
+      "step": 203
+    },
+    {
+      "epoch": 0.6039970392301999,
+      "grad_norm": 1.2169106006622314,
+      "learning_rate": 5.955129297032539e-05,
+      "loss": 1.9524,
+      "step": 204
+    },
+    {
+      "epoch": 0.6069578090303479,
+      "grad_norm": 1.110613465309143,
+      "learning_rate": 5.889989478387753e-05,
+      "loss": 1.2548,
+      "step": 205
+    },
+    {
+      "epoch": 0.6099185788304959,
+      "grad_norm": 1.059578776359558,
+      "learning_rate": 5.8246932796141704e-05,
+      "loss": 1.5665,
+      "step": 206
+    },
+    {
+      "epoch": 0.612879348630644,
+      "grad_norm": 1.1399540901184082,
+      "learning_rate": 5.7592521739125726e-05,
+      "loss": 1.5839,
+      "step": 207
+    },
+    {
+      "epoch": 0.615840118430792,
+      "grad_norm": 1.1892155408859253,
+      "learning_rate": 5.6936776599453424e-05,
+      "loss": 1.9225,
+      "step": 208
+    },
+    {
+      "epoch": 0.61880088823094,
+      "grad_norm": 1.2208551168441772,
+      "learning_rate": 5.6279812598160406e-05,
+      "loss": 1.7876,
+      "step": 209
+    },
+    {
+      "epoch": 0.6217616580310881,
+      "grad_norm": 0.8801209926605225,
+      "learning_rate": 5.5621745170448616e-05,
+      "loss": 1.4153,
+      "step": 210
+    },
+    {
+      "epoch": 0.6247224278312361,
+      "grad_norm": 1.2946308851242065,
+      "learning_rate": 5.496268994540309e-05,
+      "loss": 2.2272,
+      "step": 211
+    },
+    {
+      "epoch": 0.6276831976313841,
+      "grad_norm": 1.1825288534164429,
+      "learning_rate": 5.430276272567485e-05,
+      "loss": 2.3467,
+      "step": 212
+    },
+    {
+      "epoch": 0.6306439674315322,
+      "grad_norm": 0.972135603427887,
+      "learning_rate": 5.364207946713318e-05,
+      "loss": 1.6491,
+      "step": 213
+    },
+    {
+      "epoch": 0.6336047372316802,
+      "grad_norm": 1.0048394203186035,
+      "learning_rate": 5.2980756258490995e-05,
+      "loss": 1.7103,
+      "step": 214
+    },
+    {
+      "epoch": 0.6365655070318282,
+      "grad_norm": 1.0976014137268066,
+      "learning_rate": 5.2318909300906926e-05,
+      "loss": 1.6622,
+      "step": 215
+    },
+    {
+      "epoch": 0.6395262768319763,
+      "grad_norm": 1.0213438272476196,
+      "learning_rate": 5.165665488756754e-05,
+      "loss": 2.013,
+      "step": 216
+    },
+    {
+      "epoch": 0.6424870466321243,
+      "grad_norm": 1.091571569442749,
+      "learning_rate": 5.0994109383253506e-05,
+      "loss": 1.7667,
+      "step": 217
+    },
+    {
+      "epoch": 0.6454478164322723,
+      "grad_norm": 1.0388829708099365,
+      "learning_rate": 5.033138920389313e-05,
+      "loss": 1.8158,
+      "step": 218
+    },
+    {
+      "epoch": 0.6484085862324205,
+      "grad_norm": 1.0053703784942627,
+      "learning_rate": 4.966861079610688e-05,
+      "loss": 1.499,
+      "step": 219
+    },
+    {
+      "epoch": 0.6513693560325685,
+      "grad_norm": 1.0581315755844116,
+      "learning_rate": 4.900589061674649e-05,
+      "loss": 2.2098,
+      "step": 220
+    },
+    {
+      "epoch": 0.6543301258327165,
+      "grad_norm": 1.1170649528503418,
+      "learning_rate": 4.834334511243247e-05,
+      "loss": 2.0526,
+      "step": 221
+    },
+    {
+      "epoch": 0.6572908956328646,
+      "grad_norm": 1.1198148727416992,
+      "learning_rate": 4.768109069909307e-05,
+      "loss": 1.6998,
+      "step": 222
+    },
+    {
+      "epoch": 0.6602516654330126,
+      "grad_norm": 1.127437949180603,
+      "learning_rate": 4.701924374150901e-05,
+      "loss": 1.7269,
+      "step": 223
+    },
+    {
+      "epoch": 0.6632124352331606,
+      "grad_norm": 0.8806882500648499,
+      "learning_rate": 4.635792053286682e-05,
+      "loss": 1.4307,
+      "step": 224
+    },
+    {
+      "epoch": 0.6661732050333087,
+      "grad_norm": 1.0180091857910156,
+      "learning_rate": 4.569723727432517e-05,
+      "loss": 1.8816,
+      "step": 225
+    },
+    {
+      "epoch": 0.6691339748334567,
+      "grad_norm": 0.8961617350578308,
+      "learning_rate": 4.503731005459693e-05,
+      "loss": 1.2071,
+      "step": 226
+    },
+    {
+      "epoch": 0.6720947446336047,
+      "grad_norm": 1.143943428993225,
+      "learning_rate": 4.4378254829551396e-05,
+      "loss": 2.0122,
+      "step": 227
+    },
+    {
+      "epoch": 0.6750555144337528,
+      "grad_norm": 1.0135809183120728,
+      "learning_rate": 4.3720187401839606e-05,
+      "loss": 1.5634,
+      "step": 228
+    },
+    {
+      "epoch": 0.6780162842339008,
+      "grad_norm": 1.0653568506240845,
+      "learning_rate": 4.3063223400546594e-05,
+      "loss": 1.4539,
+      "step": 229
+    },
+    {
+      "epoch": 0.6809770540340488,
+      "grad_norm": 1.0943018198013306,
+      "learning_rate": 4.240747826087429e-05,
+      "loss": 1.7096,
+      "step": 230
+    },
+    {
+      "epoch": 0.6839378238341969,
+      "grad_norm": 1.0082052946090698,
+      "learning_rate": 4.17530672038583e-05,
+      "loss": 1.8474,
+      "step": 231
+    },
+    {
+      "epoch": 0.6868985936343449,
+      "grad_norm": 1.1438117027282715,
+      "learning_rate": 4.11001052161225e-05,
+      "loss": 1.6998,
+      "step": 232
+    },
+    {
+      "epoch": 0.689859363434493,
+      "grad_norm": 1.0718982219696045,
+      "learning_rate": 4.044870702967461e-05,
+      "loss": 1.786,
+      "step": 233
+    },
+    {
+      "epoch": 0.692820133234641,
+      "grad_norm": 1.172393798828125,
+      "learning_rate": 3.979898710174678e-05,
+      "loss": 1.9927,
+      "step": 234
+    },
+    {
+      "epoch": 0.695780903034789,
+      "grad_norm": 1.0030678510665894,
+      "learning_rate": 3.91510595946841e-05,
+      "loss": 1.5687,
+      "step": 235
+    },
+    {
+      "epoch": 0.698741672834937,
+      "grad_norm": 0.8714839816093445,
+      "learning_rate": 3.850503835588533e-05,
+      "loss": 1.5528,
+      "step": 236
+    },
+    {
+      "epoch": 0.7017024426350851,
+      "grad_norm": 0.7779592275619507,
+      "learning_rate": 3.786103689779861e-05,
+      "loss": 1.1243,
+      "step": 237
+    },
+    {
+      "epoch": 0.7046632124352331,
+      "grad_norm": 0.94936203956604,
+      "learning_rate": 3.721916837797627e-05,
+      "loss": 1.5304,
+      "step": 238
+    },
+    {
+      "epoch": 0.7076239822353811,
+      "grad_norm": 0.945186197757721,
+      "learning_rate": 3.657954557919183e-05,
+      "loss": 1.6068,
+      "step": 239
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 337,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 239,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.142671224386355e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1194f4dd5944ae1d681b7014780eb2503f81b61c68953abb392fd6893434def
+size 6840