Training in progress, step 239, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "embed_tokens",
+    "lm_head"
+  ],
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "q_proj",
+    "v_proj",
+    "down_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1024a5776d3f5a42aebe843c45b5f3f3dfa384ab03237422393292b6fcb7279f
+size 3664864144

last-checkpoint/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:003499bb43c338c5f3f89638f76647567ba1fa652de51adae55a59f4cf8ab74b
+size 7330126728

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc890496f0599c1204751da5a8f13f48c5bfb73a4a934cd75f31b1006c74831f
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36a8d1616a218e1bcbcdcf7a4b744bb28fa686454a4589cf663b78e3046a8dbb
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1706 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.1546925566343042,
+  "eval_steps": 500,
+  "global_step": 239,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0006472491909385113,
+      "grad_norm": 131.13400268554688,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 19.7936,
+      "step": 1
+    },
+    {
+      "epoch": 0.0012944983818770227,
+      "grad_norm": 134.03347778320312,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 19.9265,
+      "step": 2
+    },
+    {
+      "epoch": 0.001941747572815534,
+      "grad_norm": 135.04904174804688,
+      "learning_rate": 3e-06,
+      "loss": 19.6922,
+      "step": 3
+    },
+    {
+      "epoch": 0.0025889967637540453,
+      "grad_norm": 133.32534790039062,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 19.9163,
+      "step": 4
+    },
+    {
+      "epoch": 0.003236245954692557,
+      "grad_norm": 133.73779296875,
+      "learning_rate": 5e-06,
+      "loss": 19.5099,
+      "step": 5
+    },
+    {
+      "epoch": 0.003883495145631068,
+      "grad_norm": 136.7259521484375,
+      "learning_rate": 6e-06,
+      "loss": 19.9571,
+      "step": 6
+    },
+    {
+      "epoch": 0.004530744336569579,
+      "grad_norm": 135.7963104248047,
+      "learning_rate": 7.000000000000001e-06,
+      "loss": 19.6701,
+      "step": 7
+    },
+    {
+      "epoch": 0.005177993527508091,
+      "grad_norm": 144.41856384277344,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 19.5384,
+      "step": 8
+    },
+    {
+      "epoch": 0.005825242718446602,
+      "grad_norm": 137.59848022460938,
+      "learning_rate": 9e-06,
+      "loss": 19.4975,
+      "step": 9
+    },
+    {
+      "epoch": 0.006472491909385114,
+      "grad_norm": 137.11566162109375,
+      "learning_rate": 1e-05,
+      "loss": 19.1857,
+      "step": 10
+    },
+    {
+      "epoch": 0.007119741100323625,
+      "grad_norm": 140.35537719726562,
+      "learning_rate": 1.1000000000000001e-05,
+      "loss": 17.4217,
+      "step": 11
+    },
+    {
+      "epoch": 0.007766990291262136,
+      "grad_norm": 136.8153533935547,
+      "learning_rate": 1.2e-05,
+      "loss": 16.8234,
+      "step": 12
+    },
+    {
+      "epoch": 0.008414239482200648,
+      "grad_norm": 133.3092041015625,
+      "learning_rate": 1.3000000000000001e-05,
+      "loss": 15.1626,
+      "step": 13
+    },
+    {
+      "epoch": 0.009061488673139158,
+      "grad_norm": 129.0071258544922,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 14.1965,
+      "step": 14
+    },
+    {
+      "epoch": 0.009708737864077669,
+      "grad_norm": 126.3600082397461,
+      "learning_rate": 1.5e-05,
+      "loss": 12.8533,
+      "step": 15
+    },
+    {
+      "epoch": 0.010355987055016181,
+      "grad_norm": 124.45155334472656,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 11.6695,
+      "step": 16
+    },
+    {
+      "epoch": 0.011003236245954692,
+      "grad_norm": 121.65681457519531,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 10.6721,
+      "step": 17
+    },
+    {
+      "epoch": 0.011650485436893204,
+      "grad_norm": 129.11764526367188,
+      "learning_rate": 1.8e-05,
+      "loss": 9.8295,
+      "step": 18
+    },
+    {
+      "epoch": 0.012297734627831715,
+      "grad_norm": 130.87242126464844,
+      "learning_rate": 1.9e-05,
+      "loss": 9.0234,
+      "step": 19
+    },
+    {
+      "epoch": 0.012944983818770227,
+      "grad_norm": 132.683349609375,
+      "learning_rate": 2e-05,
+      "loss": 7.5961,
+      "step": 20
+    },
+    {
+      "epoch": 0.013592233009708738,
+      "grad_norm": 125.61486053466797,
+      "learning_rate": 2.1e-05,
+      "loss": 6.1994,
+      "step": 21
+    },
+    {
+      "epoch": 0.01423948220064725,
+      "grad_norm": 110.60992431640625,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 4.9317,
+      "step": 22
+    },
+    {
+      "epoch": 0.01488673139158576,
+      "grad_norm": 104.66587829589844,
+      "learning_rate": 2.3000000000000003e-05,
+      "loss": 3.7229,
+      "step": 23
+    },
+    {
+      "epoch": 0.015533980582524271,
+      "grad_norm": 100.7620620727539,
+      "learning_rate": 2.4e-05,
+      "loss": 2.6222,
+      "step": 24
+    },
+    {
+      "epoch": 0.016181229773462782,
+      "grad_norm": 89.32363891601562,
+      "learning_rate": 2.5e-05,
+      "loss": 1.7139,
+      "step": 25
+    },
+    {
+      "epoch": 0.016828478964401296,
+      "grad_norm": 76.68353271484375,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 1.2252,
+      "step": 26
+    },
+    {
+      "epoch": 0.017475728155339806,
+      "grad_norm": 65.069580078125,
+      "learning_rate": 2.7000000000000002e-05,
+      "loss": 0.8928,
+      "step": 27
+    },
+    {
+      "epoch": 0.018122977346278317,
+      "grad_norm": 60.632843017578125,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 0.7026,
+      "step": 28
+    },
+    {
+      "epoch": 0.018770226537216828,
+      "grad_norm": 55.2016487121582,
+      "learning_rate": 2.9e-05,
+      "loss": 0.473,
+      "step": 29
+    },
+    {
+      "epoch": 0.019417475728155338,
+      "grad_norm": 39.60809326171875,
+      "learning_rate": 3e-05,
+      "loss": 0.2482,
+      "step": 30
+    },
+    {
+      "epoch": 0.020064724919093852,
+      "grad_norm": 22.28648567199707,
+      "learning_rate": 3.1e-05,
+      "loss": 0.1241,
+      "step": 31
+    },
+    {
+      "epoch": 0.020711974110032363,
+      "grad_norm": 13.423832893371582,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 0.0634,
+      "step": 32
+    },
+    {
+      "epoch": 0.021359223300970873,
+      "grad_norm": 3.592240810394287,
+      "learning_rate": 3.3e-05,
+      "loss": 0.0174,
+      "step": 33
+    },
+    {
+      "epoch": 0.022006472491909384,
+      "grad_norm": 2.819695472717285,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 0.0127,
+      "step": 34
+    },
+    {
+      "epoch": 0.022653721682847898,
+      "grad_norm": 1.1562986373901367,
+      "learning_rate": 3.5e-05,
+      "loss": 0.0056,
+      "step": 35
+    },
+    {
+      "epoch": 0.02330097087378641,
+      "grad_norm": 0.2208443284034729,
+      "learning_rate": 3.6e-05,
+      "loss": 0.0013,
+      "step": 36
+    },
+    {
+      "epoch": 0.02394822006472492,
+      "grad_norm": 0.0697128102183342,
+      "learning_rate": 3.7e-05,
+      "loss": 0.0005,
+      "step": 37
+    },
+    {
+      "epoch": 0.02459546925566343,
+      "grad_norm": 0.04032439365983009,
+      "learning_rate": 3.8e-05,
+      "loss": 0.0003,
+      "step": 38
+    },
+    {
+      "epoch": 0.02524271844660194,
+      "grad_norm": 0.02219025231897831,
+      "learning_rate": 3.9000000000000006e-05,
+      "loss": 0.0001,
+      "step": 39
+    },
+    {
+      "epoch": 0.025889967637540454,
+      "grad_norm": 0.039339829236269,
+      "learning_rate": 4e-05,
+      "loss": 0.0002,
+      "step": 40
+    },
+    {
+      "epoch": 0.026537216828478965,
+      "grad_norm": 0.019809404388070107,
+      "learning_rate": 4.1e-05,
+      "loss": 0.0001,
+      "step": 41
+    },
+    {
+      "epoch": 0.027184466019417475,
+      "grad_norm": 1.500696063041687,
+      "learning_rate": 4.2e-05,
+      "loss": 0.005,
+      "step": 42
+    },
+    {
+      "epoch": 0.027831715210355986,
+      "grad_norm": 0.0020276184659451246,
+      "learning_rate": 4.3e-05,
+      "loss": 0.0,
+      "step": 43
+    },
+    {
+      "epoch": 0.0284789644012945,
+      "grad_norm": 0.0019223797135055065,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 0.0,
+      "step": 44
+    },
+    {
+      "epoch": 0.02912621359223301,
+      "grad_norm": 0.0024162745103240013,
+      "learning_rate": 4.5e-05,
+      "loss": 0.0,
+      "step": 45
+    },
+    {
+      "epoch": 0.02977346278317152,
+      "grad_norm": 0.002317259320989251,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 0.0,
+      "step": 46
+    },
+    {
+      "epoch": 0.030420711974110032,
+      "grad_norm": 0.002440544543787837,
+      "learning_rate": 4.7e-05,
+      "loss": 0.0,
+      "step": 47
+    },
+    {
+      "epoch": 0.031067961165048542,
+      "grad_norm": 0.0025529128033667803,
+      "learning_rate": 4.8e-05,
+      "loss": 0.0,
+      "step": 48
+    },
+    {
+      "epoch": 0.03171521035598705,
+      "grad_norm": 0.0027895700186491013,
+      "learning_rate": 4.9e-05,
+      "loss": 0.0,
+      "step": 49
+    },
+    {
+      "epoch": 0.032362459546925564,
+      "grad_norm": 0.0028908746317029,
+      "learning_rate": 5e-05,
+      "loss": 0.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.03300970873786408,
+      "grad_norm": 0.0015969317173585296,
+      "learning_rate": 5.1000000000000006e-05,
+      "loss": 0.0,
+      "step": 51
+    },
+    {
+      "epoch": 0.03365695792880259,
+      "grad_norm": 0.0016695812810212374,
+      "learning_rate": 5.2000000000000004e-05,
+      "loss": 0.0,
+      "step": 52
+    },
+    {
+      "epoch": 0.0343042071197411,
+      "grad_norm": 0.001670464058406651,
+      "learning_rate": 5.300000000000001e-05,
+      "loss": 0.0,
+      "step": 53
+    },
+    {
+      "epoch": 0.03495145631067961,
+      "grad_norm": 0.0014726085355505347,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 0.0,
+      "step": 54
+    },
+    {
+      "epoch": 0.03559870550161812,
+      "grad_norm": 0.0015096040442585945,
+      "learning_rate": 5.500000000000001e-05,
+      "loss": 0.0,
+      "step": 55
+    },
+    {
+      "epoch": 0.036245954692556634,
+      "grad_norm": 0.0015506166964769363,
+      "learning_rate": 5.6000000000000006e-05,
+      "loss": 0.0,
+      "step": 56
+    },
+    {
+      "epoch": 0.036893203883495145,
+      "grad_norm": 0.0015059434808790684,
+      "learning_rate": 5.6999999999999996e-05,
+      "loss": 0.0,
+      "step": 57
+    },
+    {
+      "epoch": 0.037540453074433655,
+      "grad_norm": 0.0015852790093049407,
+      "learning_rate": 5.8e-05,
+      "loss": 0.0,
+      "step": 58
+    },
+    {
+      "epoch": 0.038187702265372166,
+      "grad_norm": 0.0014398024650290608,
+      "learning_rate": 5.9e-05,
+      "loss": 0.0,
+      "step": 59
+    },
+    {
+      "epoch": 0.038834951456310676,
+      "grad_norm": 0.0017769384430721402,
+      "learning_rate": 6e-05,
+      "loss": 0.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.039482200647249194,
+      "grad_norm": 0.0019574002362787724,
+      "learning_rate": 6.1e-05,
+      "loss": 0.0,
+      "step": 61
+    },
+    {
+      "epoch": 0.040129449838187704,
+      "grad_norm": 0.0019588808063417673,
+      "learning_rate": 6.2e-05,
+      "loss": 0.0,
+      "step": 62
+    },
+    {
+      "epoch": 0.040776699029126215,
+      "grad_norm": 0.0017840194050222635,
+      "learning_rate": 6.3e-05,
+      "loss": 0.0,
+      "step": 63
+    },
+    {
+      "epoch": 0.041423948220064725,
+      "grad_norm": 0.001379096764139831,
+      "learning_rate": 6.400000000000001e-05,
+      "loss": 0.0,
+      "step": 64
+    },
+    {
+      "epoch": 0.042071197411003236,
+      "grad_norm": 0.0015804730355739594,
+      "learning_rate": 6.500000000000001e-05,
+      "loss": 0.0,
+      "step": 65
+    },
+    {
+      "epoch": 0.04271844660194175,
+      "grad_norm": 0.001860379590652883,
+      "learning_rate": 6.6e-05,
+      "loss": 0.0,
+      "step": 66
+    },
+    {
+      "epoch": 0.04336569579288026,
+      "grad_norm": 0.001473184092901647,
+      "learning_rate": 6.7e-05,
+      "loss": 0.0,
+      "step": 67
+    },
+    {
+      "epoch": 0.04401294498381877,
+      "grad_norm": 0.001431824523024261,
+      "learning_rate": 6.800000000000001e-05,
+      "loss": 0.0,
+      "step": 68
+    },
+    {
+      "epoch": 0.04466019417475728,
+      "grad_norm": 0.0015007014153525233,
+      "learning_rate": 6.9e-05,
+      "loss": 0.0,
+      "step": 69
+    },
+    {
+      "epoch": 0.045307443365695796,
+      "grad_norm": 0.0013901310740038753,
+      "learning_rate": 7e-05,
+      "loss": 0.0,
+      "step": 70
+    },
+    {
+      "epoch": 0.045954692556634306,
+      "grad_norm": 0.00129253591876477,
+      "learning_rate": 7.1e-05,
+      "loss": 0.0,
+      "step": 71
+    },
+    {
+      "epoch": 0.04660194174757282,
+      "grad_norm": 0.001261955825611949,
+      "learning_rate": 7.2e-05,
+      "loss": 0.0,
+      "step": 72
+    },
+    {
+      "epoch": 0.04724919093851133,
+      "grad_norm": 0.0013884572545066476,
+      "learning_rate": 7.3e-05,
+      "loss": 0.0,
+      "step": 73
+    },
+    {
+      "epoch": 0.04789644012944984,
+      "grad_norm": 0.001199840335175395,
+      "learning_rate": 7.4e-05,
+      "loss": 0.0,
+      "step": 74
+    },
+    {
+      "epoch": 0.04854368932038835,
+      "grad_norm": 0.0012201342033222318,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 0.0,
+      "step": 75
+    },
+    {
+      "epoch": 0.04919093851132686,
+      "grad_norm": 0.001058132154867053,
+      "learning_rate": 7.6e-05,
+      "loss": 0.0,
+      "step": 76
+    },
+    {
+      "epoch": 0.04983818770226537,
+      "grad_norm": 0.0009809430921450257,
+      "learning_rate": 7.7e-05,
+      "loss": 0.0,
+      "step": 77
+    },
+    {
+      "epoch": 0.05048543689320388,
+      "grad_norm": 0.000997965456917882,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 0.0,
+      "step": 78
+    },
+    {
+      "epoch": 0.0511326860841424,
+      "grad_norm": 0.0008985965396277606,
+      "learning_rate": 7.900000000000001e-05,
+      "loss": 0.0,
+      "step": 79
+    },
+    {
+      "epoch": 0.05177993527508091,
+      "grad_norm": 0.000818964617792517,
+      "learning_rate": 8e-05,
+      "loss": 0.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.05242718446601942,
+      "grad_norm": 0.0007367520011030138,
+      "learning_rate": 8.1e-05,
+      "loss": 0.0,
+      "step": 81
+    },
+    {
+      "epoch": 0.05307443365695793,
+      "grad_norm": 0.0006985078798606992,
+      "learning_rate": 8.2e-05,
+      "loss": 0.0,
+      "step": 82
+    },
+    {
+      "epoch": 0.05372168284789644,
+      "grad_norm": 0.0006616997416131198,
+      "learning_rate": 8.3e-05,
+      "loss": 0.0,
+      "step": 83
+    },
+    {
+      "epoch": 0.05436893203883495,
+      "grad_norm": 0.0006774890352971852,
+      "learning_rate": 8.4e-05,
+      "loss": 0.0,
+      "step": 84
+    },
+    {
+      "epoch": 0.05501618122977346,
+      "grad_norm": 0.0006354961660690606,
+      "learning_rate": 8.5e-05,
+      "loss": 0.0,
+      "step": 85
+    },
+    {
+      "epoch": 0.05566343042071197,
+      "grad_norm": 0.00062626349972561,
+      "learning_rate": 8.6e-05,
+      "loss": 0.0,
+      "step": 86
+    },
+    {
+      "epoch": 0.05631067961165048,
+      "grad_norm": 0.0006652774172835052,
+      "learning_rate": 8.7e-05,
+      "loss": 0.0,
+      "step": 87
+    },
+    {
+      "epoch": 0.056957928802589,
+      "grad_norm": 0.00057514215586707,
+      "learning_rate": 8.800000000000001e-05,
+      "loss": 0.0,
+      "step": 88
+    },
+    {
+      "epoch": 0.05760517799352751,
+      "grad_norm": 0.0005757372709922493,
+      "learning_rate": 8.900000000000001e-05,
+      "loss": 0.0,
+      "step": 89
+    },
+    {
+      "epoch": 0.05825242718446602,
+      "grad_norm": 0.0005509501788765192,
+      "learning_rate": 9e-05,
+      "loss": 0.0,
+      "step": 90
+    },
+    {
+      "epoch": 0.05889967637540453,
+      "grad_norm": 0.0005754235316999257,
+      "learning_rate": 9.1e-05,
+      "loss": 0.0,
+      "step": 91
+    },
+    {
+      "epoch": 0.05954692556634304,
+      "grad_norm": 0.0005598037387244403,
+      "learning_rate": 9.200000000000001e-05,
+      "loss": 0.0,
+      "step": 92
+    },
+    {
+      "epoch": 0.06019417475728155,
+      "grad_norm": 0.0004931841394864023,
+      "learning_rate": 9.300000000000001e-05,
+      "loss": 0.0,
+      "step": 93
+    },
+    {
+      "epoch": 0.060841423948220064,
+      "grad_norm": 0.0005290132248774171,
+      "learning_rate": 9.4e-05,
+      "loss": 0.0,
+      "step": 94
+    },
+    {
+      "epoch": 0.061488673139158574,
+      "grad_norm": 0.0004422192578203976,
+      "learning_rate": 9.5e-05,
+      "loss": 0.0,
+      "step": 95
+    },
+    {
+      "epoch": 0.062135922330097085,
+      "grad_norm": 0.00048634959966875613,
+      "learning_rate": 9.6e-05,
+      "loss": 0.0,
+      "step": 96
+    },
+    {
+      "epoch": 0.0627831715210356,
+      "grad_norm": 0.0004640915140043944,
+      "learning_rate": 9.7e-05,
+      "loss": 0.0,
+      "step": 97
+    },
+    {
+      "epoch": 0.0634304207119741,
+      "grad_norm": 0.0004267208860255778,
+      "learning_rate": 9.8e-05,
+      "loss": 0.0,
+      "step": 98
+    },
+    {
+      "epoch": 0.06407766990291262,
+      "grad_norm": 0.0004712632216978818,
+      "learning_rate": 9.900000000000001e-05,
+      "loss": 0.0,
+      "step": 99
+    },
+    {
+      "epoch": 0.06472491909385113,
+      "grad_norm": 0.0004027521354146302,
+      "learning_rate": 0.0001,
+      "loss": 0.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.06537216828478964,
+      "grad_norm": 0.0003516871656756848,
+      "learning_rate": 9.999988183091062e-05,
+      "loss": 0.0,
+      "step": 101
+    },
+    {
+      "epoch": 0.06601941747572816,
+      "grad_norm": 0.0003226413100492209,
+      "learning_rate": 9.999952732420107e-05,
+      "loss": 0.0,
+      "step": 102
+    },
+    {
+      "epoch": 0.06666666666666667,
+      "grad_norm": 0.0003031356318388134,
+      "learning_rate": 9.999893648154699e-05,
+      "loss": 0.0,
+      "step": 103
+    },
+    {
+      "epoch": 0.06731391585760518,
+      "grad_norm": 0.00033037332468666136,
+      "learning_rate": 9.999810930574118e-05,
+      "loss": 0.0,
+      "step": 104
+    },
+    {
+      "epoch": 0.06796116504854369,
+      "grad_norm": 0.0003044827317353338,
+      "learning_rate": 9.999704580069346e-05,
+      "loss": 0.0,
+      "step": 105
+    },
+    {
+      "epoch": 0.0686084142394822,
+      "grad_norm": 0.0003044347104150802,
+      "learning_rate": 9.999574597143082e-05,
+      "loss": 0.0,
+      "step": 106
+    },
+    {
+      "epoch": 0.06925566343042071,
+      "grad_norm": 0.00029745750362053514,
+      "learning_rate": 9.999420982409722e-05,
+      "loss": 0.0,
+      "step": 107
+    },
+    {
+      "epoch": 0.06990291262135923,
+      "grad_norm": 0.0003329005849082023,
+      "learning_rate": 9.999243736595365e-05,
+      "loss": 0.0,
+      "step": 108
+    },
+    {
+      "epoch": 0.07055016181229773,
+      "grad_norm": 0.0003853734233416617,
+      "learning_rate": 9.999042860537814e-05,
+      "loss": 0.0,
+      "step": 109
+    },
+    {
+      "epoch": 0.07119741100323625,
+      "grad_norm": 0.0003464236797299236,
+      "learning_rate": 9.99881835518656e-05,
+      "loss": 0.0,
+      "step": 110
+    },
+    {
+      "epoch": 0.07184466019417475,
+      "grad_norm": 0.00030969688668847084,
+      "learning_rate": 9.998570221602785e-05,
+      "loss": 0.0,
+      "step": 111
+    },
+    {
+      "epoch": 0.07249190938511327,
+      "grad_norm": 0.0003167603281326592,
+      "learning_rate": 9.998298460959364e-05,
+      "loss": 0.0,
+      "step": 112
+    },
+    {
+      "epoch": 0.07313915857605179,
+      "grad_norm": 0.0003232895105611533,
+      "learning_rate": 9.998003074540838e-05,
+      "loss": 0.0,
+      "step": 113
+    },
+    {
+      "epoch": 0.07378640776699029,
+      "grad_norm": 0.00031525976373814046,
+      "learning_rate": 9.997684063743433e-05,
+      "loss": 0.0,
+      "step": 114
+    },
+    {
+      "epoch": 0.0744336569579288,
+      "grad_norm": 0.00029791181441396475,
+      "learning_rate": 9.997341430075036e-05,
+      "loss": 0.0,
+      "step": 115
+    },
+    {
+      "epoch": 0.07508090614886731,
+      "grad_norm": 0.0002939928963314742,
+      "learning_rate": 9.996975175155196e-05,
+      "loss": 0.0,
+      "step": 116
+    },
+    {
+      "epoch": 0.07572815533980583,
+      "grad_norm": 0.0003014066896867007,
+      "learning_rate": 9.996585300715116e-05,
+      "loss": 0.0,
+      "step": 117
+    },
+    {
+      "epoch": 0.07637540453074433,
+      "grad_norm": 0.00028381680021993816,
+      "learning_rate": 9.996171808597634e-05,
+      "loss": 0.0,
+      "step": 118
+    },
+    {
+      "epoch": 0.07702265372168285,
+      "grad_norm": 0.00030152060207910836,
+      "learning_rate": 9.995734700757235e-05,
+      "loss": 0.0,
+      "step": 119
+    },
+    {
+      "epoch": 0.07766990291262135,
+      "grad_norm": 0.0003169839910697192,
+      "learning_rate": 9.995273979260022e-05,
+      "loss": 0.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.07831715210355987,
+      "grad_norm": 0.00029620027635246515,
+      "learning_rate": 9.994789646283719e-05,
+      "loss": 0.0,
+      "step": 121
+    },
+    {
+      "epoch": 0.07896440129449839,
+      "grad_norm": 0.00030353505280800164,
+      "learning_rate": 9.99428170411765e-05,
+      "loss": 0.0,
+      "step": 122
+    },
+    {
+      "epoch": 0.07961165048543689,
+      "grad_norm": 0.0002934718213509768,
+      "learning_rate": 9.99375015516274e-05,
+      "loss": 0.0,
+      "step": 123
+    },
+    {
+      "epoch": 0.08025889967637541,
+      "grad_norm": 0.00028418784495443106,
+      "learning_rate": 9.993195001931493e-05,
+      "loss": 0.0,
+      "step": 124
+    },
+    {
+      "epoch": 0.08090614886731391,
+      "grad_norm": 0.0002818236534949392,
+      "learning_rate": 9.99261624704799e-05,
+      "loss": 0.0,
+      "step": 125
+    },
+    {
+      "epoch": 0.08155339805825243,
+      "grad_norm": 0.0002681802143342793,
+      "learning_rate": 9.992013893247866e-05,
+      "loss": 0.0,
+      "step": 126
+    },
+    {
+      "epoch": 0.08220064724919093,
+      "grad_norm": 0.00027062796289101243,
+      "learning_rate": 9.991387943378307e-05,
+      "loss": 0.0,
+      "step": 127
+    },
+    {
+      "epoch": 0.08284789644012945,
+      "grad_norm": 0.0002684716018848121,
+      "learning_rate": 9.990738400398027e-05,
+      "loss": 0.0,
+      "step": 128
+    },
+    {
+      "epoch": 0.08349514563106795,
+      "grad_norm": 0.00023824458185117692,
+      "learning_rate": 9.990065267377267e-05,
+      "loss": 0.0,
+      "step": 129
+    },
+    {
+      "epoch": 0.08414239482200647,
+      "grad_norm": 0.00027047592448070645,
+      "learning_rate": 9.989368547497763e-05,
+      "loss": 0.0,
+      "step": 130
+    },
+    {
+      "epoch": 0.08478964401294499,
+      "grad_norm": 0.00025663335691206157,
+      "learning_rate": 9.988648244052747e-05,
+      "loss": 0.0,
+      "step": 131
+    },
+    {
+      "epoch": 0.0854368932038835,
+      "grad_norm": 0.00023156039242167026,
+      "learning_rate": 9.987904360446922e-05,
+      "loss": 0.0,
+      "step": 132
+    },
+    {
+      "epoch": 0.08608414239482201,
+      "grad_norm": 0.0002463631972204894,
+      "learning_rate": 9.987136900196453e-05,
+      "loss": 0.0,
+      "step": 133
+    },
+    {
+      "epoch": 0.08673139158576051,
+      "grad_norm": 0.00025282951537519693,
+      "learning_rate": 9.986345866928941e-05,
+      "loss": 0.0,
+      "step": 134
+    },
+    {
+      "epoch": 0.08737864077669903,
+      "grad_norm": 0.0002451727050356567,
+      "learning_rate": 9.985531264383412e-05,
+      "loss": 0.0,
+      "step": 135
+    },
+    {
+      "epoch": 0.08802588996763754,
+      "grad_norm": 0.0002607478527352214,
+      "learning_rate": 9.984693096410303e-05,
+      "loss": 0.0,
+      "step": 136
+    },
+    {
+      "epoch": 0.08867313915857605,
+      "grad_norm": 0.00024905591271817684,
+      "learning_rate": 9.983831366971432e-05,
+      "loss": 0.0,
+      "step": 137
+    },
+    {
+      "epoch": 0.08932038834951456,
+      "grad_norm": 0.00024017650866881013,
+      "learning_rate": 9.982946080139992e-05,
+      "loss": 0.0,
+      "step": 138
+    },
+    {
+      "epoch": 0.08996763754045307,
+      "grad_norm": 0.00024329974257852882,
+      "learning_rate": 9.982037240100526e-05,
+      "loss": 0.0,
+      "step": 139
+    },
+    {
+      "epoch": 0.09061488673139159,
+      "grad_norm": 0.00023055482597555965,
+      "learning_rate": 9.981104851148904e-05,
+      "loss": 0.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.0912621359223301,
+      "grad_norm": 0.00023951951880007982,
+      "learning_rate": 9.980148917692309e-05,
+      "loss": 0.0,
+      "step": 141
+    },
+    {
+      "epoch": 0.09190938511326861,
+      "grad_norm": 0.0002374964242335409,
+      "learning_rate": 9.979169444249212e-05,
+      "loss": 0.0,
+      "step": 142
+    },
+    {
+      "epoch": 0.09255663430420712,
+      "grad_norm": 0.00024244707310572267,
+      "learning_rate": 9.978166435449352e-05,
+      "loss": 0.0,
+      "step": 143
+    },
+    {
+      "epoch": 0.09320388349514563,
+      "grad_norm": 0.00022375534172169864,
+      "learning_rate": 9.977139896033716e-05,
+      "loss": 0.0,
+      "step": 144
+    },
+    {
+      "epoch": 0.09385113268608414,
+      "grad_norm": 0.00022595201153308153,
+      "learning_rate": 9.976089830854514e-05,
+      "loss": 0.0,
+      "step": 145
+    },
+    {
+      "epoch": 0.09449838187702266,
+      "grad_norm": 0.00021144020138308406,
+      "learning_rate": 9.975016244875151e-05,
+      "loss": 0.0,
+      "step": 146
+    },
+    {
+      "epoch": 0.09514563106796116,
+      "grad_norm": 0.00021918013226240873,
+      "learning_rate": 9.973919143170218e-05,
+      "loss": 0.0,
+      "step": 147
+    },
+    {
+      "epoch": 0.09579288025889968,
+      "grad_norm": 0.00022214902855921537,
+      "learning_rate": 9.972798530925456e-05,
+      "loss": 0.0,
+      "step": 148
+    },
+    {
+      "epoch": 0.0964401294498382,
+      "grad_norm": 0.00022321178403217345,
+      "learning_rate": 9.971654413437732e-05,
+      "loss": 0.0,
+      "step": 149
+    },
+    {
+      "epoch": 0.0970873786407767,
+      "grad_norm": 0.0002452544867992401,
+      "learning_rate": 9.97048679611502e-05,
+      "loss": 0.0,
+      "step": 150
+    },
+    {
+      "epoch": 0.09773462783171522,
+      "grad_norm": 0.0005459377425722778,
+      "learning_rate": 9.96929568447637e-05,
+      "loss": 0.0,
+      "step": 151
+    },
+    {
+      "epoch": 0.09838187702265372,
+      "grad_norm": 0.00016863649943843484,
+      "learning_rate": 9.968081084151884e-05,
+      "loss": 0.0,
+      "step": 152
+    },
+    {
+      "epoch": 0.09902912621359224,
+      "grad_norm": 0.00017263075278606266,
+      "learning_rate": 9.966843000882695e-05,
+      "loss": 0.0,
+      "step": 153
+    },
+    {
+      "epoch": 0.09967637540453074,
+      "grad_norm": 0.00018214034207630903,
+      "learning_rate": 9.965581440520925e-05,
+      "loss": 0.0,
+      "step": 154
+    },
+    {
+      "epoch": 0.10032362459546926,
+      "grad_norm": 0.00018302416719961911,
+      "learning_rate": 9.964296409029675e-05,
+      "loss": 0.0,
+      "step": 155
+    },
+    {
+      "epoch": 0.10097087378640776,
+      "grad_norm": 0.00020016191410832107,
+      "learning_rate": 9.962987912482984e-05,
+      "loss": 0.0,
+      "step": 156
+    },
+    {
+      "epoch": 0.10161812297734628,
+      "grad_norm": 0.00018125724454876035,
+      "learning_rate": 9.961655957065806e-05,
+      "loss": 0.0,
+      "step": 157
+    },
+    {
+      "epoch": 0.1022653721682848,
+      "grad_norm": 0.00016165344277396798,
+      "learning_rate": 9.960300549073978e-05,
+      "loss": 0.0,
+      "step": 158
+    },
+    {
+      "epoch": 0.1029126213592233,
+      "grad_norm": 0.00017234106780961156,
+      "learning_rate": 9.958921694914194e-05,
+      "loss": 0.0,
+      "step": 159
+    },
+    {
+      "epoch": 0.10355987055016182,
+      "grad_norm": 0.00018339157395530492,
+      "learning_rate": 9.957519401103972e-05,
+      "loss": 0.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.10420711974110032,
+      "grad_norm": 0.00018937568529509008,
+      "learning_rate": 9.956093674271622e-05,
+      "loss": 0.0,
+      "step": 161
+    },
+    {
+      "epoch": 0.10485436893203884,
+      "grad_norm": 0.0001959225774044171,
+      "learning_rate": 9.95464452115622e-05,
+      "loss": 0.0,
+      "step": 162
+    },
+    {
+      "epoch": 0.10550161812297734,
+      "grad_norm": 0.0001900671049952507,
+      "learning_rate": 9.953171948607569e-05,
+      "loss": 0.0,
+      "step": 163
+    },
+    {
+      "epoch": 0.10614886731391586,
+      "grad_norm": 0.00021061516599729657,
+      "learning_rate": 9.95167596358617e-05,
+      "loss": 0.0,
+      "step": 164
+    },
+    {
+      "epoch": 0.10679611650485436,
+      "grad_norm": 0.00019292550859972835,
+      "learning_rate": 9.950156573163192e-05,
+      "loss": 0.0,
+      "step": 165
+    },
+    {
+      "epoch": 0.10744336569579288,
+      "grad_norm": 0.00019005850481335074,
+      "learning_rate": 9.948613784520434e-05,
+      "loss": 0.0,
+      "step": 166
+    },
+    {
+      "epoch": 0.1080906148867314,
+      "grad_norm": 0.00017910161113832146,
+      "learning_rate": 9.947047604950294e-05,
+      "loss": 0.0,
+      "step": 167
+    },
+    {
+      "epoch": 0.1087378640776699,
+      "grad_norm": 0.00018016544345300645,
+      "learning_rate": 9.94545804185573e-05,
+      "loss": 0.0,
+      "step": 168
+    },
+    {
+      "epoch": 0.10938511326860842,
+      "grad_norm": 0.0001976732019102201,
+      "learning_rate": 9.943845102750234e-05,
+      "loss": 0.0,
+      "step": 169
+    },
+    {
+      "epoch": 0.11003236245954692,
+      "grad_norm": 0.00019848241936415434,
+      "learning_rate": 9.942208795257786e-05,
+      "loss": 0.0,
+      "step": 170
+    },
+    {
+      "epoch": 0.11067961165048544,
+      "grad_norm": 0.00019966305990237743,
+      "learning_rate": 9.940549127112823e-05,
+      "loss": 0.0,
+      "step": 171
+    },
+    {
+      "epoch": 0.11132686084142394,
+      "grad_norm": 0.00018405074661131948,
+      "learning_rate": 9.93886610616021e-05,
+      "loss": 0.0,
+      "step": 172
+    },
+    {
+      "epoch": 0.11197411003236246,
+      "grad_norm": 0.00019481593335513026,
+      "learning_rate": 9.937159740355182e-05,
+      "loss": 0.0,
+      "step": 173
+    },
+    {
+      "epoch": 0.11262135922330097,
+      "grad_norm": 0.0001841599150793627,
+      "learning_rate": 9.93543003776333e-05,
+      "loss": 0.0,
+      "step": 174
+    },
+    {
+      "epoch": 0.11326860841423948,
+      "grad_norm": 0.00018602985073812306,
+      "learning_rate": 9.93367700656055e-05,
+      "loss": 0.0,
+      "step": 175
+    },
+    {
+      "epoch": 0.113915857605178,
+      "grad_norm": 0.00017339715850539505,
+      "learning_rate": 9.931900655033004e-05,
+      "loss": 0.0,
+      "step": 176
+    },
+    {
+      "epoch": 0.1145631067961165,
+      "grad_norm": 0.0001882009528344497,
+      "learning_rate": 9.930100991577089e-05,
+      "loss": 0.0,
+      "step": 177
+    },
+    {
+      "epoch": 0.11521035598705502,
+      "grad_norm": 0.00018708783318288624,
+      "learning_rate": 9.928278024699385e-05,
+      "loss": 0.0,
+      "step": 178
+    },
+    {
+      "epoch": 0.11585760517799353,
+      "grad_norm": 0.00017336831660941243,
+      "learning_rate": 9.926431763016626e-05,
+      "loss": 0.0,
+      "step": 179
+    },
+    {
+      "epoch": 0.11650485436893204,
+      "grad_norm": 0.00015800120308995247,
+      "learning_rate": 9.924562215255655e-05,
+      "loss": 0.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.11715210355987055,
+      "grad_norm": 0.00017240441229660064,
+      "learning_rate": 9.922669390253385e-05,
+      "loss": 0.0,
+      "step": 181
+    },
+    {
+      "epoch": 0.11779935275080906,
+      "grad_norm": 0.0001618278183741495,
+      "learning_rate": 9.920753296956749e-05,
+      "loss": 0.0,
+      "step": 182
+    },
+    {
+      "epoch": 0.11844660194174757,
+      "grad_norm": 0.004332312382757664,
+      "learning_rate": 9.918813944422667e-05,
+      "loss": 0.0,
+      "step": 183
+    },
+    {
+      "epoch": 0.11909385113268608,
+      "grad_norm": 0.0001733552635414526,
+      "learning_rate": 9.916851341818001e-05,
+      "loss": 0.0,
+      "step": 184
+    },
+    {
+      "epoch": 0.11974110032362459,
+      "grad_norm": 0.00018002038996201009,
+      "learning_rate": 9.91486549841951e-05,
+      "loss": 0.0,
+      "step": 185
+    },
+    {
+      "epoch": 0.1203883495145631,
+      "grad_norm": 0.00016852655971888453,
+      "learning_rate": 9.912856423613804e-05,
+      "loss": 0.0,
+      "step": 186
+    },
+    {
+      "epoch": 0.12103559870550162,
+      "grad_norm": 0.00016074655286502093,
+      "learning_rate": 9.910824126897308e-05,
+      "loss": 0.0,
+      "step": 187
+    },
+    {
+      "epoch": 0.12168284789644013,
+      "grad_norm": 0.00018699858628679067,
+      "learning_rate": 9.908768617876205e-05,
+      "loss": 0.0,
+      "step": 188
+    },
+    {
+      "epoch": 0.12233009708737864,
+      "grad_norm": 0.00017236075655091554,
+      "learning_rate": 9.906689906266401e-05,
+      "loss": 0.0,
+      "step": 189
+    },
+    {
+      "epoch": 0.12297734627831715,
+      "grad_norm": 0.0001635546504985541,
+      "learning_rate": 9.904588001893477e-05,
+      "loss": 0.0,
+      "step": 190
+    },
+    {
+      "epoch": 0.12362459546925567,
+      "grad_norm": 0.00016175238124560565,
+      "learning_rate": 9.902462914692633e-05,
+      "loss": 0.0,
+      "step": 191
+    },
+    {
+      "epoch": 0.12427184466019417,
+      "grad_norm": 0.19127343595027924,
+      "learning_rate": 9.900314654708659e-05,
+      "loss": 0.0006,
+      "step": 192
+    },
+    {
+      "epoch": 0.12491909385113269,
+      "grad_norm": 0.00015970351523719728,
+      "learning_rate": 9.898143232095869e-05,
+      "loss": 0.0,
+      "step": 193
+    },
+    {
+      "epoch": 0.1255663430420712,
+      "grad_norm": 0.00017519606626592577,
+      "learning_rate": 9.895948657118064e-05,
+      "loss": 0.0,
+      "step": 194
+    },
+    {
+      "epoch": 0.1262135922330097,
+      "grad_norm": 0.00017720655887387693,
+      "learning_rate": 9.893730940148482e-05,
+      "loss": 0.0,
+      "step": 195
+    },
+    {
+      "epoch": 0.1268608414239482,
+      "grad_norm": 0.0001791470276657492,
+      "learning_rate": 9.891490091669747e-05,
+      "loss": 0.0,
+      "step": 196
+    },
+    {
+      "epoch": 0.12750809061488674,
+      "grad_norm": 0.00019451871048659086,
+      "learning_rate": 9.88922612227382e-05,
+      "loss": 0.0,
+      "step": 197
+    },
+    {
+      "epoch": 0.12815533980582525,
+      "grad_norm": 0.0001841304765548557,
+      "learning_rate": 9.886939042661949e-05,
+      "loss": 0.0,
+      "step": 198
+    },
+    {
+      "epoch": 0.12880258899676375,
+      "grad_norm": 0.00019261128909420222,
+      "learning_rate": 9.884628863644617e-05,
+      "loss": 0.0,
+      "step": 199
+    },
+    {
+      "epoch": 0.12944983818770225,
+      "grad_norm": 0.00020318938186392188,
+      "learning_rate": 9.882295596141496e-05,
+      "loss": 0.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.13009708737864079,
+      "grad_norm": 0.00020783746731467545,
+      "learning_rate": 9.87993925118139e-05,
+      "loss": 0.0,
+      "step": 201
+    },
+    {
+      "epoch": 0.1307443365695793,
+      "grad_norm": 0.00019535954925231636,
+      "learning_rate": 9.877559839902184e-05,
+      "loss": 0.0,
+      "step": 202
+    },
+    {
+      "epoch": 0.1313915857605178,
+      "grad_norm": 0.00019633652118500322,
+      "learning_rate": 9.875157373550792e-05,
+      "loss": 0.0,
+      "step": 203
+    },
+    {
+      "epoch": 0.13203883495145632,
+      "grad_norm": 0.00019633228657767177,
+      "learning_rate": 9.872731863483105e-05,
+      "loss": 0.0,
+      "step": 204
+    },
+    {
+      "epoch": 0.13268608414239483,
+      "grad_norm": 0.00020233304530847818,
+      "learning_rate": 9.870283321163934e-05,
+      "loss": 0.0,
+      "step": 205
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 0.00019641958351712674,
+      "learning_rate": 9.867811758166963e-05,
+      "loss": 0.0,
+      "step": 206
+    },
+    {
+      "epoch": 0.13398058252427184,
+      "grad_norm": 0.00019531269208528101,
+      "learning_rate": 9.865317186174683e-05,
+      "loss": 0.0,
+      "step": 207
+    },
+    {
+      "epoch": 0.13462783171521037,
+      "grad_norm": 0.00021375597862061113,
+      "learning_rate": 9.862799616978347e-05,
+      "loss": 0.0,
+      "step": 208
+    },
+    {
+      "epoch": 0.13527508090614887,
+      "grad_norm": 0.00019547225383576006,
+      "learning_rate": 9.86025906247791e-05,
+      "loss": 0.0,
+      "step": 209
+    },
+    {
+      "epoch": 0.13592233009708737,
+      "grad_norm": 0.00022372894454747438,
+      "learning_rate": 9.85769553468197e-05,
+      "loss": 0.0,
+      "step": 210
+    },
+    {
+      "epoch": 0.13656957928802588,
+      "grad_norm": 0.00020901462994515896,
+      "learning_rate": 9.85510904570772e-05,
+      "loss": 0.0,
+      "step": 211
+    },
+    {
+      "epoch": 0.1372168284789644,
+      "grad_norm": 0.00021962117170915008,
+      "learning_rate": 9.852499607780881e-05,
+      "loss": 0.0,
+      "step": 212
+    },
+    {
+      "epoch": 0.1378640776699029,
+      "grad_norm": 0.0002420559903839603,
+      "learning_rate": 9.849867233235647e-05,
+      "loss": 0.0,
+      "step": 213
+    },
+    {
+      "epoch": 0.13851132686084142,
+      "grad_norm": 0.00020637971465475857,
+      "learning_rate": 9.847211934514636e-05,
+      "loss": 0.0,
+      "step": 214
+    },
+    {
+      "epoch": 0.13915857605177995,
+      "grad_norm": 0.0002189079677918926,
+      "learning_rate": 9.844533724168809e-05,
+      "loss": 0.0,
+      "step": 215
+    },
+    {
+      "epoch": 0.13980582524271845,
+      "grad_norm": 0.00023518242232967168,
+      "learning_rate": 9.84183261485744e-05,
+      "loss": 0.0,
+      "step": 216
+    },
+    {
+      "epoch": 0.14045307443365695,
+      "grad_norm": 0.0002103596634697169,
+      "learning_rate": 9.839108619348029e-05,
+      "loss": 0.0,
+      "step": 217
+    },
+    {
+      "epoch": 0.14110032362459546,
+      "grad_norm": 0.00021611133706755936,
+      "learning_rate": 9.836361750516262e-05,
+      "loss": 0.0,
+      "step": 218
+    },
+    {
+      "epoch": 0.141747572815534,
+      "grad_norm": 0.00023783188953530043,
+      "learning_rate": 9.833592021345937e-05,
+      "loss": 0.0,
+      "step": 219
+    },
+    {
+      "epoch": 0.1423948220064725,
+      "grad_norm": 0.00024469022173434496,
+      "learning_rate": 9.83079944492891e-05,
+      "loss": 0.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.143042071197411,
+      "grad_norm": 0.0002635126293171197,
+      "learning_rate": 9.82798403446503e-05,
+      "loss": 0.0,
+      "step": 221
+    },
+    {
+      "epoch": 0.1436893203883495,
+      "grad_norm": 0.00025157438358291984,
+      "learning_rate": 9.825145803262075e-05,
+      "loss": 0.0,
+      "step": 222
+    },
+    {
+      "epoch": 0.14433656957928803,
+      "grad_norm": 0.00023788264661561698,
+      "learning_rate": 9.822284764735694e-05,
+      "loss": 0.0,
+      "step": 223
+    },
+    {
+      "epoch": 0.14498381877022654,
+      "grad_norm": 0.0002576147671788931,
+      "learning_rate": 9.819400932409339e-05,
+      "loss": 0.0,
+      "step": 224
+    },
+    {
+      "epoch": 0.14563106796116504,
+      "grad_norm": 0.00025982639635913074,
+      "learning_rate": 9.816494319914203e-05,
+      "loss": 0.0,
+      "step": 225
+    },
+    {
+      "epoch": 0.14627831715210357,
+      "grad_norm": 0.00024497153935953975,
+      "learning_rate": 9.813564940989157e-05,
+      "loss": 0.0,
+      "step": 226
+    },
+    {
+      "epoch": 0.14692556634304207,
+      "grad_norm": 0.0002377478958806023,
+      "learning_rate": 9.810612809480684e-05,
+      "loss": 0.0,
+      "step": 227
+    },
+    {
+      "epoch": 0.14757281553398058,
+      "grad_norm": 0.0002221636677859351,
+      "learning_rate": 9.807637939342809e-05,
+      "loss": 0.0,
+      "step": 228
+    },
+    {
+      "epoch": 0.14822006472491908,
+      "grad_norm": 0.00021500229195225984,
+      "learning_rate": 9.804640344637042e-05,
+      "loss": 0.0,
+      "step": 229
+    },
+    {
+      "epoch": 0.1488673139158576,
+      "grad_norm": 0.00021654285956174135,
+      "learning_rate": 9.801620039532302e-05,
+      "loss": 0.0,
+      "step": 230
+    },
+    {
+      "epoch": 0.14951456310679612,
+      "grad_norm": 0.0002289855619892478,
+      "learning_rate": 9.798577038304859e-05,
+      "loss": 0.0,
+      "step": 231
+    },
+    {
+      "epoch": 0.15016181229773462,
+      "grad_norm": 0.00022614987392444164,
+      "learning_rate": 9.79551135533826e-05,
+      "loss": 0.0,
+      "step": 232
+    },
+    {
+      "epoch": 0.15080906148867315,
+      "grad_norm": 0.00022708874894306064,
+      "learning_rate": 9.792423005123265e-05,
+      "loss": 0.0,
+      "step": 233
+    },
+    {
+      "epoch": 0.15145631067961166,
+      "grad_norm": 0.00020380753267090768,
+      "learning_rate": 9.789312002257774e-05,
+      "loss": 0.0,
+      "step": 234
+    },
+    {
+      "epoch": 0.15210355987055016,
+      "grad_norm": 0.00019009801326319575,
+      "learning_rate": 9.786178361446759e-05,
+      "loss": 0.0,
+      "step": 235
+    },
+    {
+      "epoch": 0.15275080906148866,
+      "grad_norm": 0.00020547406165860593,
+      "learning_rate": 9.783022097502204e-05,
+      "loss": 0.0,
+      "step": 236
+    },
+    {
+      "epoch": 0.1533980582524272,
+      "grad_norm": 0.0002082720020553097,
+      "learning_rate": 9.779843225343022e-05,
+      "loss": 0.0,
+      "step": 237
+    },
+    {
+      "epoch": 0.1540453074433657,
+      "grad_norm": 0.00019760527356993407,
+      "learning_rate": 9.776641759994987e-05,
+      "loss": 0.0,
+      "step": 238
+    },
+    {
+      "epoch": 0.1546925566343042,
+      "grad_norm": 0.00019591070304159075,
+      "learning_rate": 9.77341771659067e-05,
+      "loss": 0.0,
+      "step": 239
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1545,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 239,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.005006523457864e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91c2c337db8ffe8f2b41a238a984898b33f4f727ae6a35b1f93b6fc222248fde
+size 6840