Qwen

/

Qwen3-32B

like 259

Follow

Qwen 29k

What were your YaRN settings for the model? An optimal scaling factor is about 3.37, and in pretty much every framework it has to be set manually.

Also, I will give this a shot in a bit.

@Thireus

First, I saw your Reddit post, but they keep shadowbanning my accounts! No idea why, literally the only thing I post is long context testing on localllama, heh.

Anyway I lightly modified and tested this with exllamaV3, context size 109056 with q5_q4 cache quantization, YaRN scaling factor 3.40625, with this quantization specifically: https://huggingface.co/Downtown-Case/Qwen_Qwen3-32B-exl3-3.45bpw-hb8

I massaged the question, and did a few retries to dial in the sampling (0.3 with MinP seems good), but here's the latest response:

...
Use the context I've provided as knowledge to answer my question: In runescape, what level are you at when you get to half the xp of max level? In other words, when one's total experience is halfway to max experience, what level is the player?<|im_end|>
<|im_start|>assistant
<think>
Okay, I need to figure out at what level a player in RuneScape has half the total experience of the max level. The user is asking about the level when the total experience is halfway to the maximum possible experience. 

First, I remember that each level in a skill requires more experience to reach the next level. The experience needed increases as you level up. The max level for any skill in RuneScape is 99. The total experience required to reach level 99 is the sum of all the experience needed to get from level 1 to 99. 

The user is asking for the level at which the total experience is half of that maximum. So, I need to calculate the total experience required to reach level 99, then find the level where the cumulative experience is half of that total. 

Looking at the context provided, there's a table under "Skills/Experience Table" that lists the experience needed for each level.  For example, level 1 requires 0 experience, level 2 requires 83, level 3 is 174, and so on up to level 99.  The total experience for level 99 is listed as 13,034,431. 

Wait, the table shows the experience needed to reach each level. So the total experience for level 99 is the sum of all the experience from level 1 to 99.  The user wants to know at which level the cumulative experience is half of that total. 

Let me check the table again. The last entry in the table is level 99, which requires 13,034,431 experience. So the total experience needed to reach level 99 is 13,034,431. Half of that would be 13,034,431 / 2 = 6,517,215.5. 

Now, I need to find the level where the cumulative experience reaches approximately 6,517,215.5. Let's look through the table. 

Looking at the levels and the experience needed to reach them:

- Level 1: 0
- Level 2: 83
- Level 3: 174
- ...
- Level 50: 101,333
- Level 60: 273,742
- Level 70: 737,627
- Level 80: 1,986,068
- Level 90: 5,346,332
- Level 99: 13,034,431

So, the total experience at level 90 is 5,346,332. The half of the max experience is 6,517,215.5. So between level 90 and 99, the total experience increases from 5,346,332 to 13,034,431. 

Now, I need to find the level where the cumulative experience is around 6,517,215.5. Let's check the levels after 90. The table shows level 91 is 6,517,253. That's very close to the half of the max experience. 

So, the total experience at level 91 is 6,517,253, which is just slightly above half of the max experience (which is 6,517,215.5). Therefore, the level at which the total experience is halfway to max is level 91. 

Wait, let me confirm. The total experience for level 99 is 13,034,431. Half of that is 6,517,215.5. The experience needed to reach level 91 is 6,517,253. So the cumulative experience at level 91 is 6,517,253, which is just a bit more than half. Therefore, the level where the total experience is halfway is level 91. 

But the user is asking for the level when the total experience is half of the max. So the answer is level 91. 

I should also check if there's any rounding or if the exact half is at a different level. The exact half is 6,517,215.5. The experience for level 91 is 6,517,253. The difference between level 90 and 91 is 6,517,253 - 5,346,332 = 1,170,921. The half of the max experience is 6,517,215.5. So the cumulative experience at level 90 is 5,346,332. The remaining to reach half is 6,517,215.5 - 5,346,332 = 1,170,883.5. 

Since the experience needed to reach level 91 is 1,170,921, which is slightly more than the remaining needed (1,170,883.5), the halfway point is just a little bit into level 91. Therefore, the level at which the total experience is half of the max is level 91.
</think>

In RuneScape, when a player's total experience reaches half of the maximum experience required to reach level 99, they are at **level 91**. 

### Explanation:
1. **Total Experience for Level 99**:  
   The cumulative experience required to reach level 99 is **13,034,431** (as listed in the "Skills/Experience Table" under level 99).

2. **Half of Max Experience**:  
   Half of this total is **6,517,215.5**.

3. **Level 91**:  
   The cumulative experience at level 91 is **6,517,253**, which is just slightly above the halfway mark. This means the player reaches half of the max experience during level 91.

### Key Details:
- **Level 90**: Cumulative experience is **5,346,332**.  
- **Level 91**: Cumulative experience is **6,517,253**.  
- The difference between level 90 and 91 is **1,170,921** experience. Since **6,517,215.5 - 5,346,332 = 1,170,883.5**, the halfway point occurs **just before completing level 91**.

Thus, **level 91** is the level at which total experience is approximately half of the maximum required to reach level 99.

For reference, it squeezes into a single 3090 at 23740MB! I'm shocked it's coherent at all. Prompt processing took 452s, so 236 tok/s, and it responds at 3 tokens/s (including UI lag, I believe).

Thank you @Downtown-Case for your input, you were right I believe I was not using the settings correctly for EXL2, and I'm getting similar results as yours now. However, the answer is incorrect as it's not 91 but 92. In your answer it believes that Level 91 is 6,517,253 which is wrong (it's level 92).

This is the closest I got to a good answer, which is when using Rope Alpha 4 instead of YaRN. It finally found the table but its reasoning is still flawed, and the final answer is also wrong.
I've also tried to adjust the YaRN scaling factor to 3.37 and also to 4.0. I'm sometimes getting a lot of repetitions. And so far unable to get the right answer.

I'm still trying various settings to figure out if there's something wrong with the quatization or if this 32B model is just unable to reason properly on large context sizes...

<think>
Okay, so the user is asking about Old School RuneScape (OSRS) and wants to know at what level you are when you reach half the experience of the maximum level, which is level 99. They provided a detailed book with all the guides, quests, and skill information.

First, I need to figure out the total experience required to reach level 99 in any skill. The book has an "Experience Table" under the Skills section. Let me check that.

Looking at the "Experience Table" in the Skills section, the last entry is for level 99, which requires 13,034,431 experience. The total experience needed to reach level 99 is the sum of all the experience needed up to that point. Wait, but the table lists each level's experience needed to reach that level. For example, level 1 needs 0, level 2 needs 83, level 3 needs 174, and so on up to level 99 needing 13,034,431.

To find half of the total experience required for level 99, I need to calculate half of 13,034,431. Half of that is approximately 6,517,215.5. So the question is, which level corresponds to having accumulated around 6.5 million experience?

But wait, the total experience required to reach level 99 is the cumulative sum of all the experience needed for each level from 1 to 99. The table shows each level's experience needed to reach that level. For example, level 1 requires 0, level 2 requires 83, level 3 adds 91, making the total experience up to level 3 as 83 + 91 = 174? Wait, no, the table lists the experience needed to reach that level. So each entry is the total experience needed to reach that level. For example, to get to level 2, you need 83, level 3 needs 174, which is 83 + 91. So the cumulative experience for level 99 is 13,034,431.

So half of that is 6,517,215.5. Now, I need to find the level where the cumulative experience is closest to this value. Let me look through the Experience Table to see which level's experience is around this number.

Looking at the table:

Level 50: 101,333 experience needed to reach level 50.

Level 60: 273,742

Level 70: 737,627

Level 75: 1,210,421

Level 80: 1,986,068

Level 85: 3,258,594

Level 90: 5,346,332

Level 95: 8,771,558

Level 99: 13,034,431

Wait, so the cumulative experience at level 90 is 5,346,332, which is less than 6.5 million. Level 95 is 8,771,558, which is more than 6.5 million. So the halfway point is between level 90 and 95. To find the exact level, we can see how much more is needed from level 90 to reach half of the total.

Half of total is 6,517,215.5. Level 90 is at 5,346,332. The difference is 6,517,215.5 - 5,346,332 = 1,170,883.5 experience needed after level 90. Now, looking at the levels between 90 and 95:

Level 91: 5,346,332 + 1,008,499 = 6,354,831

Level 92: 6,354,831 + 614,422 = 6,969,253

Wait, the experience needed to reach each level is as follows:

Level 90: 5,346,332

Level 91: 5,346,332 + 1,008,499 = 6,354,831

Level 92: 6,354,831 + 614,422 = 6,969,253

So 6,517,215.5 falls between level 91 and 92. The difference between 6,517,215.5 and level 91's 6,354,831 is 162,384.5 experience. The experience needed for level 92 is 614,422. So 162,384.5 / 614,422 is about 26.4% of the way through level 92. So the approximate level would be 91.26, which is around level 91.

But the user is asking for the level at which you have half the experience of max level (99). Since the experience is cumulative, the halfway point is around level 91 or 92. However, since you can't have a fraction of a level, the closest whole level would be level 91. But to be precise, maybe the answer is level 91 or 92. However, typically, in OSRS, people often say that halfway is around level 91 or 92. But perhaps the exact answer is level 91. Let me check the exact numbers again.

Wait, the total experience for level 99 is 13,034,431. Half is 6,517,215.5. The cumulative experience for level 91 is 6,354,831, which is less than half. Level 92 is 6,969,253, which is more than half. So the halfway point is between level 91 and 92. Since the user is asking for the level, the answer would be that you reach half the experience of max level (level 99) at level 91 or 92. But since you can't be partway through a level, the answer is approximately level 91. However, in practice, players might refer to it as around level 91-92. But the exact answer would require interpolation, but the user might expect a specific level. Looking at the Experience Table, the closest level where the total experience is just below half is level 91. So the answer is level 91.
</think>

In Old School RuneScape (OSRS), the total experience required to reach level 99 in any skill is **13,034,431**. Half of this total is **6,517,215.5** experience. 

From the **Experience Table** in the provided guide, the cumulative experience required to reach level 91 is **6,354,831**, and for level 92 it is **6,969,253**. Since **6,517,215.5** falls between these two values, you reach **half the experience of the maximum level (99)** at approximately **level 91**. 

### Key Breakdown:
- **Level 91**: 6,354,831 total experience (still below half).
- **Level 92**: 6,969,253 total experience (exceeds half).

Thus, **level 91** is the closest whole level where you are **very close to half the total experience required for level 99**. For practical purposes, players often round this to **level 91** as the halfway point.

On further testing, it does tend to pull wrong numbers in the thoughts, even at zero temperature.

This is what happened at high YaRN scales with Qwen 2.5 too. I think "fuzzy" tasks like summarization tend to work better out there.

rope alpha 4

Rope alpha should be at its default value.

The setting you want is not in any UI, you have to edit it in the model's config file:

{
    "architectures": [
        "Qwen3ForCausalLM"
    ],
    "attention_bias": false,
    "attention_dropout": 0.0,
    "bos_token_id": 151643,
    "eos_token_id": 151645,
    "head_dim": 128,
    "hidden_act": "silu",
    "hidden_size": 5120,
    "initializer_range": 0.02,
    "intermediate_size": 25600,
    "max_position_embeddings": 40960,
    "max_window_layers": 64,
    "model_type": "qwen3",
    "num_attention_heads": 64,
    "num_hidden_layers": 64,
    "num_key_value_heads": 8,
    "rms_norm_eps": 1e-06,
    "rope_scaling": {
        "rope_type": "yarn",
        "factor": 3.40625, # <THIS IS THE SCALING FACTOR
        "original_max_position_embeddings": 32768
    },
    "rope_theta": 1000000,
    "sliding_window": null,
    "tie_word_embeddings": false,
    "torch_dtype": "bfloat16",
    "transformers_version": "4.51.0",
    "use_cache": true,
    "use_sliding_window": false,
    "vocab_size": 151936,
    "quantization_config": {
        "quant_method": "exl3",
        "version": "0.0.1",
        "bits": 3.45,
        "head_bits": 8,
        "calibration": {
            "rows": 100,
            "cols": 2048
        },
        "out_scales": "auto"
    }
}

It should be set to max_context/original_max_position_embeddings, which is about 3.40625 for 109K. Or, as another example, 2.0 for 64K.

There is an open discussion of exllamav3 setting this automatically since this is such obsucre information, heh: https://github.com/turboderp-org/exllamav3/issues/29

Oh, and on this:

I'm still trying various settings to figure out if there's something wrong with the quatization or if this 32B model is just unable to reason properly on large context sizes...

One can quantize long-context exllamav2 models to maximize the first/last layer and always use a head (-hb) of 8 instead of the default 6. This supposedly works better at longer contexts.

...But if you're using a ~5bpw exl2, it should be okay. Qwen's 32K models are just not super reliable at this context size, I don't think.

Thank you, that's really helpful. I am using Qwen3-32B_exl2_8.0bpw-hb8with fp16 cache.

I was indeed using a custom config (bypassing UI):

        config.yarn_rope_original_max_position_embeddings = 32768 # THIREUS
        config.alt_rope_method = "yarn" # THIREUS
        config.yarn_rope_factor = 3.37 # THIREUS

I finally got a correct answer:

<think>
Okay, let's see. The user is asking about Old School RuneScape, specifically what level you are at when you get to half the XP of max level. I need to figure this out using the provided book.

First, I remember the XP table is in the "Experience Table" section under the Skills section. Let me check that. The table lists the experience needed for each level up to 99. The max level is 99, so the total XP required to reach 99 is the sum of all the XP needed from level 1 to 99. The user wants the level where the cumulative XP is half of that total.

Looking at the Experience Table in the book, the last entry is level 99, which requires 130344351 experience. Wait, let me check the exact numbers. The table shows the XP needed to reach each level and the difference from the previous level. The total XP to reach level 99 is the sum of all the "Experience needed to reach level" from 1 to 99. But the user is asking for the level where the cumulative XP is half of the max. 

Wait, the max XP for level 99 is 130344351. So half of that would be around 65,172,175.5. I need to find the level where the cumulative XP is approximately that. Let me go through the table and add up the experience incrementally until I reach about that half.

Let me start adding the experience needed for each level. Starting from level 1, which is 0 XP. Level 2 is 83 XP, level 3 is 174 (total 83 + 174 = 257?), but the table shows "Experience needed to reach level" for each level. Wait, the "Experience needed to reach level" is the total XP required to reach that level. So for each level, that's the total so far. For example, level 1 is 0. Level 2 is 83. Level 3 is 174. Wait, but the "Experience needed to reach level" for level 99 is 130344351. So the total XP needed to reach level 99 is 130,344,351. Half of that is 65,172,175.5. 

Now I need to find the level where the "Experience needed to reach level" is approximately 65 million. Let me check the table entries. Let me look through the levels:

Level 1: 0
Level 2: 83
Level 3: 174
...
Level 40: 202,240
Level 50: 1,666,630
Level 60: 4,478,900
Level 70: 10,366,900
Level 80: 20,873,600
Level 90: 40,567,500
Level 99: 130,344,351

Wait, the Experience needed to reach level 90 is 40,567,500. The half of the max XP is about 65 million. The cumulative XP to reach level 90 is 40 million. That's less than half. Level 90 is 40,567,500. Level 95 is 78,212,300. Level 96 is 93,255,500. Level 97 is 110,509,000. Level 98 is 129,619,000. Level 99 is 130,344,351.

Wait, the cumulative XP for level 98 is 129,619,000. So the difference between level 98 and 99 is 725,351 XP. So the total XP to reach 99 is 130,344,351. Half of that is 65,172,175.5. Let me see the Experience needed for each level and add them up until I reach that half.

Let me see the cumulative XP for each level. Let me check the numbers step by step. Maybe there's a formula or a way to calculate the midpoint. Alternatively, perhaps the user is referring to the level where the XP needed to reach it is half of the XP needed to reach level 99. But the question is phrased as "half the XP of max level," which probably means the cumulative XP for level 99 is the max. So half of that total is 65 million.

Looking at the Experience Table:

Level 1: 0
Level 2: 83
Level 3: 174
Level 4: 276
Level 5: 388
Level 6: 512
Level 7: 650
Level 8: 801
Level 9: 969
Level 10: 1,154
Level 11: 1,358
Level 12: 1,584
Level 13: 1,833
Level 14: 2,107
Level 15: 2,411
Level 16: 2,746
Level 17: 3,115
Level 18: 3,523
Level 19: 3,973
Level 20: 4,470
Level 21: 5,018
Level 22: 5,624
Level 23: 6,291
Level 24: 7,017
Level 25: 7,842
Level 26: 8,740
Level 27: 9,730
Level 28: 10,824
Level 29: 12,031
Level 30: 13,363
Level 31: 14,833
Level 32: 16,456
Level 33: 18,247
Level 34: 20,224
Level 35: 22,406
Level 36: 24,815
Level 37: 27,473
Level 38: 30,408
Level 39: 33,648
Level 40: 37,224
Level 41: 41,171
Level 42: 45,529
Level 43: 50,339
Level 44: 55,649
Level 45: 61,512
Level 46: 67,983
Level 47: 75,127
Level 48: 83,014
Level 49: 91,721
Level 50: 101,333
Level 51: 111,945
Level 52: 123,660
Level 53: 136,594
Level 54: 150,872
Level 55: 166,636
Level 56: 184,040
Level 57: 203,254
Level 58: 224,466
Level 59: 247,886
Level 60: 273,742
Level 61: 302,288
Level 62: 333,804
Level 63: 368,599
Level 64: 407,015
Level 65: 449,428
Level 66: 496,254
Level 67: 547,953
Level 68: 605,032
Level 69: 668,051
Level 70: 737,627
Level 71: 814,465
Level 72: 899,278
Level 73: 992,716
Level 74: 1,096,278
Level 75: 1,210,421
Level 76: 1,336,443
Level 77: 1,475,581
Level 78: 1,629,200
Level 79: 1,798,808
Level 80: 1,986,068
Level 81: 2,192,818
Level 82: 2,421,087
Level 83: 2,673,114
Level 84: 2,951,373
Level 85: 3,258,594
Level 86: 3,597,792
Level 87: 3,972,294
Level 88: 4,385,776
Level 89: 4,842,285
Level 90: 5,346,322
Level 91: 5,902,816
Level 92: 6,517,238
Level 93: 7,195,614
Level 94: 7,944,603
Level 95: 8,771,547
Level 96: 93,255,500
Level 97: 101,108,500
Level 98: 109,218,500
Level 99: 130,344,351

Wait, the total XP needed to reach level 99 is 130,344,351. So half of that is 65,172,175.5. Let me see which level's "Experience needed to reach level" is closest to this.

Looking at the cumulative XP:

Level 80: 1,986,068
Level 90: 5,346,322
Level 99: 130,344,351

Hmm. Wait, the total XP to reach level 99 is 130,344,351. Half of that is 65,172,175.5. Let's see the levels:

Level 49: 91,721
Level 50: 101,333
Level 51: 111,945
Level 52: 123,660
Level 53: 136,594
Level 54: 150,872
Level 55: 166,636
Level 56: 184,040
Level 57: 203,254
Level 58: 224,466
Level 59: 247,886
Level 60: 273,742
Level 61: 302,288
Level 62: 333,804
Level 63: 368,599
Level 64: 407,015
Level 65: 449,428
Level 66: 496,254
Level 67: 547,953
Level 68: 605,032
Level 69: 668,051
Level 70: 737,627
Level 71: 814,465
Level 72: 899,278
Level 73: 992,716
Level 74: 1,096,278
Level 75: 1,210,421
Level 76: 1,336,443
Level 77: 1,475,581
Level 78: 1,629,200
Level 79: 1,798,808
Level 80: 1,986,068
Level 81: 2,192,818
Level 82: 2,421,087
Level 83: 2,673,114
Level 84: 2,951,373
Level 85: 3,258,594
Level 86: 3,597,792
Level 87: 3,972,294
Level 88: 4,385,776
Level 89: 4,842,285
Level 90: 5,346,322
Level 91: 5,902,816
Level 92: 6,517,238
Level 93: 7,195,614
Level 94: 7,944,603
Level 95: 8,771,547
Level 96: 93,255,500
Level 97: 101,108,500
Level 98: 109,218,500
Level 99: 130,344,351

So the half of 130,344,351 is 65,172,175.5. Looking through the levels, the Experience needed to reach level 92 is 6,517,238. Let me check the level 92 entry. Wait, the Experience needed to reach level 92 is 6,517,238. Wait, but the total to reach 99 is 130 million. So the half is 65 million. The cumulative XP at level 92 is 6,517,238. Wait, no, that's not right. Wait, the numbers in the table are cumulative. Let me check again.

Wait, the Experience needed to reach level 99 is 130,344,351. The half of that is 65,172,175.5. Let me check the cumulative XP for each level:

Looking at the numbers:

Level 1: 0
Level 2: 83
Level 3: 174 (0 +83 +91=174)
...
Let me see if the numbers are the total required to reach that level. For example, level 2 requires 83 XP, level 3 requires 174 XP in total, so the difference from level 2 is 174-83=91 XP. So the cumulative XP for each level is the sum from level 1 up to that level.

The total XP for level 99 is 130,344,351. So half is 65,172,175.5. Let me see which level's cumulative XP is closest to that. Let me check the numbers:

Level 99: 130,344,351
Level 98: 129,200,000 (probably)
Level 97: 101,108,500
Wait, the numbers in the table for level 92 is 6,517,238. Wait, that can't be right. Let me check the numbers again.

Wait, the user provided the Experience Table in the Skills/Experience Table section. Let me recheck the numbers. The Experience needed to reach level 99 is 130,344,351. The half is 65,172,175.5. Let me see the levels and their cumulative XP:

Level 1: 0
Level 2: 83
Level 3: 174
Level 4: 276
Level 5: 388
Level 6: 512
Level 7: 650
Level 8: 801
Level 9: 969
Level 10: 1,154
Level 11: 1,358
Level 12: 1,584
Level 13: 1,833
Level 14: 2,107
Level 15: 2,411
Level 16: 2,746
Level 17: 3,115
Level 18: 3,523
Level 19: 3,973
Level 20: 4,470
Level 21: 5,018
Level 22: 5,624
Level 23: 6,291
Level 24: 7,017
Level 25: 7,842
Level 26: 8,740
Level 27: 9,730
Level 28: 10,824
Level 29: 12,031
Level 30: 13,363
Level 31: 14,833
Level 32: 16,456
Level 33: 18,247
Level 34: 20,224
Level 35: 22,406
Level 36: 24,815
Level 37: 27,473
Level 38: 30,408
Level 39: 33,648
Level 40: 37,224
Level 41: 41,171
Level 42: 45,529
Level 43: 50,339
Level 44: 55,649
Level 45: 61,512
Level 46: 67,983
Level 47: 75,127
Level 48: 83,014
Level 49: 91,721
Level 50: 101,333
Level 51: 111,945
Level 52: 123,660
Level 53: 136,594
Level 54: 150,872
Level 55: 166,636
Level 56: 184,040
Level 57: 203,254
Level 58: 224,466
Level 59: 247,886
Level 60: 273,742
Level 61: 302,288
Level 62: 333,804
Level 63: 368,599
Level 64: 407,015
Level 65: 449,428
Level 66: 496,254
Level 67: 547,953
Level 68: 605,032
Level 69: 668,051
Level 70: 737,627
Level 71: 814,465
Level 72: 899,278
Level 73: 992,716
Level 74: 1,096,278
Level 75: 1,210,421
Level 76: 1,336,443
Level 77: 1,475,581
Level 78: 1,629,200
Level 79: 1,798,808
Level 80: 1,986,068
Level 81: 2,192,818
Level 82: 2,421,087
Level 83: 2,673,114
Level 84: 2,951,373
Level 85: 3,258,594
Level 86: 3,597,792
Level 87: 3,972,294
Level 88: 4,385,776
Level 89: 4,842,285
Level 90: 5,346,322
Level 91: 5,902,816
Level 92: 6,517,238
Level 93: 7,195,614
Level 94: 7,944,603
Level 95: 8,771,547
Level 96: 93,255,500
Level 97: 101,108,500
Level 98: 109,218,500
Level 99: 130,344,351

Wait, the Experience needed to reach level 92 is 6,517,238. Wait, that's only 6.5 million, which is way below the half of 130 million. That can't be right. Oh, I must have misread the numbers. The numbers in the table are in the Experience needed to reach level, which is the total XP required to reach that level. For example, level 2 is 83 XP total, level 3 is 174 XP total. But by level 99, it's 130 million. So the halfway point would be at around 65 million XP. Let's see which level has a total of 65 million.

Looking through the levels:

Level 80: 1,986,068 XP (approx 1.98 million)
Level 85: 3,258,594 XP (3.26 million)
Level 90: 5,346,322 XP (5.35 million)
Level 95: 8,771,547 XP (8.77 million)
Level 96: 93,255,500 XP (93 million)
Wait, but the total for level 99 is 130 million. So half is 65 million. Let me check the cumulative XP for each level:

Looking for when the total XP reaches 65 million. Let's see the numbers:

Level 80: 1.98 million
Level 85: 3.26 million
Level 90: 5.35 million
Level 95: 8.77 million
Wait, but the user is asking for when you get to half the XP of max level. The max level's total is 130 million, so half is 65 million. Let's see which level's total is 65 million. The Experience needed to reach level 99 is 130 million. So the halfway point in terms of total XP is 65 million. Let me check the levels:

Level 95 is 8.77 million, which is still below 65 million. Wait, but the numbers in the table for level 95 is 8,771,547. That's only 8.77 million. Wait, there's a mistake here. The numbers in the table must be in the correct order. Let me check the Experience Table again.

Wait, the numbers in the table are formatted with commas, but perhaps the commas are separating thousands and millions. For example, Level 99 is 130,344,351. So 130 million. So the half is 65 million. Let's look for the level where the Experience needed to reach level is around 65 million. Let's check the levels:

Level 80: 1,986,068 (1.986 million)
Level 85: 3,258,594 (3.258 million)
Level 90: 5,346,322 (5.346 million)
Level 95: 8,771,547 (8.771 million)
Level 96: 93,255,500 (93 million)
Wait, but 93 million is already over 65 million. Wait, that can't be right. The numbers in the table for level 95 is 8.77 million. So the cumulative XP for level 95 is 8.77 million. Level 96 is 93 million? That seems like a jump. Wait, maybe the numbers are not cumulative. Wait, the column is labeled "Experience needed to reach level". So for level 96, the total XP needed is 93,255,500. But that's way over the half of 130 million. Wait, perhaps the numbers are formatted with commas as thousand separators. For example, 130,344,351 is 130 million. So 65 million is 65,000,000. Let me check the levels:

Looking for the level where the Experience needed is around 65,000,000. Let me see the numbers in the table:

Level 90: 5,346,322 (5.346 million)
Level 91: 5,902,816 (5.902 million)
Level 92: 6,517,238 (6.517 million)
Level 93: 7,195,614 (7.195 million)
So the Experience needed to reach level 92 is 6,517,238, which is approximately 6.5 million. The max is 130 million. So the halfway point is 65 million. So the level where the cumulative XP is 65 million is somewhere between level 92 and 93. But in the table, level 92 is 6.5 million. Wait, that's not possible. Unless the numbers in the table are not in the correct order. Let me check the table again.

Wait, the Experience needed to reach level 99 is 130,344,351. So the half would be at 65,172,175.5. Looking at the table:

Level 92: 6,517,238 (6.517 million)
Level 93: 7,195,614 (7.195 million)
Level 94: 7,944,603 (7.944 million)
Level 95: 8,771,547 (8.771 million)
Level 96: 93,255,500 (93.255 million)
Level 97: 101,108,500 (101.108 million)
Level 98: 109,218,500 (109.218 million)
Level 99: 130,344,351 (130.344 million)

Wait, there's a jump from level 95 to 96. That's not possible. Maybe the numbers are incorrect in the table. Let me check the original text. In the Experience Table, the entries for levels 96 and 97 show:

Level 96: 93255500
Level 97: 101108500
Level 98: 109218500
Level 99: 130344351

Wait, the numbers for level 96 is 93,255,500, level 97 is 101,108,500, level 98 is 109,218,500, and level 99 is 130,344,351. So the cumulative XP to reach level 96 is 93 million, which is more than 65 million. Therefore, the halfway point is somewhere between level 95 and 96. But according to the table, the XP needed to reach level 95 is 8.77 million, and level 96 is 93 million. That's a big jump. So the halfway point would be at level 95.5, but since levels are integers, it's likely around level 95. But the user is asking for the level when you get to half the XP of max. So the answer is around level 95.5, but since there's no half level, the closest is level 95 (8.77 million) and level 96 (93 million). The half of 130 million is 65 million. So level 95 is 8.77 million, which is 8.77 million. The difference between level 95 and 96 is 93 million - 8.77 million = 84.485 million. So the halfway point is 65 million. So between level 95 and 96. To find the exact level, we can see how much XP is needed to reach 65 million. Let's see:

Total XP to reach level 95 is 8,771,547. So the remaining XP from 95 to 96 is 93,255,500 - 8,771,547 = 84,483,953 XP. To reach 65 million, which is 65,000,000. The difference between 65 million and 8.771 million is 65,000,000 - 8,771,547 = 56,228,453 XP needed after level 95. Since the XP from 95 to 96 is 84.48 million, but you need only 56 million to reach the halfway. So the halfway point is at level 95 plus 56 million XP. But the XP gained per level 96 is 84 million. So the halfway point is at level 95 + 56 million / (84.48 million per XP). So the level is 95 + 56/84.48 ≈ 95 + 0.66 = 95.66. So approximately level 95.66. But since levels are whole numbers, the closest is level 95 and 96. But the question is asking for the level at which you get to half the XP of max level. So the answer is that you reach half the XP at level 95, which is 8.77 million, which is less than half. The halfway point in terms of XP would be at a level where the total XP is 65 million. Since the table shows that level 96 requires 93 million, and level 95 is 8.77 million, there's a big jump. Maybe the table is incorrect. Alternatively, perhaps the numbers are in the format without commas, like 93255500 is 93 million. So the total XP for level 95 is 8,771,547. So the halfway is at 65 million. So between level 95 and 96. Therefore, the answer is that you reach half the XP needed to reach max level at around level 95.66, which is not an integer. But in the game, you can't be at a fractional level, so the closest is level 95. However, the table might have an error. Alternatively, maybe the numbers are cumulative, and the halfway is at a lower level. Wait, let me check the Experience Table again. The user provided the table up to level 99, but perhaps the numbers for higher levels are cumulative. Wait, the total XP for level 99 is 130 million. The halfway is 65 million. Let's see which level's cumulative XP is 65 million. From the table:

Level 92: 6,517,238 XP (6.517 million)
Level 93: 7,195,614 XP (7.195 million)
Level 94: 7,944,603 XP (7.944 million)
Level 95: 8,771,547 XP (8.771 million)

Wait, the Experience needed to reach level 92 is 6.517 million. So the halfway point is at level 92. Because 6.517 million is about 65% of the total 130 million. So 6.517 million is about half. Wait, 6.517 million is roughly half of 130 million. So the answer is level 92. Because the total XP needed to reach level 99 is 130 million, and the halfway is 65 million. The level 92 is 6.517 million, which is approximately half. So the answer is that you reach half the XP of max level at level 92. But I need to confirm this. Let me do the math:

Total XP for level 99: 130,344,351.

Half of that is 65,172,175.5.

Looking at the Experience Table:

Level 92: 6,517,238 XP. Wait, 6,517,238 is 6.517 million, which is 6.517 million. The half is 65 million. So 6.517 million is 65.17 million, which is 6517238. So 6.517 million is 6517238, which is 65.17 million. The half is 65.172 million. So level 92 is 65.172 million, which is very close to the half. So the answer is level 92. So when you reach level 92, you've earned about half the XP required to reach max level 99.

But wait, the Experience Table shows level 92 as 6,517,238 XP. So 6,517,238 is 6.517 million. The total XP for level 99 is 130 million. So 6.517 million is about half. So the answer is level 92. Therefore, the user is asking what level you are at when you get to half the XP of max level. The answer is level 92.
</think>

In Old School RuneScape, the total experience (XP) required to reach the maximum level of 99 is **130,344,351 XP**. Half of this amount is **65,172,175.5 XP**. 

According to the **Experience Table** provided in the guide, the cumulative XP needed to reach **level 92** is **6,517,238 XP** (6.517 million). This is the closest integer level to the halfway point of the total XP for level 99. 

### Key Details:
- **Level 92** corresponds to **6,517,238 XP**, which is approximately 50% of the total XP for level 99 (130 million).
- The jump from level 95 to 96 in the table appears to be a formatting error (likely a misplaced comma), as the cumulative XP for level 92 already reaches ~6.5 million, which is about half of the total.
- The table confirms that level 92 is the midpoint in terms of XP, as the values for higher levels (93–99) are significantly higher due to the exponential XP curve.

### Final Answer:
**Level 92** is the approximate level at which you reach half the total XP needed to achieve the maximum level of 99 in Old School RuneScape.

This was painfully slow.

{   'temperature': 0.6,
    'top_p': 0.95,
    'top_k': 20}
Output generated in 3966.19 seconds (2.20 tokens/s, 8726 tokens, context 107202, seed 1866517172)

It still spat wrong numbers in its reasoning such as Level 99: 130,344,351. I'll do a couple more rounds.

Oh yeah! 8bpw should be basically lossless. I think Q6 cache should be fine too, but fp16 may be faster.

Slow is right.

Yeah, numbers and math seem hard. This is difficult to explain, but I suspect YaRN makes it “skip” some tokens or get them confused at a fine-grained level, like it makes Qwen “myopic” even though it sees and understands the context.

I will be testing it on story continuation and comprehension some.

Finally got some positive results! I was indeed trusting my UI too much and had to revisit each load param, the main issue was that I was using compress_pos_emb 4 instead of 1 and wasn't loading the model with yarn.

EXL2/EXL3 do not work well with YaRN (at least I could never get any decent output with the correct answer).

GGUF with llama.cpp works most of the time (I did 4 runs, 3 were good, one miserably failed stating that 69 was the correct answer...).

Qwen3-32B-128K-Q8_0.gguf loaded with fp16 cache, compress_pos_emb 1 with the following options:
--rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768

Mostly gives the right answer and doesn't always seem to hallucinate numbers:

• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_0.txt (win)
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_1.txt (win)
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_2.txt (fail)
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_3.txt (win)

The unsloth's UD version of the model isn't giving the correct answer and is having a hard time with the commas. Example: "user's table shows level 99 with an XP needed of 130,344,31" - https://huggingface.co/unsloth/Qwen3-32B-128K-GGUF/discussions/4.

@Thireus Re on the larger Q8_K_XL quants vs Q8_0 - I did a 12K in length calibration dataset, so it's possible the 16bit upcasted layers are not actually calibrated - I forgot though if Q8_0 does in fact get affected via imatrix as well. That might be one explanation.

I'm unsure if it's a hard ask, but would it be possible to test the pure BF16 version?

@danielhanchen , I probably should have done more runs. Here is what I've got so far:

• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_1.txt - WIN (self-corrected) - Self-corrected mixed up table rows
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_2.txt - FAIL - Slightly incorrect numbers / mixed with comma issue, such as "70 is 7,376,270"
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_3.txt - WIN (self-corrected) - Self-corrected initial incorrect numbers
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_4.txt - WIN
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_5.txt - WIN
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_6.txt - FAIL - Comma issue, such as "level 99 with 130,344,310"
• https://thireus.com/REDDIT/Qwen3-32B-128K-Q8_0.gguf_proper_params_7.txt - WIN (self-corrected) - Self-corrected initial incorrect numbers

6 WIN of which 3 are SELF-CORRECTED (self-corrected to WIN), 2 FAIL
General observation: On average the model produced 1,718 new tokens (less than the UD version) and was able to self-correct to find the correct answer. When FAIL happens, the reason can be explained because of "genuine" mistakes such as comma-related or mixing up a table row for another.

• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_0.txt - WIN
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_1.txt - FAIL - Incorrect numbers pulled from the table, such as "Level 95: 6,692,219" which appears nowhere.
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_2.txt - WIN
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_3.txt - WIN
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_4.txt - FAIL - Comma issue and incorrect numbers, such as "Level 60: 73,762.7" or "level 99 is 130,344" (partial self-correction happened)
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_5.txt - FAIL - Incorrect numbers pulled from the table, example: "Level 84: 3,278,259" which appears nowhere.
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_6.txt - WIN
• https://thireus.com/REDDIT/Qwen3-32B-128K-UD-Q8_K_XL.gguf_proper_params_7.txt - WIN

5 WIN of which 0 are SELF-CORRECTED (self-corrected to WIN), 3 FAIL
General observation: On average the model produced 2,500 new tokens (more than the non-UD version) and was never able to self-correct its mistakes and often appears to hallucinate numbers.

I did several env upgrades since yesterday, including Flash_Attention and NVidia drivers, hence the speed improvements.

To run the BF16 GGUF, I've had to unload several layers to the CPU. All other loading configs remain the same, that includes cache which is fp16.

• https://thireus.com/REDDIT/Qwen3-32B-128K-BF16.gguf_proper_params_0.txt - FAIL - Mixed up table rows
• https://thireus.com/REDDIT/Qwen3-32B-128K-BF16.gguf_proper_params_1.txt - WIN (self-corrected) - Self-corrected initial incorrect numbers pulled from the table
• https://thireus.com/REDDIT/Qwen3-32B-128K-BF16.gguf_proper_params_2.txt - WIN

I'm tempted to think the quality issue comes from the original model, not the quantized versions. Although, it seems strange that the UD version was not able to self-correct... but that remains quite inconclusive as too much randomness is involved.

With temperature, it feels like a shot in the dark for Qwen to "pick" all the right numbers for the math. Its confidence in those number tokens has to overcome a ton of randomness.

At zero or extremely low temperature, getting/not getting stuck in a grammer loop also feels like luck.

I'm just amazed it's recognized at all. My experience with poor "long context" models like Phi or Mistral is that they outright ignore parts of the context, or get increasingly incoherent out there. Qwen3 32B plucks the table out, but just fumbles it with the thinking's randomness from what I've seen.