Qwen

/

Qwen3-30B-A3B

like 305

Follow

Qwen 28.3k

Command used:

CUDA_VISIBLE_DEVICES=0,2 vllm serve Qwen/Qwen3-30B-A3B -tp 2 --enable-reasoning --reasoning-parser deepseek_r1 --rope-scaling '{"type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072 --gpu-memory-utilization 0.97

Error + full traceback:

INFO 04-28 18:14:25 [config.py:456] Overriding HF config with {'rope_scaling': {'type': 'yarn', 'factor': 4.0, 'original_max_position_embeddings': 32768}}
Traceback (most recent call last):
  File "/root/anaconda3/envs/trainer/bin/vllm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/entrypoints/cli/main.py", line 53, in main
    args.dispatch_function(args)
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/entrypoints/cli/serve.py", line 27, in cmd
    uvloop.run(run_server(args))
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/uvloop/__init__.py", line 105, in run
    return runner.run(wrapper())
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
           ^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1078, in run_server
    async with build_async_engine_client(args) as engine_client:
  File "/root/anaconda3/envs/trainer/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 146, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
  File "/root/anaconda3/envs/trainer/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 166, in build_async_engine_client_from_engine_args
    vllm_config = engine_args.create_engine_config(usage_context=usage_context)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 1099, in create_engine_config
    model_config = self.create_model_config()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 987, in create_model_config
    return ModelConfig(
           ^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/config.py", line 508, in __init__
    self.max_model_len = _get_and_verify_max_len(
                         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/trainer/lib/python3.11/site-packages/vllm/config.py", line 3054, in _get_and_verify_max_len
    rope_type = rope_scaling["rope_type"]
                ~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'rope_type'