KeyError: 'rope_type'
#7
by
Rolfie33
- opened
Are the docs correct for vllm with the rope type? I got an error when i use:
vllm serve ... --rope-scaling '{"type":"rope_type","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072
While the following works well:
vllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072
you are correct, it is "rope_type":"yarn"
jklj077
changed discussion status to
closed