How to make request in AWS SageMaker for this model? And will it support vllm?
#47
by
xupeng1023
- opened
How can I make request to this model if I would like to host the model in AWS SageMaker? Let's say I would like to give a video input, how can I do that?
The following request is not working.
predictor.predict(
{
"inputs": "What is is the capital of France?",
"parameters": {
"do_sample": True,
"max_new_tokens": 128,
"temperature": 0.7,
"top_k": 50,
"top_p": 0.95,
}
}
)