Please help - error when using Triton Inference Ensemble
I've been trying to get these models to work in an ensemble, but im running into this error:
Inference failed: [StatusCode.INTERNAL] in ensemble 'phi_3_5_ensemble', onnx runtime error 6: Non-zero status code returned while running Loop node. Name:'Loop_3924' Status Message: Non-zero status code returned while running Reshape node. Name:'Reshape_4005' Status Message: /workspace/onnxruntime/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:39 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) size != 0 && (input_shape_size % size) == 0 was false. The input tensor cannot be reshaped to the requested shape. Input shape:{0,12,12,2,2,1024}, requested shape:{0,-1,4096}
It's really tricky setting up the config.pbtxt files for each of the models and the ensemble, and I think I have them correct, but it gives me the error above.
Has anyone got these models to work within Triton? If so please help! :)
I'm not sure if this is related but I also see an error when running:
onnx.checker.check_model("phi-3.5-v-instruct-vision.onnx")
onnx.onnx_cpp2py_export.checker.ValidationError: Nodes in a graph must be topologically sorted, however input '/Reshape_1_output_0' of node:
name: Gather_3939 OpType: Gather is not output of any previous nodes.