What is the best way to run saved model with different batch size in TensorFlow?
<p>I trained <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/models/image/cifar10" rel="noreferrer">Cifar10 example model from TensorFlow's repository</a> with batch_size 128 and it worked fine. Then I <a href="https://github.com/tensorflow/tensorflow/blob/816ecb7a342c25c19daa8b19627346e1fb38e56f/tensorflow/python/tools/freeze_graph.py" rel="noreferrer">froze graph</a> and managed to run it with C++ just like they do it in their <a href="https://github.com/tensorflow/tensorflow/tree/754048a0453a04a761e112ae5d99c149eb9910dd/tensorflow/examples/label_image" rel="noreferrer">C++ label image example</a>.</p>

<p>The only problem was that I had to artificially generate tensor of shape [128, image_height, image_width, channels] to classify single image with C++ because saved model expects input of 128 samples in a batch since that is number of samples that comes from queue.</p>

<p>I tried training Cifar10 example with batch_size = 1 and then I managed to      classify examples one by one when I run model with C++, but that doesn't seem like a great solution. I also tried manually changing tensor shapes in saved graph file but it didn't work.</p>

<p>My question is what is the best way to train model with fixed batch size (like 32, 64, 128 etc.) and then save model so that it can be used with batch size of arbitrary length? If that's not possible, then how to save model to be able to classify samples one by one.</p>