How to convert .pb to TFLite format?

I downloaded a retrained_graph.pb and retrained_labels.txt file of a model I trained in Azure cognitive service. Now I want to make an Android app using that model and to do so I have to convert it to TFLite format. I used toco and I am getting the following error:

ValueError: Invalid tensors 'input' were found.

I am basically following this tutorial and have problem on step 4 and direcly copy pasted the terminal code: https://heartbeat.fritz.ai/neural-networks-on-mobile-devices-with-tensorflow-lite-a-tutorial-85b41f53230c

Solution 1:

I am making a wild guess here, maybe you entered input_arrays=input. Which may not be true. Use this script to find the name of the input and output arrays of the frozen inference graph

import tensorflow as tf
gf = tf.GraphDef()   
m_file = open('frozen_inference_graph.pb','rb')
gf.ParseFromString(m_file.read())

with open('somefile.txt', 'a') as the_file:
    for n in gf.node:
        the_file.write(n.name+'\n')

file = open('somefile.txt','r')
data = file.readlines()
print "output name = "
print data[len(data)-1]

print "Input name = "
file.seek ( 0 )
print file.readline()

In my case they are:

output name: SemanticPredictions
input name: ImageTensor

Solution 2:

You can use utility tflite_convert which is the part of tensorflow 1.10 (or higher) package.

The simple use for float inference is something like:

tflite_convert \
    --output_file=/tmp/retrained_graph.tflite \
    --graph_def_file=/tmp/retrained_graph.pb \
    --input_arrays=input \
    --output_arrays=output

Where input and output - are input and ouput tensors of your tensorflow graph

Solution 3:

import tensorflow as tf
gf = tf.GraphDef()
m_file = open('frozen_inference_graph.pb','rb')
for n in gf.node:
    print( n.name )

first one is input_arrays last names are output_arrays (could be more than one depends on your number of output of the model)

my output

image_tensor <--- input_array
Cast
Preprocessor/map/Shape Preprocessor/map/strided_slice/stack
Preprocessor/map/strided_slice/stack_1
.
.
.
Postprocessor/BatchMultiClassNonMaxSuppression/map/
TensorArrayStack_5/TensorArrayGatherV3
Postprocessor/Cast_3
Postprocessor/Squeeze
add/y
add
detection_boxes <---output_array
detection_scores <---output_array
detection_multiclass_scores
detection_classes <---output_array
num_detections <---output_array
raw_detection_boxes
raw_detection_scores

Solution 4:

Most of the answers here prove to be broken due to the version issues. This worked for me:

Note: First find the name of the input and output layers using Netron, as I mentioned here. In my case they are input and output.

!pip install tensorflow-gpu==1.15.0

# Convert
!toco --graph_def_file /content/yolo-v2-tiny-coco.pb \
    --output_file yolo-v2-tiny-coco.tflite \
    --output_format TFLITE \
    --inference_type FLOAT \
    --inference_input_type FLOAT \
    --input_arrays input \
    --output_arrays output

Also, as per zldrobit's amazing work, you can also fetch a better quantized version of this TFLite model as:


# Now let's quantize it
!toco --graph_def_file /content/yolo-v2-tiny-coco.pb \
    --output_file quantized-yolo-v2-tiny-coco.tflite \
    --output_format TFLITE \
    --inference_type FLOAT \
    --inference_input_type FLOAT \
    --input_arrays input \
    --output_arrays output \
    --post_training_quantize

Solution 5:

The error hints that you have not entered the correct

--input_arrays

From TF Lite Developer Guide I quote :

"Setting the input_array and output_array arguments is not straightforward. The easiest way to find these values is to explore the graph using TensorBoard."

Using the Tensorboard isn't hard either, by simply running this command

tensorboard --logdir=path/to/log-directory

View the TensorBoard at

localhost:6006