Skip to content

Instantly share code, notes, and snippets.

@bmabir17
Last active February 6, 2022 20:54
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bmabir17/754a6e0450ec4fd5e25e462af949cde6 to your computer and use it in GitHub Desktop.
Save bmabir17/754a6e0450ec4fd5e25e462af949cde6 to your computer and use it in GitHub Desktop.
Converts the mask-rcnn keras model https://github.com/matterport/Mask_RCNN/releases/tag/v2.0 to tflite
import tensorflow as tf
import numpy as np
import mrcnn.model as modellib # https://github.com/matterport/Mask_RCNN/
from mrcnn.config import Config
import keras.backend as keras
PATH_TO_SAVE_FROZEN_PB ="./"
FROZEN_NAME ="saved_model.pb"
def load_model(Weights):
global model, graph
class InferenceConfig(Config):
NAME = "coco"
NUM_CLASSES = 1 + 80
IMAGE_META_SIZE = 1 + 3 + 3 + 4 + 1 + NUM_CLASSES
DETECTION_MAX_INSTANCES = 100
DETECTION_MIN_CONFIDENCE = 0.7
DETECTION_NMS_THRESHOLD = 0.3
GPU_COUNT = 1
IMAGES_PER_GPU = 1
config = InferenceConfig()
Weights = Weights
Logs = "./logs"
model = modellib.MaskRCNN(mode="inference", config=config,
model_dir=Logs)
model.load_weights(Weights, by_name=True)
graph = tf.get_default_graph()
# Reference https://github.com/bendangnuksung/mrcnn_serving_ready/blob/master/main.py
def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
graph = session.graph
with graph.as_default():
freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
output_names = output_names or []
input_graph_def = graph.as_graph_def()
if clear_devices:
for node in input_graph_def.node:
node.device = ""
frozen_graph = tf.graph_util.convert_variables_to_constants(
session, input_graph_def, output_names, freeze_var_names)
return frozen_graph
def freeze_model(model, name):
frozen_graph = freeze_session(
sess,
output_names=[out.op.name for out in model.outputs][:4])
directory = PATH_TO_SAVE_FROZEN_PB
tf.train.write_graph(frozen_graph, directory, name , as_text=False)
def keras_to_tflite(in_weight_file, out_weight_file):
sess = tf.Session()
keras.set_session(sess)
load_model(in_weight_file)
global model
freeze_model(model.keras_model, FROZEN_NAME)
# https://github.com/matterport/Mask_RCNN/issues/2020#issuecomment-596449757
input_arrays = ["input_image"]
output_arrays = ["mrcnn_class/Softmax","mrcnn_bbox/Reshape"]
converter = tf.contrib.lite.TocoConverter.from_frozen_graph(
PATH_TO_SAVE_FROZEN_PB+"/"+FROZEN_NAME,
input_arrays, output_arrays,
input_shapes={"input_image":[1,256,256,3]}
)
converter.target_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,tf.lite.OpsSet.SELECT_TF_OPS]
converter.post_training_quantize = True
tflite_model = converter.convert()
open(out_weight_file, "wb").write(tflite_model)
print("*"*80)
print("Finished converting keras model to Frozen tflite")
print('PATH: ', out_weight_file)
print("*" * 80)
keras_to_tflite("./mask_rcnn_coco.h5","./mask_rcnn_coco.tflite")
@bmabir17
Copy link
Author

@bmabir17
previous issue was solved it was some session error i was facing , can you help me with this . i have converted the epoch 10 .h5 in to . pb no wile converting it into .tflte is not working and giving some error like this
UnicodeDecodeError Traceback (most recent call last)
in ()
4 export_dir1 = "/content/drive/My Drive/NumberPlateDetection/saved_model.pb"
5 with open(export_dir1, 'r', encoding="utf8") as f:
----> 6 save_file = f.read()
7 input_arrays = ["input_image"]
8 #input_arrays = ["input_image","input_anchors","input_image_meta"]

/usr/lib/python3.6/codecs.py in decode(self, input, final)
319 # decode input (taking the buffer into account)
320 data = self.buffer + input
--> 321 (result, consumed) = self._buffer_decode(data, self.errors, final)
322 # keep undecoded input until the next call
323 self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 46: invalid start byte

Your model was not saved properly or your model file has become corrupted

@Rubeen-Mohammad
Copy link

I'm getting this error :( please help me..........................................

FailedPreconditionError: 2 root error(s) found.
(0) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[{{node bn4a_branch2a_8/gamma}}]]
[[res3d_branch2a_8/bias/_2417]]
(1) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[{{node bn4a_branch2a_8/gamma}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

FailedPreconditionError Traceback (most recent call last)
/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _do_call(self, fn, *args)
1382 '\nsession_config.graph_options.rewrite_options.'
1383 'disable_meta_optimizer = True')
-> 1384 raise type(e)(node_def, op, message)
1385
1386 def _extend_graph(self):

FailedPreconditionError: 2 root error(s) found.
(0) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[node bn4a_branch2a_8/gamma (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
[[res3d_branch2a_8/bias/_2417]]
(1) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[node bn4a_branch2a_8/gamma (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.

Original stack trace for 'bn4a_branch2a_8/gamma':
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 664, in launch_instance
app.start()
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelapp.py", line 499, in start
self.io_loop.start()
File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 456, in _handle_events
self._handle_recv()
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 486, in _handle_recv
self._run_callback(callback, msg)
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 438, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2828, in run_ast_nodes
if self.run_code(code, result):
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 81, in
keras_to_tflite("./mask_rcnn_coco.h5","./mask_rcnn_coco.tflite")
File "", line 61, in keras_to_tflite
load_model(in_weight_file)
File "", line 27, in load_model
model_dir=Logs)
File "/content/Mask_RCNN/mrcnn/model.py", line 1837, in init
self.keras_model = self.build(mode=mode, config=config)
File "/content/Mask_RCNN/mrcnn/model.py", line 1901, in build
stage5=True, train_bn=config.TRAIN_BN)
File "/content/Mask_RCNN/mrcnn/model.py", line 194, in resnet_graph
x = conv_block(x, 3, [256, 256, 1024], stage=4, block='a', train_bn=train_bn)
File "/content/Mask_RCNN/mrcnn/model.py", line 150, in conv_block
x = BatchNorm(name=bn_name_base + '2a')(x, training=train_bn)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 576, in call
self.build(input_shapes[0])
File "/usr/local/lib/python3.6/dist-packages/keras/layers/normalization.py", line 103, in build
constraint=self.gamma_constraint)
File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 400, in add_weight
constraint=constraint)
File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 380, in variable
v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 258, in call
return cls._variable_v1_call(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 197, in
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variable_scope.py", line 2519, in default_variable_creator
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 262, in call
return super(VariableMetaclass, cls).call(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 1688, in init
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 1846, in _init_from_args
shape, self._initial_value.dtype.base_dtype, name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/state_ops.py", line 79, in variable_op_v2
shared_name=shared_name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/gen_state_ops.py", line 1621, in variable_v2
shared_name=shared_name, name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

Hi @Jainam0! I'm getting this issue again and again, but in different lines, can you please help me in resolving this?
Thank you

@Jainam0
Copy link

Jainam0 commented Nov 26, 2020

I'm getting this error :( please help me..........................................
FailedPreconditionError: 2 root error(s) found.
(0) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[{{node bn4a_branch2a_8/gamma}}]]
[[res3d_branch2a_8/bias/_2417]]
(1) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[{{node bn4a_branch2a_8/gamma}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
FailedPreconditionError Traceback (most recent call last)
/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py in _do_call(self, fn, *args)
1382 '\nsession_config.graph_options.rewrite_options.'
1383 'disable_meta_optimizer = True')
-> 1384 raise type(e)(node_def, op, message)
1385
1386 def _extend_graph(self):
FailedPreconditionError: 2 root error(s) found.
(0) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[node bn4a_branch2a_8/gamma (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
[[res3d_branch2a_8/bias/_2417]]
(1) Failed precondition: Attempting to use uninitialized value bn4a_branch2a_8/gamma
[[node bn4a_branch2a_8/gamma (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'bn4a_branch2a_8/gamma':
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 664, in launch_instance
app.start()
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelapp.py", line 499, in start
self.io_loop.start()
File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 456, in _handle_events
self._handle_recv()
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 486, in _handle_recv
self._run_callback(callback, msg)
File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 438, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2828, in run_ast_nodes
if self.run_code(code, result):
File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 81, in
keras_to_tflite("./mask_rcnn_coco.h5","./mask_rcnn_coco.tflite")
File "", line 61, in keras_to_tflite
load_model(in_weight_file)
File "", line 27, in load_model
model_dir=Logs)
File "/content/Mask_RCNN/mrcnn/model.py", line 1837, in init
self.keras_model = self.build(mode=mode, config=config)
File "/content/Mask_RCNN/mrcnn/model.py", line 1901, in build
stage5=True, train_bn=config.TRAIN_BN)
File "/content/Mask_RCNN/mrcnn/model.py", line 194, in resnet_graph
x = conv_block(x, 3, [256, 256, 1024], stage=4, block='a', train_bn=train_bn)
File "/content/Mask_RCNN/mrcnn/model.py", line 150, in conv_block
x = BatchNorm(name=bn_name_base + '2a')(x, training=train_bn)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 576, in call
self.build(input_shapes[0])
File "/usr/local/lib/python3.6/dist-packages/keras/layers/normalization.py", line 103, in build
constraint=self.gamma_constraint)
File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 400, in add_weight
constraint=constraint)
File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 380, in variable
v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 258, in call
return cls._variable_v1_call(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 197, in
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variable_scope.py", line 2519, in default_variable_creator
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 262, in call
return super(VariableMetaclass, cls).call(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 1688, in init
shape=shape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/variables.py", line 1846, in _init_from_args
shape, self._initial_value.dtype.base_dtype, name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/state_ops.py", line 79, in variable_op_v2
shared_name=shared_name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/gen_state_ops.py", line 1621, in variable_v2
shared_name=shared_name, name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

Hi @Jainam0! I'm getting this issue again and again, but in different lines, can you please help me in resolving this?
Thank you

tensorflow/tensorflow#28287 (comment) , i solved that from here but couldn't able to convert it into .tflite , i had only converted ssd models into .tflite

@lbininhbl
Copy link

Hi @bmabir17
Thanks your code that I can convert the tflite successfully. But when I ran the tflite on iOS, I got bellow error
TensorFlow Lite Error: Input tensor 314 lacks data

I look through the code and notice this code in keras_to_tflite function:
input_arrays = ["input_image"] output_arrays = ["mrcnn_class/Softmax","mrcnn_bbox/Reshape"]

But the real keras model needs more inputs when predict function was executed. I don't know if that has anything to do with it. What do think?

@kaamlaS
Copy link

kaamlaS commented Jan 27, 2022

hi @bmabir17
I cant seem to move ahead from this point
ValueError: Invalid tensors 'input_image' were found.
i checked the model using model.keras_model.summary() and it shows that the first layer has the name input_image. What do i do?

@Tubhalooter
Copy link

@kaamlaS i got past this , check this out matterport/Mask_RCNN#2020

let me know if you get it working
also i cant get the output names to work so i left it out but that is stopping the conversion , how did you pass the output names to the parameter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment