object_detection_client.py

# Create stub
host, port = FLAGS.server.split(':')
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)

# Create prediction request object
request = predict_pb2.PredictRequest()

# Specify model name (must be the same as when the TensorFlow serving serving was started)
request.model_spec.name = 'obj_det'

# Initalize prediction 
# Specify signature name (should be the same as specified when exporting model)
request.model_spec.signature_name = "detection_signature"
request.inputs['inputs'].CopyFrom(
        tf.contrib.util.make_tensor_proto({FLAGS.input_image}))

# Call the prediction server
result = stub.Predict(request, 10.0)  # 10 secs timeout

# Plot boxes on the input image
category_index = load_label_map(FLAGS.path_to_labels)
boxes = result.outputs['detection_boxes'].float_val
classes = result.outputs['detection_classes'].float_val
scores = result.outputs['detection_scores'].float_val
image_vis = vis_util.visualize_boxes_and_labels_on_image_array(
    FLAGS.input_image,
    np.reshape(boxes,[100,4]),
    np.squeeze(classes).astype(np.int32),
    np.squeeze(scores),
    category_index,
    use_normalized_coordinates=True,
    line_thickness=8)

# Save inference to disk
scipy.misc.imsave('%s.jpg'%(FLAGS.input_image), image_vis)

need help on this as well. Where are all the relevant imports to make this work?

Hello. I try to follow your tutorial, but throw error in this line "tf.contrib.util.make_tensor_proto({FLAGS.input_image})",
error is "TypeError: Failed to convert object of type <class 'set'> to Tensor. Contents: {'path/test1.jpg'}. Consider casting elements to a supported type."

definitios in code
tf.app.flags.DEFINE_string('server', 'localhost:3000', 'PredictionService localhost:3000')
tf.app.flags.DEFINE_string('input_image', 'path/test1.jpg', 'Path to image in JPEG format')
FLAGS = tf.app.flags.FLAGS

I'm jumping something?

Hi,
I'm doing this tutorial "https://medium.freecodecamp.org/how-to-deploy-an-object-detection-model-with-tensorflow-serving-d6436e65d1d9".
I have this code.

import tensorflow as tf
import object_detection.exporter
from object_detection.utils.config_util import create_pipeline_proto_from_configs
from object_detection.utils.config_util import get_configs_from_pipeline_file

Configuration for model to be exported

config_pathname = "C:/tmp/pineapple/0.001/500/"

Input checkpoint for the model to be exported

Path to the directory which consists of the saved model on disk (see above)

trained_model_dir = "C:/tmp/pineapple/0.001/500/model.ckpt"

Create proto from model confguration

configs = get_configs_from_pipeline_file(config_pathname)
pipeline_proto = create_pipeline_proto_from_configs(configs=configs)

Read .ckpt and .meta files from model directory

checkpoint = tf.train.get_checkpoint_state(trained_model_dir)
input_checkpoint = checkpoint.model_checkpoint_path

Model Version

model_version_id = 1

Output Directory

output_directory = "c:/output_model/" + str(model_version_id) + "/"

Export model for serving

object_detection.exporter.export_inference_graph(input_type='image_tensor',pipeline_config=pipeline_proto,trained_checkpoint_prefix=input_checkpoint,output_directory=output_directory)

and found these errors:

C:>python C:\Users\Michael\Desktop\my_model.py
Traceback (most recent call last):
File "C:\Users\Michael\Desktop\my_model.py", line 15, in
configs = get_configs_from_pipeline_file(config_pathname)
File "C:\ProgramData\Anaconda2\lib\site-packages\object_detection-0.1-py3.6.egg\object_detection\utils\config_util.py", line 92, in get_configs_from_pipeline_file
proto_str = f.read()
File "C:\ProgramData\Anaconda2\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 119, in read
self._preread_check()
File "C:\ProgramData\Anaconda2\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 79, in _preread_check
compat.as_bytes(self.__name), 1024 * 512, status)
File "C:\ProgramData\Anaconda2\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.UnknownError: NewRandomAccessFile failed to Create/Open: C:/tmp/pineapple/0.001/500/ : Access is denied.
; Input/output error

Can anyone help me this error.

Thanks
Michael

@yavac

I'm not sure why it's written this way...maybe it worked in older version of tensorflow. In tensorflow 1.10, make_tensor_proto expects a list or numpy array. In the example above, the file location is being passed in a set. I converted image to numpy array like using: scipy.misc.imread(FLAGS.input_image)
I then got another error, because the shape of the array wasn't passed. The full solution is to replace make_tensor_proto(FLAGS.input_image) with make_tensor_proto(scipy.misc.imread(FLAGS.input_image), shape=[1] + list(img.shape))

https://www.tensorflow.org/api_docs/python/tf/make_tensor_proto

This all took me a long time to figure out. Hope it saves the next person some time.

@abatkins much thanks for this, i was stuck on this for a long time. 👍

I'm getting failed prediction error. Any ideas?

python object_detection_client.py -server=localhost:8500 -input_image=/home/osteinnes/Fy01_Fv112_hp01_f1_m05917.jpg -path_to_labels=/home/osteinnes/programming/rcnn-p18-object-detection/label.pbtxt
/home/osteinnes/.conda/envs/doNotDelete/lib/python3.6/site-packages/tensorflow_serving/apis/prediction_service_pb2.py:131: DeprecationWarning: beta_create_PredictionService_stub() method is deprecated. This method will be removed in near future versions of TF Serving. Please switch to GA gRPC API in prediction_service_pb2_grpc.
  'prediction_service_pb2_grpc.', DeprecationWarning)
object_detection_client.py:31: DeprecationWarning: `imread` is deprecated!
`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.
Use ``imageio.imread`` instead.
  img = scipy.misc.imread(FLAGS.input_image)
/home/osteinnes/.conda/envs/doNotDelete/lib/python3.6/site-packages/scipy/misc/pilutil.py:165: DeprecationWarning: `fromimage` is deprecated!
`fromimage` is deprecated in SciPy 1.0.0. and will be removed in 1.2.0.
Use ``np.asarray(im)`` instead.
  return fromimage(im, flatten=flatten, mode=mode)
/home/osteinnes/.conda/envs/doNotDelete/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:75: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  return _inspect.getargspec(target)
Traceback (most recent call last):
  File "object_detection_client.py", line 35, in <module>
    result = stub.Predict(request, 10.0)  # 10 secs timeout
  File "/home/osteinnes/.conda/envs/doNotDelete/lib/python3.6/site-packages/grpc/_channel.py", line 550, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/osteinnes/.conda/envs/doNotDelete/lib/python3.6/site-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.FAILED_PRECONDITION
        details = "Serving signature key "model" not found."
        debug_error_string = "{"created":"@1550853885.188677522","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1036,"grpc_message":"Serving signature key "model" not found.","grpc_status":9}"

EDIT: Figured it out. Even though I specified the model signature when exporting the re-trained model, it turned out that it was saved as "".
Inspected this by the following GET-request.

GET http://host:port/v1/models/model/metadata

@abatkins what are the necessary imports for this object_detection_client file to make this work. Do you know??