Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save revilokeb/3940c2bbdcf9f25dc4eb809d5b51a220 to your computer and use it in GitHub Desktop.
Save revilokeb/3940c2bbdcf9f25dc4eb809d5b51a220 to your computer and use it in GitHub Desktop.
Tensorflow Serving with CUDA 7.5 / cuDNN5 on Docker serving inception v3, query with client from within docker and from host
#docker, start with port 9000 as in TF serving example on old commit https://gist.github.com/revilokeb/02894302701fcea7ef4e85a92514e246
nvidia-docker run --rm -ti -p 9000 -v /myhosthome:/mydockerhome tf_serving_old_commit_cuda7.5_cudnn5:latest bash
#docker, upgrade six
sudo pip install --upgrade six
#docker, install grpcio
sudo pip install grpcio
#docker, download inception-v3 pre-trained model and export (https://tensorflow.github.io/serving/serving_inception.html)
curl -O http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz //obtain inception 3 pre-trained
tar xzf inception-v3-2016-03-01.tar.gz
bazel-bin/tensorflow_serving/example/inception_export --checkpoint_dir=inception-v3 --export_dir=inception-export
#docker, start inception server
cd serving
bazel-bin/tensorflow_serving/example/inception_inference --port=9000 inception-export &> inception_log & //starting inception
#docker, run client from within docker
bazel-bin/tensorflow_serving/example/inception_client --server=localhost:9000 --image=/mydockerhome/path/to/image
#now run client from host (or any other IP)
#host, check binding (https://docs.docker.com/engine/userguide/networking/default_network/binding/)
sudo iptables -t nat -L -n //from Chain DOCKER note binding address such as e.g. 172.17.0.2
#host, use compiled inception client on host (!) to query server running on docker
bazel-bin/tensorflow_serving/example/inception_client --server=172.17.0.2:9000 --image=/myhosthome/path/to/image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment