How to convert ONNX to ORT

ort.md

This example uses Docker to install a specific Python version

One liner version

Given the file /home/matt/mobilenetv2-7.onnx as an example replace /home/matt and mobilenetv2-7.onnx as needed.

docker run --rm -v /home/matt:/mnt python:3.9-buster bash -c "pip install onnx onnxruntime && python -m onnxruntime.tools.convert_onnx_models_to_ort /mnt/mobilenetv2-7.onnx"

Detailed version

1. Install docker for your system, make sure your user can run it OK

❯ docker run --rm hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete 
Digest: sha256:13e367d31ae85359f42d637adf6da428f76d75dc9afeb3c21faea0d976f5c651
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

(etc)

2. Run a Python 3.9 container

docker run --rm -it python:3.9-buster bash

3. Install onnx and onnxruntime

pip install onnx onnxruntime

Ignore root user or upgrade warnings

4. Ensure the convertion tool works

python -m onnxruntime.tools.convert_onnx_models_to_ort

Should output usage and complain that you havent given a model file.

5. Copy the file to the container

In another terminal window run:

docker container ls

Take note of the container ID.

Run this command to copy in the ONNX file:

# where 5bb3aea53cd1 is your ID and onnx_file.onnx is your .onnx file
docker cp onnx_file.onnx 5bb3aea53cd1:/

6. Convert it!

Switch back to your first terminal and run:

# where onnx_file.onnx is your .onnx file
python -m onnxruntime.tools.convert_onnx_models_to_ort onnx_file.onnx

7. Copy back the ort file

Back in your second terminal run:

# where 5bb3aea53cd1 is your ID and onnx_file.ort is your converted .ort file
docker cp 5bb3aea53cd1:/onnx_file.ort .