Skip to content

Instantly share code, notes, and snippets.

@pranavsharma

pranavsharma/gist:c26a30a4b90b94f8c7f0334420427c1d

Created May 5, 2022 23:43
Show Gist options
  • Save pranavsharma/c26a30a4b90b94f8c7f0334420427c1d to your computer and use it in GitHub Desktop.
Save pranavsharma/c26a30a4b90b94f8c7f0334420427c1d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
root@d9f3a6d38dc9:/opt/tritonserver/qa/L0_io# LD_LIBRARY_PATH=/opt/tritonserver/lib ./memory_alloc -i 0 -o -1 -r /opt/tritonserver/qa/L0_io/models -m onnx_float32_float32
_float32
I0505 23:42:49.968124 643 libtorch.cc:1381] TRITONBACKEND_Initialize: pytorch
I0505 23:42:49.968240 643 libtorch.cc:1391] Triton TRITONBACKEND API version: 1.9
I0505 23:42:49.968253 643 libtorch.cc:1397] 'pytorch' TRITONBACKEND API version: 1.9
2022-05-05 23:42:50.145089: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
I0505 23:42:50.185708 643 tensorflow.cc:2181] TRITONBACKEND_Initialize: tensorflow
I0505 23:42:50.185750 643 tensorflow.cc:2191] Triton TRITONBACKEND API version: 1.9
I0505 23:42:50.185765 643 tensorflow.cc:2197] 'tensorflow' TRITONBACKEND API version: 1.9
I0505 23:42:50.185775 643 tensorflow.cc:2221] backend configuration:
{}
I0505 23:42:50.187129 643 onnxruntime.cc:2407] TRITONBACKEND_Initialize: onnxruntime
I0505 23:42:50.187156 643 onnxruntime.cc:2417] Triton TRITONBACKEND API version: 1.9
I0505 23:42:50.187167 643 onnxruntime.cc:2423] 'onnxruntime' TRITONBACKEND API version: 1.9
I0505 23:42:50.187179 643 onnxruntime.cc:2453] backend configuration:
{}
I0505 23:42:50.210870 643 openvino.cc:1207] TRITONBACKEND_Initialize: openvino
I0505 23:42:50.210896 643 openvino.cc:1217] Triton TRITONBACKEND API version: 1.9
I0505 23:42:50.210908 643 openvino.cc:1223] 'openvino' TRITONBACKEND API version: 1.9
I0505 23:42:51.999414 643 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f8c7c000000' with size 268435456
I0505 23:42:51.999854 643 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
I0505 23:42:52.001919 643 model_repository_manager.cc:1077] loading: onnx_float32_float32_float32:1
I0505 23:42:52.102139 643 model_repository_manager.cc:1077] loading: onnx_float32_float32_float32:3
I0505 23:42:52.102248 643 onnxruntime.cc:2488] TRITONBACKEND_ModelInitialize: onnx_float32_float32_float32 (version 1)
I0505 23:42:52.103569 643 onnxruntime.cc:2531] TRITONBACKEND_ModelInstanceInitialize: onnx_float32_float32_float32_0 (GPU device 0)
I0505 23:42:52.900964 643 onnxruntime.cc:2488] TRITONBACKEND_ModelInitialize: onnx_float32_float32_float32 (version 3)
I0505 23:42:52.901331 643 model_repository_manager.cc:1231] successfully loaded 'onnx_float32_float32_float32' version 1
I0505 23:42:52.901774 643 onnxruntime.cc:2531] TRITONBACKEND_ModelInstanceInitialize: onnx_float32_float32_float32_0 (GPU device 0)
I0505 23:42:52.905162 643 model_repository_manager.cc:1231] successfully loaded 'onnx_float32_float32_float32' version 3
I0505 23:42:52.905252 643 server.cc:549]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I0505 23:42:52.905362 643 server.cc:576]
+-------------+-------------------------------------------------------------------------+--------+
| Backend | Path | Config |
+-------------+-------------------------------------------------------------------------+--------+
| pytorch | /opt/tritonserver/backends/pytorch/libtriton_pytorch.so | {} |
| tensorflow | /opt/tritonserver/backends/tensorflow1/libtriton_tensorflow1.so | {} |
| onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so | {} |
| openvino | /opt/tritonserver/backends/openvino_2021_4/libtriton_openvino_2021_4.so | {} |
+-------------+-------------------------------------------------------------------------+--------+
I0505 23:42:52.905407 643 server.cc:619]
+------------------------------+---------+--------+
| Model | Version | Status |
+------------------------------+---------+--------+
| onnx_float32_float32_float32 | 1 | READY |
| onnx_float32_float32_float32 | 3 | READY |
+------------------------------+---------+--------+
I0505 23:42:52.938574 643 metrics.cc:650] Collecting metrics for GPU 0: Tesla V100-PCIE-16GB
I0505 23:42:52.939545 643 tritonserver.cc:2123]
+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.21.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_mem |
| | ory cuda_shared_memory binary_tensor_data statistics trace |
| model_repository_path[0] | /opt/tritonserver/qa/L0_io/models |
| model_control_mode | MODE_EXPLICIT |
| startup_models_0 | onnx_float32_float32_float32 |
| strict_model_config | 1 |
| rate_limit | OFF |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 67108864 |
| response_cache_byte_size | 0 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
Server Health: live 1, ready 1
Server Status:
{"name":"triton","version":"2.21.0","extensions":["classification","sequence","model_repository","model_repository(unload_dependents)","schedule_policy","model_configuration","system_shared_memory","cuda_shared_memory","binary_tensor_data","statistics","trace"]}
allocated 64 bytes in CPU for result tensor OUTPUT0
allocated 64 bytes in CPU for result tensor OUTPUT1
OUTPUT0 are stored in CPU memory
OUTPUT1 are stored in CPU memory
0 + 1 = -1
0 - 1 = 1
error: incorrect sum in OUTPUT0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment