output_tf1_model.txt

2021-05-11 07:30:13.923634: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
/usr/local/lib/python3.8/dist-packages/tensorflow_addons/utils/ensure_tf_install.py:54: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or e
qual to 2.2.0 and strictly below 2.4.0 (nightly versions are not supported).
 The versions of TensorFlow you are currently using is 2.4.0 and is not supported.
Some things might work, some things might not.
If you were to encounter a bug, do not file an issue.
If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version.
You can find the compatibility matrix in TensorFlow Addon's readme:
https://github.com/tensorflow/addons
  warnings.warn(
I0511 07:30:15.427670 139652671039296 export_model.py:111] SAVED MODEL DIR IS ['xxxx', '/trainer/models', '/trainer/models/xxxx', '/trainer/mod
els/xxxx/1', '/trainer/models/xxxx/1/model.savedmodel']
2021-05-11 07:30:15.648103: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-05-11 07:30:15.648842: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2021-05-11 07:30:16.047658: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Found device 0 with properties:
pciBusID: 0000:17:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.607GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2021-05-11 07:30:16.047724: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-05-11 07:30:16.052478: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-05-11 07:30:16.052560: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-05-11 07:30:16.053756: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-05-11 07:30:16.054074: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-05-11 07:30:16.055170: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2021-05-11 07:30:16.056121: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-05-11 07:30:16.056315: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-05-11 07:30:16.058198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1888] Adding visible gpu devices: 0
2021-05-11 07:30:16.059950: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-05-11 07:30:16.062082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Found device 0 with properties:
pciBusID: 0000:17:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.607GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2021-05-11 07:30:16.062155: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-05-11 07:30:16.062207: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-05-11 07:30:16.062246: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-05-11 07:30:16.062282: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-05-11 07:30:16.062318: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-05-11 07:30:16.062359: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2021-05-11 07:30:16.062402: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-05-11 07:30:16.062448: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-05-11 07:30:16.066173: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1888] Adding visible gpu devices: 0
2021-05-11 07:30:16.066251: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-05-11 07:30:16.494868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1287] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-11 07:30:16.494927: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1293]      0
2021-05-11 07:30:16.494938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1306] 0:   N
2021-05-11 07:30:16.496706: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10408 MB memory) -> p
hysical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:17:00.0, compute capability: 6.1)


# Truncated description of all the nodes

WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:200: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
W0511 01:05:18.537761 140170105308992 deprecation.py:333] From /usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:200: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:No assets to save.
I0511 01:05:18.538428 140170105308992 builder_impl.py:641] No assets to save.
INFO:tensorflow:No assets to write.
I0511 01:05:18.538529 140170105308992 builder_impl.py:461] No assets to write.
INFO:tensorflow:SavedModel written to: 021220_Inception_V3_100k/saved/saved_model.pb
I0511 01:05:18.790151 140170105308992 builder_impl.py:426] SavedModel written to: 021220_Inception_V3_100k/saved/saved_model.pb
2021-05-11 01:05:18.816610: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer.so.7
INFO:tensorflow:Linked TensorRT version: (7, 2, 2)
I0511 01:05:18.816717 140170105308992 trt_convert.py:265] Linked TensorRT version: (7, 2, 2)
INFO:tensorflow:Loaded TensorRT version: (7, 2, 2)
I0511 01:05:18.816910 140170105308992 trt_convert.py:266] Loaded TensorRT version: (7, 2, 2)
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
I0511 01:05:19.481142 140170105308992 saver.py:1511] Saver not created because there are no variables in the graph to restore
2021-05-11 01:05:19.691362: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-05-11 01:05:19.692454: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Found device 0 with properties: 
pciBusID: 0000:17:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.607GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2021-05-11 01:05:19.692516: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-05-11 01:05:19.692579: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-05-11 01:05:19.692603: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-05-11 01:05:19.692623: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-05-11 01:05:19.692644: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-05-11 01:05:19.692665: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2021-05-11 01:05:19.692686: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-05-11 01:05:19.692710: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-05-11 01:05:19.694336: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1888] Adding visible gpu devices: 0
2021-05-11 01:05:19.694382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1287] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-11 01:05:19.694399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1293]      0 
2021-05-11 01:05:19.694414: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1306] 0:   N 
2021-05-11 01:05:19.696012: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10408 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:17:00.0, compute capability: 6.1)
2021-05-11 01:05:19.714074: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-05-11 01:05:19.732990: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 3299990000 Hz
2021-05-11 01:05:20.819953: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1
2021-05-11 01:05:20.820233: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2021-05-11 01:05:20.820572: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-05-11 01:05:20.820924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Found device 0 with properties: 
pciBusID: 0000:17:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.607GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2021-05-11 01:05:20.820951: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-05-11 01:05:20.820981: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-05-11 01:05:20.820997: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-05-11 01:05:20.821010: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-05-11 01:05:20.821023: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-05-11 01:05:20.821035: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2021-05-11 01:05:20.821048: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-05-11 01:05:20.821062: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-05-11 01:05:20.821483: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1888] Adding visible gpu devices: 0
2021-05-11 01:05:20.821518: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1287] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-11 01:05:20.821529: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1293]      0 
2021-05-11 01:05:20.821541: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1306] 0:   N 
2021-05-11 01:05:20.821989: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10408 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:17:00.0, compute capability: 6.1)
2021-05-11 01:05:21.000037: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:954] Optimization results for grappler item: graph_to_optimize
  function_optimizer: function_optimizer did nothing. time = 0.004ms.
  function_optimizer: function_optimizer did nothing. time = 0ms.

2021-05-11 01:05:22.560601: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1
2021-05-11 01:05:22.560762: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2021-05-11 01:05:22.561053: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-05-11 01:05:22.561379: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Found device 0 with properties: 
pciBusID: 0000:17:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.607GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2021-05-11 01:05:22.561408: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-05-11 01:05:22.561446: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-05-11 01:05:22.561461: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-05-11 01:05:22.561476: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-05-11 01:05:22.561489: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-05-11 01:05:22.561505: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2021-05-11 01:05:22.561521: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-05-11 01:05:22.561544: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-05-11 01:05:22.561982: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1888] Adding visible gpu devices: 0
2021-05-11 01:05:22.562016: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1287] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-11 01:05:22.562026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1293]      0 
2021-05-11 01:05:22.562037: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1306] 0:   N 
2021-05-11 01:05:22.562483: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10408 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:17:00.0, compute capability: 6.1)
2021-05-11 01:05:23.559157: I tensorflow/compiler/tf2tensorrt/segment/segment.cc:790] There are 5 ops of 4 different types in the graph that are not converted to TensorRT: Reshape, Shape, Placeholder, NoOp, (For more information see https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#supported-ops).
2021-05-11 01:05:23.623233: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:757] Number of TensorRT candidate segments: 1
2021-05-11 01:05:23.774583: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:851] Replaced segment 0 consisting of 801 nodes by InceptionV3/TRTEngineOp_0_0.
2021-05-11 01:05:24.323988: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:954] Optimization results for grappler item: tf_graph
  constant_folding: Graph size after: 800 nodes (-386), 835 edges (-386), time = 308.869ms.
  layout: Graph size after: 815 nodes (15), 850 edges (15), time = 218.619ms.
  constant_folding: Graph size after: 804 nodes (-11), 839 edges (-11), time = 108.149ms.
  TensorRTOptimizer: Graph size after: 4 nodes (-800), 4 edges (-835), time = 437.802ms.
  constant_folding: Graph size after: 4 nodes (0), 4 edges (0), time = 2.365ms.
Optimization results for grappler item: InceptionV3/TRTEngineOp_0_0_native_segment
  constant_folding: Graph size after: 804 nodes (0), 838 edges (0), time = 70.074ms.
  layout: Graph size after: 804 nodes (0), 838 edges (0), time = 124.898ms.
  constant_folding: Graph size after: 804 nodes (0), 838 edges (0), time = 73.681ms.
  TensorRTOptimizer: Graph size after: 804 nodes (0), 838 edges (0), time = 14.649ms.
  constant_folding: Graph size after: 804 nodes (0), 838 edges (0), time = 74.884ms.

2021-05-11 01:05:24.917187: W tensorflow/core/framework/op_kernel.cc:1763] OP_REQUIRES failed at trt_engine_resource_ops.cc:193 : Not found: Container TF-TRT does not exist. (Could not find resource: TF-TRT/TRTEngineOp_0_0)
INFO:tensorflow:Could not find TRTEngineOp_0_0 in TF-TRT cache. This can happen if build() is not called, which means TensorRT engines will be built and cached at runtime.
I0511 01:05:24.917315 140170105308992 trt_convert.py:1237] Could not find TRTEngineOp_0_0 in TF-TRT cache. This can happen if build() is not called, which means TensorRT engines will be built and cached at runtime.
INFO:tensorflow:Assets written to: /trainer/models/xxxx/1/model.savedmodel/assets
I0511 01:05:26.047748 140170105308992 builder_impl.py:774] Assets written to: /trainer/models/021220_Inception_V3_100k/1/model.savedmodel/assets
I0511 01:05:26.249925 140170105308992 export_model.py:216] Writing /trainer/models/xxxx/config.pbtxt