- [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Div node GPU版时出现,CPU版正常
- INVALID_ARGUMENT : Unexpected input data type. 关注输入数据类型
TensorRT版本性能差异测试ResNet50.onnx
模型-耗时ms/张 | TRT7.1.3.4 | TRT5.1.5.0 | 版本性能提升 |
---|---|---|---|
ResNet50-b1-fp32-Nvidia2080Ti | 2.0444 | 2.3022 | 11.20% |
ResNet50-b4-fp32 | 0.6491 | 1.1063 | 41.32% |
ResNet50-b8-fp32 | 0.3533 | 0.8849 | 60.07% |
ResNet50-b16-fp32 | 0.2037 | 0.7502 | 72.85% |
ResNet50-b1-int8 | 0.72 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import matplotlib | |
# Needed before pyplot import for matplotlib to work properly. | |
# matplotlib.use('Agg') | |
import matplotlib.image as mpimg | |
import matplotlib.pyplot as plt | |
from matplotlib.widgets import RectangleSelector | |
from matplotlib.patches import Circle,Rectangle | |
fig = plt.figure() # make a new plotting range |
TRT7.1.3支持到Opset 11 TensorFlow2.0使用tf.saved_model.save自定义保存模型 python3 -m tf2onnx.convert
错误问题
- Use
tf.compat.v1.graph_util.extract_sub_graph
AssertionError: unknown_317 is not in graph - trt不支持Round,改用tf.floor或者tf.ceil解决
- trt不支持NonZero,由于tf.where(condition, x=None, y=None, name=None)的x,y没有赋值导致,补全xy解决 indices = tf.reshape(tf.where(attention_prob >= abs_thres,tf.ones_like(attention_prob,dtype=tf.int64),tf.zeros_like(attention_prob,dtype=tf.int64)), [-1])
- Opset10时trt报错ERROR: builtin_op_importers.cpp:2549 In function importResize: Assertion failed: scales.is_weights() && "Resize scales must be an initializer!" ,使用Opset12转换
索引方法 | latency time | mean average precision | 0.65 false alarm rate |
---|---|---|---|
FLAT | |||
IVF_FLAT | |||
IVF_SQ8 | |||
IVF_SQ8H | |||
IVF_PQ | |||
RNSG | |||
HNSW | |||
ANNOY |
NewerOlder