changtimwu/openvinowin_anson.md

## openvinowin_anson.md

      
    Raw
  

              openvinowin_anson.md
            
          
    OPENVINO for Windows on IEI DRPC-240


硬體: DRPC-240-TGL-U
OS: Windows 10
底下文中的 C:\Users\B622 代表使用者文件夾, 實際上會因使用者名稱會有不同, 在 PowerShell 中, 可以此變數 $env:USERPROFILE 獲得

OpenVINO 2022.2


簡介: OpenVINO是intel開發的高速高性能計算機視覺和深度學習推理應用開發套件, 2022 版本對 Intel 核顯有較好的支援, 可善用核顯作硬體推理裝置.
安裝位置: 照官方建議, 以 python virtualenv 方式安裝, 所以實際位置是在 C:\Users\B622\openvino_env\Libs\site-packages\openvino
啟動方式:

開啟 PowerShell V7, 執行 .\openvino_env\Scripts\activate, 應該會看到命令提示符號如下變化


C:\Users\B622\> .\openvino_env\Scripts\activate
(openvino_env) PS C:\Users\B622\>

Open Model Zoo


簡介 openvino 提供大量預編模型, 使用範例, 與模型處理工具
來源: https://github.com/openvinotoolkit/open_model_zoo
安裝位置 C:\Users\B622\open_model_zoo-master

執行範例


底下示範如何執行 segmentation_demo 範例, 此範例主要示範語意分割
切換到目錄

cd open_model_zoo-master\demos\segmentation_demo\python


下載此範例所需的預先編譯模型

omz_downloader --list models.lst


執行範例, 輸入為官方附上動畫 gif , 語意分割結果繪製在 out.avi 上

python .\segmentation_demo.py -d GPU -i ..\segmentation.gif  -at segmentation  -o out.avi -m .\intel\semantic-segmentation-adas-0001\FP16\semantic-segmentation-adas-0001.xml

Yolov5s 測速


安裝位置 C:\Users\B622\yolov5-master
以 yolov5 官方提供的 yolov5s.pt 基礎, 先照官方步驟轉換出 yolov5s.onnx, 再使用 openvino 內附的 mo 工具轉換出 openvino 專屬的 IR 格式
指定量化至 FP16
使用 Intel 官方工具 benchmark_app 進行測速, 指定 GPU(核顯) 為推理裝置, 效能為 55fps

(openvino_env) PS C:\Users\B622\yolov5-master> mkdir ovir
(openvino_env) PS C:\Users\B622\yolov5-master> mo --input_model yolov5s.onnx --input_shape [1,3,640,640] --model_name yolov5s -s 255 --reverse_input_channels  --data_type FP16 -o ovir
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      C:\Users\B622\yolov5-master\yolov5s.onnx
        - Path for generated IR:        C:\Users\B622\yolov5-master\ovir
        - IR output name:       yolov5s
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         Not specified, inherited from the model
        - Output layers:        Not specified, inherited from the model
        - Input shapes:         [1,3,640,640]
        - Source layout:        Not specified
        - Target layout:        Not specified
        - Layout:       Not specified
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         255.0
        - Precision of IR:      FP16
        - Enable fusing:        True
        - User transformations:         Not specified
        - Reverse input channels:       True
        - Enable IR generation for fixed input shape:   False
        - Use the transformations config file:  None
Advanced parameters:
        - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR:   False
        - Force the usage of new Frontend of Model Optimizer for model conversion into IR:      False
OpenVINO runtime found in:      c:\users\b622\openvino_env\lib\site-packages\openvino
OpenVINO runtime version:       2022.2.0-7713-af16ea1d79a-releases/2022/2
Model Optimizer version:        2022.2.0-7713-af16ea1d79a-releases/2022/2
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: C:\Users\B622\yolov5-master\ovir\yolov5s.xml
[ SUCCESS ] BIN file: C:\Users\B622\yolov5-master\ovir\yolov5s.bin
[ SUCCESS ] Total execution time: 1.86 seconds.
[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai
(openvino_env) PS C:\Users\B622\yolov5-master> ls ovir/

    Directory: C:\Users\B622\yolov5-master\ovir

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           9/27/2022  3:08 PM       14653514 yolov5s.bin
-a---           9/27/2022  3:08 PM          34446 yolov5s.mapping
-a---           9/27/2022  3:08 PM         333805 yolov5s.xml

(openvino_env) PS C:\Users\B622\yolov5-master> mv ovir ovir_FP16
(openvino_env) PS C:\Users\B622\yolov5-master> cd .\ovir_FP16\
(openvino_env) PS C:\Users\B622\yolov5-master\ovir_FP16> benchmark_app.exe -d GPU -m yolov5s.xml
[Step 1/11] Parsing and validating input arguments
[ WARNING ]  -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README. 
[Step 2/11] Loading OpenVINO
[ WARNING ] PerformanceMode was not explicitly specified in command line. Device GPU performance hint will be set to THROUGHPUT.
[ INFO ] OpenVINO:
         API version............. 2022.2.0-7713-af16ea1d79a-releases/2022/2
[ INFO ] Device info
         GPU
         Intel GPU plugin........ version 2022.2
         Build................... 2022.2.0-7713-af16ea1d79a-releases/2022/2

[Step 3/11] Setting device configuration
[ WARNING ] -nstreams default value is determined automatically for GPU device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 4/11] Reading network files
[ INFO ] Read model took 49.00 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Model input 'images' precision u8, dimensions ([N,C,H,W]): 1 3 640 640
[ INFO ] Model output 'output' precision f32, dimensions ([...]): 1 25200 85
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 7485.90 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] DEVICE: GPU
[ INFO ]   AVAILABLE_DEVICES  , ['0']
[ INFO ]   RANGE_FOR_ASYNC_INFER_REQUESTS  , (1, 2, 1)
[ INFO ]   RANGE_FOR_STREAMS  , (1, 2)
[ INFO ]   OPTIMAL_BATCH_SIZE  , 1
[ INFO ]   MAX_BATCH_SIZE  , 1
[ INFO ]   FULL_DEVICE_NAME  , Intel(R) Iris(R) Xe Graphics (iGPU)
[ INFO ]   DEVICE_TYPE  , Type.INTEGRATED
[ INFO ]   OPTIMIZATION_CAPABILITIES  , ['FP32', 'BIN', 'FP16', 'INT8']
[ INFO ]   GPU_UARCH_VERSION  , unknown
[ INFO ]   GPU_EXECUTION_UNITS_COUNT  , 80
[ INFO ]   PERF_COUNT  , False
[ INFO ]   MODEL_PRIORITY  , Priority.MEDIUM
[ INFO ]   GPU_HOST_TASK_PRIORITY  , Priority.MEDIUM
[ INFO ]   GPU_QUEUE_PRIORITY  , Priority.MEDIUM
[ INFO ]   GPU_QUEUE_THROTTLE  , Priority.MEDIUM
[ INFO ]   GPU_ENABLE_LOOP_UNROLLING  , True
[ INFO ]   CACHE_DIR  ,
[ INFO ]   PERFORMANCE_HINT  , PerformanceMode.THROUGHPUT
[ INFO ]   COMPILATION_NUM_THREADS  , 8
[ INFO ]   NUM_STREAMS  , 1
[ INFO ]   PERFORMANCE_HINT_NUM_REQUESTS  , 0
[ INFO ]   DEVICE_ID  , 0
[Step 9/11] Creating infer requests and preparing input data
[ INFO ] Create 2 infer requests took 9.00 ms
[ WARNING ] No input files were given for input 'images'!. This input will be filled with random values!
[ INFO ] Fill input 'images' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 2 inference requests using 1 streams for GPU, inference only: True, limits: 60000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 18.34 ms
[Step 11/11] Dumping statistics report
Count:          3334 iterations
Duration:       60141.23 ms
Latency:
    Median:     33.60 ms
    AVG:        36.01 ms
    MIN:        17.23 ms
    MAX:        95.07 ms
Throughput: 55.44 FPS
(openvino_env) PS C:\Users\B622\yolov5-master\ovir_FP16>