Linux NPU YOLOV5S Object Detection
RKNN Deployment
The system supports deployment of rknn-toolkit2 and rknn_model_zoo. Click to jump to the Rockchip official documentation.
What is NPU YOLOV5S?
YOLOV5S is an object detection algorithm in computer vision, part of the YOLO (You Only Look Once) series developed by Joseph Redmon, Alexey Bochkovskiy, and others. The YOLO algorithm family is known for fast real-time object detection, capable of detecting and localizing multiple objects in images or videos, including both class and position information.
YOLOv5S is a variant in the YOLOv5 series. Different YOLOv5 versions have varying performance and model complexity. The "S" in YOLOv5S stands for "Small", indicating a lightweight model suitable for resource-constrained environments such as mobile devices or embedded systems. YOLOv5S trades off between speed and accuracy for different application scenarios.
Image and Video Parsing
Rockchip Linux provides the rknn_yolov5_demo with source code supporting RK3562, RK3566, RK3568, and RK3588. The compiled tools can parse both images and videos.
Source Code Path
Note
Currently, this demo works on RK3568 and RK3588. Ubuntu system library compatibility issues prevent it from working on Ubuntu for now.
Source Code Reference and Compilation
1. Configure environment
Tip
Replace SDK with the actual SDK directory path.
$ export TOOL_CHAIN=SDK/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/
$ export GCC_COMPILER=SDK/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
2. Compile by selecting the script matching your IC
3. Output tool path
Tip
Copy to the board for execution.
rknn_yolov5_demo$ ls install/rknn_yolov5_demo_Linux/
lib model rknn_yolov5_demo rknn_yolov5_video_demo
Tool Usage
rknn_yolov5_demo
Example:
Tip
person @ / bus @ shows detection results.
$ ./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
Read model/bus.jpg ...
img width = 640, img height = 640
Loading mode...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.2
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, w_stride = 640, size_with_stride=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=output, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, w_stride = 0, size_with_stride=1638400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003860
index=1, name=283, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, w_stride = 0, size_with_stride=491520, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=2, name=285, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, w_stride = 0, size_with_stride=163840, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003915
model is NHWC input fmt
model input height=640, width=640, channel=3
once run use 31.119000 ms
loadLabelName ./model/coco_80_labels_list.txt
person @ (209 244 286 506) 0.884139
person @ (478 238 559 526) 0.867678
person @ (110 238 230 534) 0.824685
bus @ (94 129 553 468) 0.705055
person @ (79 354 122 516) 0.339254
loop count = 10 , average run 23.615900 ms
rknn_yolov5_video_demo
Tip
Requires h264/h265 bitstream video.
Example:
Tip
car @ / bus @ shows video detection results.
$ ./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn model/yolov5_test.h264 264
Loading mode...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.2
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=output, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003860
index=1, name=283, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=2, name=285, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003915
model is NHWC input fmt
model input height=640, width=640, channel=3
mpi_dec_test start mpi_dec_test decoder test start mpp_type 7 app_ctx=0x7fc1305278 decoder=0x23ad0100
read video size=720664
receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192 decoder require buffer w:h [1280:720] stride [1280:720] buf_size 1843200 pts=0 dts=0 decode_get_frame get info changed found receive packet size=8192 receive packet size=8192 decoder require buffer w:h [1280:720] stride [1280:720] buf_size 1843200 pts=0 dts=0 get one frame 1687102942317 data_vir=0x7f80024000 fd=13 0x24133340 encoder test start w 1280 h 720 type 7
input image 1280x720 stride 1280x720 format=2560
resize with RGA!
rga_api version 1.9.1_[4]
once run use 26.630000 ms
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
loadLabelName ./model/coco_80_labels_list.txt
car @ (380 412 438 461) 0.868004
car @ (532 362 574 399) 0.866127
car @ (496 328 530 354) 0.736625
car @ (564 311 592 331) 0.729147
car @ (860 433 928 497) 0.695733
car @ (676 306 704 327) 0.642370
car @ (738 402 792 457) 0.611211
car @ (202 441 296 498) 0.610788
bus @ (748 272 806 329) 0.527151
car @ (440 322 474 347) 0.526350
car @ (514 310 540 326) 0.507014
car @ (280 666 444 720) 0.347807
car @ (670 288 696 302) 0.271905
car @ (540 289 566 307) 0.176622
car @ (494 297 516 312) 0.168131
car @ (702 263 716 273) 0.163944
Debian 11
Local Video Stream Parsing
Test environment: RK3568 Debian 11
Note
Currently, this demo works on RK3568 and RK3588. Ubuntu system library compatibility issues prevent it from working on Ubuntu for now.
Demo Program Test
1. YOLOV5S object detection demo is pre-installed in the Debian 11 filesystem. Navigate to the demo directory.
2. Run the demo
Getting Demo Source Code
Source code is located in the SDK directory
$ ls external/rknpu2/examples/rknn_yolov5_video_demo/
build build-android_RK356X.sh build-android_RK3588.sh build-linux_RK356X.sh build-linux_RK3588.sh
CMakeLists.txt convert_rknn_demo include install model README.md src
Compiling the Source
1. Specify the cross-compilation toolchain path
$ export TOOL_CHAIN=SDK/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/
$ export GCC_COMPILER=SDK/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
2. Compile the source. After successful compilation, the executable will be in install/rknn_yolov5_demo_Linux/
3. Copy the program to the board (via USB drive, network, etc.)
Tip
The /lib directory in rknn_yolov5_demo_Linux must use the board's /rockchip-test/npu2/rknn_yolov5_demo_Linux/lib
If compilation is not needed, download the executable directly from the cloud drive.
Example:
$ cd rknn_yolov5_demo_Linux/
$ export LD_LIBRARY_PATH=/usr/local/opencv4/lib/
$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opencv4/lib/
$ export PKG_CONFIG_LIBDIR=$PKG_CONFIG_LIBDIR:/usr/local/opencv4/lib/pkgconfig
$ cp /rockchip-test/npu2/rknn_yolov5_demo_Linux/lib . -rf
$ ./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn model/test.mp4
Running the Program
1. Program execution command
Tip
On RK356X platforms, the maximum frame rate is only 7 fps. Optimize program performance as needed.
2. Program interface
Camera Video Stream Parsing
Test environment: RK3568 Debian 11
Note
Currently, this demo works on RK3568 and RK3588. Ubuntu system library compatibility issues prevent it from working on Ubuntu for now.
Demo Program Test
1. YOLOV5S object detection demo is pre-installed in the Debian 11 filesystem. Navigate to the demo directory.
2. Run the demo
Tip
Running for too long may terminate the process due to insufficient memory.
Getting Demo Source Code
Source code is located in the SDK directory
$ ls external/rknpu2/examples/rknn_yolov5_video_demo/
build build-android_RK356X.sh build-android_RK3588.sh build-linux_RK356X.sh build-linux_RK3588.sh
CMakeLists.txt convert_rknn_demo include install model README.md src
Debian 12 / Ubuntu 24.04
Camera and Video Stream Parsing
Obtaining Yolov5
yolov5_video_demo download link
Environment Setup
Compilation
Note
Compile yolov5_video directly on the board.
Source code description:
main_camera.cc: camera mode capture
main_gst.cc: GStreamer mode capture (preferred on Ubuntu)
Before compiling, choose one and overwrite it as main.cc, then compile.
rknn_model_zoo/examples/yolov5_video/cpp
├── CMakeLists.txt
├── camera_preview.cpp
├── main.cc
├── main_camera.cc
├── main_gst.cc
├── postprocess.cc
├── postprocess.h
├── rknpu1
│ └── yolov5.cc
├── rknpu2
│ ├── yolov5.cc
│ └── yolov5_rv1106_1103.cc
└── yolov5.h
On RK Linux Ubuntu 24.04, GStreamer MPP hardware codec and Chromium MPP hardware codec conflict. Only one can be enabled at a time.
To use GStreamer MPP properly, run the following command. After the fix, Chromium will no longer use hardware codec:
- Compilation steps
# Enter rknn_model_zoo directory
cd rknn_model_zoo
# Run build-linux.sh script
# Usage: ./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
# -t : target (rk356x/rk3588) # Platform type: rk3568/rk3566 both use rk356x
# -a : arch (aarch64/armhf) # Board system architecture
# -d : demo name # Subfolder name under examples directory, e.g., yolov5, mobilenet
# -b : build_type (Debug/Release)
# -m : enable address sanitizer, build_type must be set to Debug
./build-linux.sh -t rk356x -a aarch64 -d yolov5_video
Test Video Stream Recognition
rknn_model_zoo/install/rk356x_linux_aarch64/rknn_yolov5_video_demo$ ./rknn_yolov5_demo model/yolov5s_relu_rk3566.rknn model/yolo.mp4
Test Camera Recognition
rknn_model_zoo/install/rk356x_linux_aarch64/rknn_yolov5_video_demo$ ./rknn_yolov5_demo model/yolov5s_relu_rk3566.rknn 0
