Skip to content

Linux RKNPU

Rockchip RK356X, RK3576, and RK3588 series chips all feature a built-in independent NPU (Neural Processing Unit), with INT8 computing power reaching up to 1 TOPS and 6 TOPS levels respectively. They fully support mainstream deep learning frameworks such as TensorFlow, TF-lite, PyTorch, Caffe, and ONNX. For detailed specifications, refer to the corresponding chip datasheets.

The NPU is dedicated hardware designed specifically for neural network operations. Its core mission is to efficiently accelerate neural network algorithms in artificial intelligence, particularly suited for scenarios such as machine vision and natural language processing. As AI applications continue to expand, RKNPU-based solutions now cover multiple domains including facial tracking, gesture and body pose tracking, image classification, video surveillance, automatic speech recognition (ASR), and advanced driver assistance systems (ADAS), meeting diverse AI deployment requirements.

To facilitate AI project development based on RKNPU, Rockchip provides a complete set of RKNPU components covering the entire process of development, deployment, and optimization. These include RKNPU2, RKNN Toolkit2, RKLLM-Toolkit, and RKNPU drivers. Each component has a clear division of responsibilities and works together seamlessly:

  • RKNPU2: Core component of the development suite, providing runtime libraries such as librknnrt.so and supporting C/C++ programming interfaces. It can be directly used for RKNN model deployment and inference on Linux and Android systems, providing underlying runtime support for AI applications.
  • RKNN Toolkit2: Python-based development suite supporting both x86 and arm64 platforms. It integrates model conversion, quantization, inference testing, performance and memory evaluation, quantization accuracy analysis, and model encryption into one package. The included RKNN Toolkit Lite2 submodule allows RKNN model deployment directly on Rockchip boards, simplifying on-board development.
  • RKNN Model: Rockchip's proprietary model format customized for its NPU hardware architecture. Compared to general-purpose model formats, it achieves higher inference performance and operational efficiency on Rockchip NPUs, serving as the core carrier of RKNPU acceleration capabilities.
  • RKLLM-Toolkit: Specialized suite for large language model (LLM) development, supporting quantization and conversion of Hugging Face format LLMs on host computers, ultimately outputting RKLLM models compatible with RKNPU, enabling rapid deployment of large language models on Rockchip platforms.
  • RKNPU Driver: Provides the interface between NPU hardware and the system. Rockchip board firmware includes driver adaptation by default, so developers do not need to develop drivers separately and can directly use existing firmware for AI project development, lowering the deployment barrier.

Additionally, Rockchip provides the RKNN Model Zoo repository, developed based on the RKNN toolchain. It integrates complete deployment examples for mainstream AI algorithms, covering both Python API and C API development interfaces. Developers can directly refer to these examples to get started quickly, significantly shortening AI project development cycles.

rknn-toolkit2 supported platforms

  • RK3588 Series
  • RK3576 Series
  • RK3566/RK3568 Series
  • RK3562 Series
  • RV1103/RV1106
  • RV1103B/RV1106B
  • RV1126B
  • RK2118

rknn_model_zoo supported platforms

  • Support RK3562, RK3566, RK3568, RK3576, RK3588, RV1126B platforms.
  • Limited support for RV1103, RV1106
  • Support RV1109, RV1126, RK1808 platforms.

rknn-llm supported platforms

  • RK3588 Series
  • RK3576 Series
  • RK3562 Series
  • RV1126B Series

The system supports deployment of rknn-toolkit2, rknn_model_zoo, and rknn-llm. Click to jump to the Rockchip official online documentation.

PC-side RKNN Deployment

rknn-toolkit2 Environment Setup

Download RKNN Repositories

Download RKNN repositories via command line or cloud drive.

# Create Projects folder
mkdir Projects

# Enter the directory
cd Projects

# Clone RKNN-Toolkit2 repository
git clone https://github.com/airockchip/rknn-toolkit2.git --depth 1

# Clone RKNN Model Zoo repository
git clone https://github.com/airockchip/rknn_model_zoo.git --depth 1

# Note:
# 1. The --depth 1 parameter clones only the latest commit
# 2. If git clone fails, download the zip from GitHub and extract it to this directory

Install Miniforge Conda

conda -V
# Expected output: conda 23.3.1 (or similar)
# If "conda: command not found" appears, Miniforge is not installed

wget -c https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh

chmod 777 Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh

source ~/miniforge3/bin/activate # Miniforge installation directory
# After success, the prompt changes to:
# (base) xxx@xxx:~$

conda create -n toolkit2 python=3.8

conda activate toolkit2
# After success, the prompt changes to:
# (toolkit2) xxx@xxx:~$

Install RKNN-Toolkit2

Install via pip

pip install rknn-toolkit2 -i https://pypi.org/simple

# If RKNN-Toolkit2 is already installed, upgrade it with:
pip install rknn-toolkit2 -i https://pypi.org/simple --upgrade

Install via local wheel package

# Enter rknn-toolkit2 directory
cd Projects/rknn-toolkit2/rknn-toolkit2

# Choose the appropriate requirements file based on Python version and architecture:
# cpxx is the Python version number
pip install -r packages/x86_64/requirements_cpxx.txt
# pip install -r packages/arm64/arm64_requirements_cpxx.txt

# Install RKNN-Toolkit2
# Choose the appropriate wheel package based on Python version and architecture:
# x.x.x is the RKNN-Toolkit2 version, cpxx is the Python version number
pip install packages/x86_64/rknn_toolkit2-x.x.x-cpxx-cpxx-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
# pip install packages/arm64/rknn_toolkit2-x.x.x-cpxx-cpxx-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Verify the installation. If installation fails, refer to Chapter 10.2 "Tool Installation Issues" in the Rockchip_RKNPU_User_Guide_RKNN_SDK_CN.pdf document.

# Enter Python interactive mode
python
# Import RKNN class
from rknn.api import RKNN

rknn Model Conversion

# Enter rknn_model_zoo/examples/yolov5/model directory
cd Projects/rknn_model_zoo/examples/yolov5/model

# Run download_model.sh to download yolov5 ONNX model
# Downloaded ONNX model will be saved to model/yolov5s_relu.onnx
./download_model.sh

# Enter rknn_model_zoo/examples/yolov5/python directory
cd Projects/rknn_model_zoo/examples/yolov5/python

# Run convert.py to convert ONNX model to RKNN model
# Usage: python convert.py model_path [rk3566|rk3588|rk3562] [i8/fp] [output_path]
python convert.py ../model/yolov5s_relu.onnx rk3588 i8 ../model/yolov5s_relu.rknn

rknn_model_zoo Cross-compilation

Build Environment Setup

sudo apt install libopencv-dev git cmake make gcc g++ libsndfile1-dev -y

Download GCC Cross-compilation Toolchain

For 64-bit target systems:

https://releases.linaro.org/components/toolchain/binaries/6.3-2017.05/aarch64-linux-gnu/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz

For 32-bit target systems:

https://releases.linaro.org/components/toolchain/binaries/6.3-2017.05/arm-linuxgnueabihf/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf.tar.xz

# Directory structure
Projects
├── rknn-toolkit2
├── rknn_model_zoo
└── gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu # This path is used later when compiling RKNN C demos

Cross-compilation

# Add to the beginning of build-linux.sh
GCC_COMPILER=Projects/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu

# Enter rknn_model_zoo directory
cd Projects/rknn_model_zoo

# Run build-linux.sh script
# Usage: ./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
# -t : target (rk356x/rk3588) # Platform type: rk3568/rk3566 both use rk356x
# -a : arch (aarch64/armhf) # Target system architecture
# -d : demo name # Subfolder name under the examples directory, e.g., yolov5, mobilenet
# -b : build_type (Debug/Release)
# -m : enable address sanitizer, build_type must be set to Debug
./build-linux.sh -t rk356x -a aarch64 -d yolov5

Board-side RKNN Deployment

If RKNN Server and Runtime libraries are not installed on the board, or if the versions are inconsistent, reinstall the RKNPU2 environment.

Typically, development boards come with a consistent RKNPU2 environment pre-installed.

(Note: 1. For RKNN models with dynamic dimension inputs, RKNN Server and Runtime library versions must be >= 1.5.0. 2. Ensure RKNN Server, Runtime library, and RKNN-Toolkit2 versions are consistent. Installing the latest versions is recommended.)

Verify RKNPU2 Environment

# Check RKNPU2 driver version
dmesg | grep -i rknpu

# Start rknn_server
restart_rknn.sh
# "start rknn server, version: x.x.x" indicates rknn_server started successfully, meaning RKNPU2 environment is installed

# Check rknn_server version
strings /usr/bin/rknn_server | grep -i "rknn_server version"

# Check librknnrt.so version
strings /usr/lib/librknnrt.so | grep -i "librknnrt version"

Install/Update RKNPU2 Environment

# rknpu2 directory
Projects/rknn-toolkit2/rknpu2

# Copy rknn_server to the board directory
# Note: For 64-bit Linux systems, BOARD_ARCH corresponds to aarch64 directory; for 32-bit systems, corresponds to armhf directory.
Projects/rknn-toolkit2/rknpu2/runtime/Linux/rknn_server/${BOARD_ARCH}/usr/bin/* /usr/bin

# Copy librknnrt.so to the board directory
Projects/rknn-toolkit2/rknpu2/runtime/Linux/librknn_api/${BOARD_ARCH}/librknnrt.so /usr/lib

# Grant execute permissions
chmod +x /usr/bin/rknn_server
chmod +x /usr/bin/start_rknn.sh
chmod +x /usr/bin/restart_rknn.sh

# Restart rknn_server service
restart_rknn.sh

rknn_model_zoo Board-side Compilation

Install build dependencies

sudo apt install libopencv-dev git cmake make gcc g++ libsndfile1-dev -y

Compile

# Enter rknn_model_zoo directory
cd rknn_model_zoo

# Run build-linux.sh script
# Usage: ./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
# -t : target (rk356x/rk3588) # Platform type: rk3568/rk3566 both use rk356x
# -a : arch (aarch64/armhf) # Target system architecture
# -d : demo name # Subfolder name under the examples directory, e.g., yolov5, mobilenet
# -b : build_type (Debug/Release)
# -m : enable address sanitizer, build_type must be set to Debug
./build-linux.sh -t rk356x -a aarch64 -d yolov5