Camera

Tip

There are currently issues with the ISP in Armbian/Ubuntu systems, MIPI cameras might not work properly.

The MIPI CSI interface information for Rockchip platforms is as follows:

Board	SoC	Interface	Module	Specifications	Interface Pins
K1	RK3568	MIPI_CSI	gc5035	4 Lane x1 2 Lane x2	40 Pin
K1B	RK3568	MIPI_CSI	gc5035	4 Lane x1 2 Lane x2	40 Pin
K3	RK3562	MIPI_CSI_RX0, MIPI_CSI_RX1	gc5035	4 Lane x2 2 Lane x4	40 Pin
K7 V1	RK3576	MIPI_CSI0, MIPI_CSI1, MIPI_CSI3	None	4 Lane x3	40 Pin
K7 V2	RK3576	MIPI_CSI0, MIPI_CSI1, MIPI_CSI3	imx415	4 Lane x3	20 Pin
K7C	RK3576	MIPI_CSI0, MIPI_CSI1, MIPI_CSI3	imx415	4 Lane x3	20 Pin
K8	RK3588	MIPI_CSI0, MIPI_CSI1, MIPI_DPHY0, MIPI_DPHY1	None	4 Lane x4 4 Lane x3+2 Lane x2 4 Lane x2 2 Lane x4	40 Pin

DTS Node Configuration

RK3562

The MIPI camera dts configuration for the RK3562 platform (K3) is as follows:

kernel-5.10/arch/arm64/boot/dts/rockchip/rk3562-kickpi-mipi-camera0.dtsi
kernel-5.10/arch/arm64/boot/dts/rockchip/rk3562-kickpi-mipi-camera1.dtsi

RK3568

The MIPI camera dts configuration for the RK3568 platform (K1/K1B) is as follows:

kernel-5.10/arch/arm64/boot/dts/rockchip/rk3568-kickpi-mipi-camera-gc5035.dtsi

RK3576

The MIPI camera dts configuration for the RK3576 platform (K7) is as follows:

kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7-cam0.dtsi
kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7-cam1.dtsi
kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7-cam3.dtsi

Single Camera Configuration

Operation Example: Taking K7 using cam0 as a single camera example, modify the dts configuration: enable the cam0 device tree header file and comment out other camera device tree header files.

--- a/kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7.dtsi
+++ b/kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7.dtsi
@@ -29,8 +29,8 @@
 #include "rk3576-kickpi-ir.dtsi"

+#include "rk3576-kickpi-k7-cam0.dtsi"
-#include "rk3576-kickpi-k7-cam1.dtsi"
-#include "rk3576-kickpi-k7-cam3.dtsi"
+//#include "rk3576-kickpi-k7-cam1.dtsi"
+//#include "rk3576-kickpi-k7-cam3.dtsi"

Multi-Camera Configuration

Operation Example: Taking K7 using three cameras as an example, modify the dts configuration: enable the cam0, cam1, and cam3 device tree header files.

--- a/kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7.dtsi
+++ b/kernel-6.1/arch/arm64/boot/dts/rockchip/rk3576-kickpi-k7.dtsi
@@ -29,8 +29,8 @@
 #include "rk3576-kickpi-ir.dtsi"

+#include "rk3576-kickpi-k7-cam0.dtsi"
+#include "rk3576-kickpi-k7-cam1.dtsi"
+#include "rk3576-kickpi-k7-cam3.dtsi"

v4l2-ctl

v4l2-ctl is a command-line tool in Linux systems for controlling V4L2 (Video4Linux2) video devices, primarily used to configure and query parameters of devices like cameras and video capture cards.

Install Tool

sudo apt update
sudo apt install v4l-utils

Camera Nodes

List camera devices:

sudo v4l2-ctl --list-devices

Operation Example: From rkisp_mainpath, we can see there are two cameras, typically operating on the first corresponding devices /dev/video33 and /dev/video42.

root@linaro-alip:/#sudo v4l2-ctl --list-devices
...
rkisp_mainpath (platform:rkisp-vir0):
        /dev/video33
        /dev/video34
        /dev/video35
        /dev/video36
        /dev/video37
        /dev/video38
        /dev/video41
        /dev/media3

rkisp_mainpath (platform:rkisp-vir1):
        /dev/video42
        /dev/video43
        /dev/video44
        /dev/video45
        /dev/video46
        /dev/video47
        /dev/video50
        /dev/media4
...

Camera Information

Query the detailed formats supported by a specific camera node:

sudo v4l2-ctl --list-formats-ext --device=/dev/video*

Operation Example: View information for the camera at node /dev/video33.

root@linaro-alip:/#sudo v4l2-ctl --list-formats-ext --device=/dev/video33
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'UYVY' (UYVY 4:2:2)
                Size: Stepwise 32x32 - 3840x2160 with step 8/8
        [1]: 'NV16' (Y/UV 4:2:2)
                Size: Stepwise 32x32 - 3840x2160 with step 8/8
        [2]: 'NV61' (Y/VU 4:2:2)
                Size: Stepwise 32x32 - 3840x2160 with step 8/8
        [3]: 'NV21' (Y/VU 4:2:0)
                Size: Stepwise 32x32 - 3840x2160 with step 8/8
        [4]: 'NV12' (Y/UV 4:2:0)
                Size: Stepwise 32x32 - 3840x2160 with step 8/8
        [5]: 'NM21' (Y/VU 4:2:0 (N-C))
                Size: Stepwise 32x32 - 3840x2160 with step 8/8
        [6]: 'NM12' (Y/UV 4:2:0 (N-C))
                Size: Stepwise 32x32 - 3840x2160 with step 8/8

Camera Frame Rate

sudo v4l2-ctl -d /dev/video* \
 --set-fmt-video=width=1920,height=1080,pixelformat='MJPG' \
 --stream-mmap=4 \
 --set-selection=target=crop,flags=0,top=0,left=0,width=1920,height=1080 \
 --stream-count=500

Operation example:

root@linaro-alip:/#sudo v4l2-ctl -d /dev/video33 \
 --set-fmt-video=width=3840,height=2160\
 --stream-mmap=4 \
 --set-selection=target=crop,flags=0,top=0,left=0,width=3840,height=2160 \
 --stream-count=500
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<^C
root@linaro-alip:/#

Note

Command Analysis:
1. v4l2-ctl -d /dev/video33:
v4l2-ctl: Video4Linux2 control tool for configuring and controlling V4L2-compatible video devices.
-d /dev/video33: Specifies the video device to operate on as /dev/video33
2. --set-fmt-video=width=1920,height=1080,pixelformat='MJPG': Configures the video output format.
width=1920,height=1080: Sets the video resolution to 1920x1080 (1080P)
pixelformat='MJPG': Sets the pixel format to MJPEG (Motion JPEG, a compression format)
3. --stream-mmap=4: Enables memory mapping (mmap) for video stream capture, =4 allocates 4 buffers for stream capture (multiple buffers can reduce frame loss).
4. --set-selection=target=crop,flags=0,top=0,left=0,width=1920,height=1080: Sets the video cropping area.
target=crop: Specifies the operation target as the crop area.
flags=0: No special flags (default).
top=0,left=0,width=1920,height=1080: Sets the crop area starting from the top-left corner (0,0) with a size of 1920x1080 (i.e., using the full frame, no cropping).
5. --stream-count=500: Specifies the number of video frames to capture as 500 frames; automatically stops after capture is complete.

GStreamer

GStreamer is a cross-platform, open-source multimedia framework. Its core purpose is to be a "pipeline engine" for multimedia processing – it breaks down functions like audio/video capture, encoding, decoding, filtering, transmission, and rendering into independent "plugins", and then uses flexible "pipeline" combinations to quickly implement complex multimedia applications (such as video playback, live streaming, audio/video transcoding, camera capture, etc.).

Install Tool

sudo apt update
sudo apt install gstreamer1.0-tools
sudo apt install gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly

Command Line Capture Picture

sudo gst-launch-1.0 v4l2src num-buffers=1 device=/dev/video* ! jpegenc ! filesink location=picture.jpg

Command Line Preview

Using autovideosink:

sudo gst-launch-1.0 v4l2src device=/dev/video* ! video/x-raw,format=NV12,width=800,height=600,framerate=30/1 ! videoconvert ! autovideosink

Using waylandsink (Use this method for preview on GNOME Wayland architecture desktops):

sudo gst-launch-1.0 v4l2src device=/dev/video* ! video/x-raw,width=1280,height=720,framerate=30/1 ! videoconvert ! waylandsink

Operation Example:

sudo gst-launch-1.0 v4l2src device=/dev/video33 ! video/x-raw,width=800,height=600,framerate=30/1 ! videoconvert ! waylandsink

Note

Command Analysis:
1. gst-launch-1.0: GStreamer's command-line tool for quickly building and running GStreamer pipelines.
2. v4l2src device=/dev/video33:
v4l2src: Video4Linux2 source element, used to capture data from V4L2-compliant video devices (like cameras).
device=/dev/video33: Specifies the video device file to use, here it's /dev/video33 (the camera number may vary for different devices).
3. video/x-raw,format=NV12,width=800,height=600,framerate=30/1: Used to specify the video format.
video/x-raw: Indicates raw, uncompressed video.
format=NV12: Video pixel format is NV12 (a common YUV format suitable for hardware acceleration).
width=800,height=600: Video resolution is 800x600.
framerate=30/1: Frame rate is 30 frames per second.
4. videoconvert: Video format conversion element, used to convert the input video format to a format supported by subsequent elements (here mainly to adapt to the requirements of the display element).
5. autovideosink: Automatically selects an appropriate video output element; it chooses the best video display method based on the system environment (such as X11, Wayland, or direct rendering).