MV Camera Application Development Guide on Rochchip

查看中文

Application Layer Usage and Development Guide for MV/RAW Series Camera Modules on the Rockchip Platform

1 Overview

This document applies to scenarios where the camera hardware is correctly connected, the driver is properly installed, and the MV/RAW series camera has been successfully recognized.

The main objectives of this guide are to:

• Query the device information of the connected MV/RAW series camera
• Configure and prepare the camera operating modes
• Introduce several methods for previewing and capturing images
• Explain camera parameter configuration
• Provide guidance for customer-specific development

On the Rockchip platform, regardless of the method used to access the camera, it is necessary to complete device detection and media-ctl configuration before performing any further operations. Therefore, the following sections first cover device detection and media-ctl setup.

2 Device Detection and Environment Configuration

Here, we provide two scripts that can automatically retrieve key information about the connected camera.

2.1 probe_camera_info-rk.sh

This script is used to probe connected and successfully registered camera devices, retrieving underlying information such as the corresponding media device node, video device node, sub-device node, I²C bus, and device name.

After execution, an auto_camera_index.json file will be generated in the current directory, containing the retrieved information.

Example usage:

$ ./probe_camera_info-rk.sh

cat auto_camera_index.json

[

  {

    "media_node": "/dev/media0",

    "video_node": "/dev/video0",

    "video_subnode": "/dev/v4l-subdev2",

    "media_entity_name": "m00_b_mvcam 7-003b",

    "i2c_bus": "7"

  }

]

Each {} block represents a single camera. If the board supports multiple camera modules, multiple {} blocks will be present in the file.

Explanation of Camera Information：

Camera Information

Field	Name	Purpose	Usage
media_node	Media device node	Used to access the device within the media-controller framework	Used when configuring resolution and format via the media-ctl command
video_node	Video capture device node	Standard V4L2 video device	Used with v4l2-ctl or customer applications to capture images
video_subnode	V4L2 sub-device node	Used for configuring certain camera parameters	Accessed via v4l2-ctl commands
media_entity_name	Media entity name	Describes the device, e.g., "m00_b_mvcam 7-003b"	Used when setting resolution and format with media-ctl
i2c_bus	I²C bus	Indicates the I²C bus to which the device is connected	Used as the underlying communication channel for camera parameter configuration, e.g., by the mv_mipi_i2c.sh script

The media device node, video device node, sub-device node, I²C bus, and device name used in the following sections can all be replaced with the corresponding information obtained from the JSON file generated by this probe script.

2.2 mv_probe.sh

The mv_probe.sh script can configure environment variables with key information for a specific camera, including the I²C bus number, camera model, resolution (width and height), and frame rate. This simplifies subsequent use of media-ctl for format configuration.

The usage method is:

$ source ./mv_probe.sh <i2c_bus>

Example:

$ source ./mv_probe.sh 7

Found veye_mvcam camera on i2c-7.

Setenv I2C_BUS = 7

Setenv CAMERAMODEL = MV-MIPI-AR0234M

Setenv FPS = 60

Setenv WIDTH = 1920

Setenv HEIGHT = 1200

You can verify the environment variable output using:

echo $CAMERAMODEL

Note that these environment variables are only valid for the current session.

Important Notes:

• This script requires the mvcam driver version 1.1.06 or later.
• If your driver version is earlier than 1.1.06 or you need to use different width, height, or frame rate values, refer to the camera module manual and manually configure the following environment variables. Otherwise, subsequent programs may not function correctly.

Example:

export WIDTH=1920

export HEIGHT=1200

export FPS=30

3 Configuring Formats with media-ctl

3.1 Viewing the Topology with media-ctl

By using the "media-ctl" command, the current topology structure can be clearly displayed.

media-ctl -p -d /dev/media0

3.1.1 Link relationship

mv camera->rockchip-csi2-dphy0->rockchip-mipi-csi2->stream_cif_mipi_id0 - - ->DDR(/dev/video0)

3.1.2 mv camera entity information

Take GX-MIPI-IMX662 as an example:

- entity 63: m00_b_mvcam 7-003b (1 pad, 1 link)

             type V4L2 subdev subtype Sensor flags 0

             device node name /dev/v4l-subdev2

        pad0: Source

                [fmt:Y8_1X8/1920x1200@100/6000 field:none

                 crop.bounds:(0,0)/1920x1200

                 crop:(0,0)/1920x1200]

                -> "rockchip-csi2-dphy0":0 [ENABLED]

It can be seen that:

• The complete name of this Entity is: m00_b_mvcam 7-003b. (On the ROC-RK3566-PC, the name of this Entity is m00_b_mvcam 4-003b).
• It is a V4L2 subdev (Sub-Device) sensor.
• The corresponding node is /dev/v4l-subdev2. The application (such as v4l2-ctl) can open it and make configurations.
• Its output format is [fmt:Y8_1X8/1920x1200@100/6000 field:none], where Y8_1X8 is a shortened form of an mbus-code. The supported mbus-codes will be listed in the next section.
• The current resolution is 1920x1200.
• The current frame interval is 10000/600000, which means the frame rate is 60.

The data format output by the camera can be modified through the "media-ctl" command.

3.2 Camera-supported MBUS code

The MV/RAW series cameras utilize the data format capability. For details, please refer to the data manuals of each model of the cameras.

Format correspondence relationship

Format on datasheet	mbus-code for media-ctl	FourCC pixelformat for v4l2-ctl
RAW8	Y8_1X8	GREY
RAW10	Y10_1X10	'Y10 '
RAW12	Y12_1X12	'Y12 '
UYVY	UYVY8_2X8	UYVY

3.3 Configuring Image Formats with media-ctl

The camera data format, resolution, and frame rate can be configured using the following command:

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_mvcam '"$I2C_BUS"'-003b":0[fmt:Y8_1X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'

• "m00_b_mvcam 7-003b" specifies the full entity name of the camera.
• Y8_1X8 is the mbus-code.
• "$WIDTH"x"$HEIGHT" indicates the resolution.
• 1/"$FPS" specifies the frame rate.

The width and height here cooperate with the roi_x and roi_y of the v4l2-ctl command to form the ROI parameter.

For example, for MV-MIPI-AR0234M, the command after variable replacement would be:

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_mvcam 7-003b":0[fmt:Y8_1X8/1920x1200@1/60 field:none]'

You can not only configure the data format, resolution, and frame rate in one command, but also modify them separately as needed.

4 Raw data format

The VICAP module of RK3588 supports two data saving formats, Compact and Noncompact RAW. You can modify the mode using the RKCIF_CMD_SET_CSI_MEMORY_MODE ioctl command of RKCIF. By default, the output is in Compact RAW format.

Compact raw and noncompact raw of rk3588 VICAP

4.1 Noncompact RAW

For pixel data with 10-bit depth or 12-bit depth, two bytes are always used to store one pixel. This storage method is convenient for software processing, but it has the disadvantage of occupying a large amount of space.

Depending on whether the effective data is stored in the high bits or low bits, it can be further divided into two types: high align and low align.

4.1.1 Noncompact RAW(high align)

Data is saved to the high bits, and the unused low bits are filled with 0. This is one of the data formats supported by RK VICAP.

4.1.2 Noncompact RAW(low align)

In Noncompact RAW (low align) format, data is saved to the low bits, and the unused high bits are filled with 0. The V4L2 standard 'Y10' (10-bit Greyscale) and 'Y12' (12-bit Greyscale) formats are both stored in this way.

The pixel_layer_convert conversion tool mentioned later in the article also converts Compact RAW to this storage format for easy display using image players.

4.2 Compact RAW

As shown above，there is no bit padding between pixels in this storage format.

4.3 Line stride

To facilitate fast operations on images, the system usually provides row-aligned buffer sizes for each line of data. RK3588 uses 256-byte alignment for this purpose.

line_stride = ALIGN_UP（image_width*bits_per_pixel/8,256）

For example, when the image width is 1456:

8bit depth，line_stride=1536

10bit depth，preferred_stride=2048

12bit depth，preferred_stride=2304

4.4 Format convert tool

We have written a small tool: pixel_layer_convert, which can easily convert Compact images to Noncompact (low align) images.

For example, the following command can convert a Compact RAW10 image with a width of 1456 to Noncompact RAW10 format:

./pixel_layer_convert -I R10C -i y10-1456x1088_0001.raw -o y10-1456x1088_0001_new.raw -w 1456

4.5 Raw data image player

We recommend using vooya as the player, which supports GREY, and unpacked image formats.

Also, y8 file can be used with this player: YUV Displayer Deluxe.

4.6 veye_viewer

The veye_viewer is an open-source, Qt-based client application that allows users to easily evaluate cameras and configure parameters.

Its operation logs, register listings, and open-source nature provide a convenient reference for users and support custom development.

The source code of veye_viewer can be downloaded here, or platform-specific executable programs are available directly in its release packages.

5 yavta

5.1 Installing Yavta

git clone git://git.ideasonboard.org/yavta.git

$ cd yavta;make

5.2 Saving Images to a File

After configuring the data format, resolution, and frame rate, execute the following command to capture and save images:

./yavta -c1 -Fy8-${WIDTH}x${HEIGHT}.raw --skip 0 -f Y8 -s ${WIDTH}x${HEIGHT} /dev/video0

6 Importing Camera Data into OpenCV

Install OpenCV for Python:

$ sudo apt install python3-opencv

Example scripts can be found in the samples directory of the GitHub repository.

Run the sample program with appropriate parameters:

$ python3 ./v4l2dev_2_opencv_show_grey.py --width 1456 --height 1088 --fps 60 --i2c 6

Note: Ensure that the program is executed with the correct parameters matching your camera setup.

7 GStreamer Examples

Several GStreamer example pipelines are provided to implement camera preview functionality. Detailed examples are available in the samples directory on GitHub.

8 v4l2-ctl Examples

The following example demonstrates how to configure the camera and capture images directly from the command line using /dev/v4l-subdev2, /dev/media0, and /dev/video0.

8.1 install v4l2-utils​

sudo apt-get install v4l-utils

8.2 Configure parameters using v4l2-ctl

$ v4l2-ctl -d /dev/v4l-subdev2 -L

User Controls

                    trigger_mode 0x00981901 (int)    : min=0 max=2 step=1 default=0 value=0 flags=volatile, execute-on-write

                    trigger_src 0x00981902 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write

                    soft_trgone 0x00981903 (button) : flags=write-only, execute-on-write

                     frame_rate 0x00981904 (int)    : min=1 max=60 step=1 default=60 value=60 flags=volatile, execute-on-write

                          roi_x 0x00981905 (int)    : min=0 max=1376 step=8 default=0 value=0

                          roi_y 0x00981906 (int)    : min=0 max=1024 step=4 default=0 value=0

Parameters can be set and get using the following methods.

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl [ctrl_type]=[val]

v4l2-ctl -d /dev/v4l-subdev2 --get-ctrl [ctrl_type]