GX Camera on Firfly Boards

查看中文

1 Overview

The GX series cameras are designed for AI applications in the industrial field. They use the MIPI CSI-2 interface and are particularly suitable for embedded computing platforms. They feature a wide range of data formats and triggering capabilities, extremely low latency, high bandwidth, and reliable stability.

This article takes the Firefly's ROC-RK3588S-PC, ROC-RK3576-PC and ROC-RK3566-PC motherboards as examples to introduce how to connect GX series cameras to the RK3566/RK3568, RK3576 and RK3588S/RK3588 systems.

We provide drivers for the Linux operating system (taking Ubuntu as an example).

1.1 Camera Module List

Series	Model	Status
GX series	GX-MIPI-IMX662	Done

In addition, the driver for the V-by-One HS connection mode has been finished on the Ubuntu system.

2 Hardware Setup

The GX series and RAW series cameras require an ADP-MV2 adapter board to connect to the ROC-RK35xx-PC motherboard.

2.1 Connection of new ADP-MV2

2.1.1 Connection of GX series camera and ADP-MV2

The two are connected using 0.5 mm pitch*30P FFC cable with opposite-side contacts. The cable must be inserted with the silver contacts facing outside.

TOP	BOTTOM
ADP-MV2 to MV-MIPI-X	ADP-MV2 to MV-MIPI-X

2.1.2 Connection of RAW-MIPI-SC132M and ADP-MV2

The two are connected using 1.0 mm pitch*15P FFC cable with opposite-side contacts. The cable must be inserted with the silver contacts facing outside.

TOP	BOTTOM
ADP-MV2 to RAW-MIPI-SC132M	ADP-MV2 to RAW-MIPI-SC132M

2.1.3 Connection of other RAW series camera and ADP-MV2

The two are connected using 0.5 mm pitch*pin FFC cable with opposite-side contacts. The cable must be inserted with the silver contacts facing outside.

TOP	BOTTOM
ADP-MV2 to RAW series camera	ADP-MV2 to RAW series camera

2.1.4 Connection with Main board using ADP-MV2

The two are connected using 0.5mm pitch * 30P FFC coaxial wires, paying attention to the direction of the contact surfaces, silver contacts facing outside on the ADP-MV2 and facing inside on the RK board.

RK to ADP-MV2 and MV cam

2.2 V-by-One-HS-KIT Camera Connection Diagram

VBYONE connection with rk3588

3 Introduction to github repositories

https://github.com/veyeimaging/rk35xx_veye_bsp

https://github.com/veyeimaging/rk35xx_firefly

includes：

• driver source code
• i2c toolkits
• application demo

In addition, a compiled linux kernel installation package and Android image is provided in the releases.

4 Upgrade Firefly Ubuntu system

For the ROC-RK3566-PC,ROC-RK3576-PC and ROC-RK3588S-PC, we have provided an image of the release system.

Download the latest Ubuntu imge from https://github.com/veyeimaging/rk35xx_firefly/releases/ .

Refer to the Firefly documentation ROC-RK3588S-PC ROC-RK3566-PC ROC-RK3576-PC to burn in a standard system.

5 Check system status

5.1 Whether the camera is correctly recognized

After system update, reboot the main board.

Execute the following command on the main board to check if the camera is properly connected.

dmesg | grep gxcam

You can see the camera model and the camera version number probed.

A prompt as below indicates that the GX-MIPI-IMX662 camera is detected on the i2c-7 bus.

[6.667547] gxcam 7-003b: veye gx series camera driver version: 01.00.01

[6.781681] gxcam 7-003b: camera is: GX-MIPI-IMX662

[6.820210] gxcam 7-003b: Success to get gxcam endpoint data lanes, dts uses 2 lanes,will set to camera

[6.834597] gxcam 7-003b: gxcam_enum_controls success

[6.891209] rockchip-csi2-dphy csi2-dphy0: dphy0 matches m00_b_gxcam 7-003b:bus type 5

On the ROC-RK3588S-PC, the camera is mounted on i2c-7, with an i2c address of 0x3b.

On the ROC-RK3566-PC and ROC-RK3576-PC, the camera is mounted on i2c-4.

• Check the video0 device node:

ls /dev/video0

You should see:

video0

After successfully identifying the camera, the camera will be recognized as /dev/video0.

5.2 State Detection and Environment Variable Configuration

Here, We have provided two scripts that can automatically retrieve some information about the camera.

First, try using the probe_camera_info-rk.sh script. This script is used to detect the connected and successfully registered camera devices, retrieve the corresponding media device nodes, video device nodes, sub-device nodes, I²C buses, and device identifiers and other low-level information. After execution, it will generate an auto_camera_index.json file in the current directory and record the retrieved information in the file.

./probe_camera_info-rk.sh

cat auto_camera_index.json

[

  {

    "media_node": "/dev/media0",

    "video_node": "/dev/video0",

    "video_subnode": "/dev/v4l-subdev2",

    "media_entity_name": "m00_b_gxcam 7-003b",

    "i2c_bus": "7"

  }

]

By referring to the index information, we can see the "i2c_bus": "7" corresponding i2c_bus information, as well as the number of devices connected. The current index shows that only one device is connected, and the i2c_bus number is 7. If multiple devices are connected, the index information may include "i2c_bus": "10", "i2c_bus": "11" and so on. Then, using the gx_probe.sh script, if it is a multi-camera system, the i2c_bus can be executed based on the information read from the previous script, and the corresponding camera model, width, height, frame rate and other information can be configured in the environment variables.

Usage:

source ./gx_probe.sh 7

A typical output:

$ source ./gx_probe.sh 7

Found veye_gxcam camera on i2c-7.

Setenv CAMERAMODEL = GX-MIPI-IMX662

Setenv FPS = 60

Setenv WIDTH = 1920

Setenv HEIGHT = 1080

You can verify the environment variable output using:

echo $CAMERAMODEL

Note that these environment variables are only valid for the current session.

5.3 Configuring  global variables

Based on the board model, configure the I2C_BUS global variable as follows:

• ROC-RK3588S-PC

export I2C_BUS=7

• ROC-RK3566-PC and ROC-RK3576-PC

export I2C_BUS=4

5.4 Using media-ctl to view topology

Using the media-ctl command can clearly display the current topography structure.

media-ctl -p -d /dev/media0

5.4.1 Link relationship

gx camera->rockchip-csi2-dphy0->rockchip-mipi-csi2->stream_cif_mipi_id0 - - ->DDR(/dev/video0)

The application can obtain images through the /dev/video0 node.

5.4.2 gx camera entity information

Taking the GX-MIPI-IMX662 as an example:

- entity 63: m00_b_gxcam 7-003b (1 pad, 1 link)

             type V4L2 subdev subtype Sensor flags 0

             device node name /dev/v4l-subdev2

        pad0: Source

                [fmt:UYVY8_2X8/1920x1080@10000/600000 field:none colorspace:rec709

                 crop:(0,0)/1920x1080]

                -> "rockchip-csi2-dphy0":0 [ENABLED]

You can see that:

• The complete name of this entity is: m00_b_gxcam 7-003b.(It is m00_b_gxcam 4-003bon ROC-RK3566-PC.)
• It is a V4L2 subdev (Sub-Device) Sensor.
• Its corresponding node is /dev/v4l-subdev2, which can be opened and configured by applications (such as v4l2-ctl).
• Its output format is [fmt:UYVY8_2X8/1920x1080@100/6000 field:none], where UYVY8_2X8 is a shorthand for a mbus-code, which will be listed in the next section of this article.
• The current resolution is 1920x1080.
• The current frame interval is 100/6000, which means the frame rate is 60.
• The data format output by the camera can be modified using the media-ctl command.

5.4.3 mbus-code list

GX series cameras have different data format capabilities, which can be found in the data manual for each camera model.

Format on datasheet	mbus-code for media-ctl	FourCC pixelformat for v4l2-ctl
UYVY	UYVY8_2X8	UYVY

5.5 veye_viewe工具

可通过veye_viewe下载

6 Application examples

6.1 Configure parameters using v4l2-ctl

$ v4l2-ctl -d /dev/v4l-subdev2 -L

User Controls

                   trigger_mode 0x00981a01 (int)    : min=0 max=4 step=1 default=0 value=0 flags=volatile, execute-on-write

                    trigger_src 0x00981a02 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write

                    soft_trgone 0x00981a03 (button) : value=0 flags=write-only, execute-on-write

                      sync_role 0x00981a04 (int)    : min=0 max=1 step=1 default=0 value=0 flags=volatile, execute-on-write

                     frame_rate 0x00981a05 (int)    : min=0 max=60 step=1 default=60 value=60 flags=volatile, execute-on-write

Parameters can be set and get using the following methods.

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl [ctrl_type]=[val]

v4l2-ctl -d /dev/v4l-subdev2 --get-ctrl [ctrl_type]

All the above functions can be implemented using gx_mipi_i2c.sh.

Note that the above parameters cannot be modified during the capture process.

The following is an explanation of each parameter:

6.1.1 Trigger Mode

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_mode=[0-2]

0：Video streaming mode

1：Normal trigger mode.

4：Multi-camera synchronization mode.

6.1.2 Trigger Source

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_src=[0-1]

0: Software trigger mode.

1: Hardware trigger mode.

6.1.3 Software trigger command

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl soft_trgone=1

6.1.4 Set frame rate

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl frame_rate=[1-max]

The maximum frame rate is automatically updated as the resolution changed.

6.2 Set image format using media-ctl

use the following command to configure the camera's data format, resolution, and frame rate using media-ctl:

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'

Among them: "m00_b_gxcam '"$I2C_BUS"'-003b" refers to the complete name of the camera entity, UYVY8_2X8 is the mbus-code, '"$WIDTH"'x'"$HEIGHT"' indicates the resolution, 1/'"$FPS"' indicates the resolution frame rate.

For example, for GX-MIPI-IMX662, the command after variable replacement would be:

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam 7-003b":0[fmt:UYVY8_2X8/1920x1080@1/60 field:none]'

You can not only configure the data format, resolution, and frame rate in one command, but also modify them separately as needed.

6.3 Video Streaming mode

6.3.1 Set data format, resolution, frame rate

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'

6.3.2 Frame rate statistics

In streaming mode, the following commands can be used for frame rate statistics：

v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to=/dev/null

Or：

./yavta -c1000 --skip 0 -f UYVY -s ${WIDTH}x${HEIGHT} /dev/video0

6.3.3 Save image to file

• UYVY

v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-${WIDTH}x${HEIGHT}.yuv

Please refer to the description in the previous section for the image format.

6.3.4 Example of yavta

6.3.4.1 Install yavta

git clone git://git.ideasonboard.org/yavta.git

cd yavta;make

6.3.4.2 Save image to file

After setting data format, resolution, frame rate，run:

./yavta -c1 -Fuyvy-${WIDTH}x${HEIGHT}.yuv --skip 0 -f UYVY -s ${WIDTH}x${HEIGHT} /dev/video0

6.3.5 Example of import image to OpenCV

sudo apt install python3-opencv

See the samples directory on github for details.

python ./v4l2_opencv_show2.py --width 1920 --height 1080 --fps 60 --i2c 7

6.3.6 Example of gstreamer application

We provide several gstreamer routines that implement the preview function. See the samples directory on github for details.

6.4 Trigger mode

6.4.1 Set data format, resolution, frame rate

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'

6.4.2 Software trigger mode

6.4.2.1 Set mode

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_mode=1

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_src=0

6.4.2.2 Start acquisition

v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-${WIDTH}x${HEIGHT}.yuv

6.4.2.3 Perform soft trigger operation

In other shell terminals, you can execute the following command multiple times for multiple triggers.

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl soft_trgone=1

6.4.3 Hardware trigger mode

6.4.3.1 Set mode

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_mode=1

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_src=1

The gx_mipi_i2c.sh script can be used to set rich trigger parameters.

6.4.3.2 Start acquisition

v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-${WIDTH}x${HEIGHT}.yuv

6.4.3.3 Perform hardware trigger operation

Connect the appropriate trigger signal to the trigger pin of the camera and trigger it.

6.5 synchronous mode

6.5.1 Set data format, resolution, frame rate

media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'

6.5.2 Switch to synchronous mode

Note: The RK platform only supports one camera connection. Perform the following operations on both mainboard terminals to switch to synchronous mode.

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl trigger_mode=4

6.5.3 Set the camera as the master or slave.

master camera：

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl sync_role=0

slave camera：

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl sync_role=1

6.5.4 Start taking pictures

Several methods in the streaming mode can be used to take pictures.

7 i2c script for parameter configuration

We provide shell scripts to configure the parameters.

gx_mipi_i2c.sh user guide

8 Question Feedback

We are committed to providing richer possibilities for image applications on embedded platforms. Therefore, our software for embedded platforms is based on the principle of open source.

If you have any questions or suggestions about our existing software, please feel free to submit them to the forum or email our technical staff at xumm#csoneplus.com.

9 References

• ROC-RK3566-PC Manual

https://wiki.t-firefly.com/en/ROC-RK3566-PC/

• ROC-RK3588S-PC Manual

https://wiki.t-firefly.com/en/ROC-RK3588S-PC/

• ROC-RK3576-PC Manual

https://wiki.t-firefly.com/en/ROC-RK3576-PC/

• Firefly Linux User Guide

https://wiki.t-firefly.com/en/Firefly-Linux-Guide/index.html

10 Document History

• 2025-04-14

Add support for RK3576.

• 2025-03-23

Add description of mv_probe.sh.

• 2024-07-09

Add support for RAW-MIPI-SC535M.

• 2024-04-17

Support RK3566.

• 2024-03-10

Add pictures and descriptions of hardware connections for the new version of ADP-MV2.

• 2023-08-30

Add support for RAW-MIPI-IMX462M and RAW-MIPI-AR0234M.

• 2023-07-31

Support V-by-One on ubuntu system.

• 2023-04-12

Release 1st version.