View source for GX Camera Application Development Guide on Rochchip

'''Application Layer Usage and Development Guide for GX Series Camera Modules on the Rockchip Platform'''

=== Outline ===
This document is applicable to scenarios where the driver has been correctly installed, the hardware connections are correct, and the camera has been properly recognized. 

This article mainly aims to achieve the following goals:

* Query the device information of the GX series cameras that have been connected. 
* Complete the setting and preparation of the camera working mode;
* Introduce several methods for previewing and taking screenshots;
* Introduce the method of parameter configuration;
* Guide the customer for further development.

For the Rockchip platform, regardless of the method used to call the camera, the steps of device detection and media-ctl configuration format must be completed before proceeding with the subsequent operations. Therefore, the following will first introduce the device detection and media-ctl configuration.

Note that the veye_viewer client has already incorporated device detection and media-ctl configuration functions. Therefore, after completing the driver installation and confirming the hardware connection, you can directly refer to the veye_viewer client section described later to preview the camera images.
<br />

=== The equipment detects and configures the environment variables. ===
[https://github.com/veyeimaging/rk35xx_veye_bsp/tree/main/gx_i2c_tools Here], we have provided two scripts that can automatically retrieve some information about the camera.

==== probe_camera_info-rk.sh ====
This script is used to detect the connected and successfully registered camera devices, and retrieve the underlying information such as the media device nodes, video device nodes, sub-device nodes, I²C bus and device names corresponding to the devices.

After execution, an "auto_camera_index.json" file will be generated in the current directory, and the retrieved information will be recorded in this file. 

Here is an example:

<code>$ ./probe_camera_info-rk.sh</code>

<code>cat auto_camera_index.json</code>

<code>[</code>

<code>  {</code>

<code>    "media_node": "/dev/media0",</code>

<code>    "video_node": "/dev/video0",</code>

<code>    "video_subnode": "/dev/v4l-subdev2",</code>

<code>    "media_entity_name": "m00_b_gxcam 7-003b",</code>

<code>    "i2c_bus": "7"</code>

<code>  }</code>

<code>]</code>

The content within {} represents information about a camera. If the motherboard supports multiple camera modules, multiple sets of {} will appear, each containing the corresponding camera information. 

The explanation of the camera information is as follows:
{| class="wikitable"
|+Camera information
!antonomasia
!designation
!effect
!Where used
|-
|media_node
|Media device node
|Used for accessing devices within the media-controller framework
|Used when configuring the resolution and format through the "media-ctl" command
|-
|video_node
|Video capture device node
|Typical V4L2 video device
|v4l2-ctl or the programs developed by the customers, are used to obtain the images.
|-
|video_subnode
|V4L2 sub-device node
|When configuring the parameters of the camera, use
|The v4l2-ctl command is used
|-
|media_entity_name
|Media entity name
|Describe the name of the equipment, for example

<code>"m00_b_gxcam 7-003b"</code>
|Used when configuring the resolution and format through the "media-ctl" command
|-
|i2c_bus
|I²C bus number
|The I²C bus number indicating the device connection
|The underlying communication channel for configuring the camera parameters, the gx_mipi_i2c.sh script uses
|}
The media device nodes, video device nodes, sub-device nodes, I²C bus and device names used in the following text can all be replaced with the information in the JSON file obtained from this detection script.

==== gx_probe.sh ====
By using the gx_probe.sh script, the I²C bus number corresponding to a certain camera, the camera model, width, height, frame rate, etc. can be configured into the environment variables. 

In this way, it will facilitate the subsequent use of the media-ctl configuration format, making it easier to use. 

The usage method is:

<code>$ source ./gx_probe.sh i2c_bus</code>

A typical output is as follows:

<code>$ source ./gx_probe.sh 7</code>

<code>Found veye_gxcam camera on i2c-7.</code>

<code>Setenv I2C_BUS = 7</code>

<code>Setenv CAMERAMODEL = GX-MIPI-IMX662</code>

<code>Setenv FPS = 60</code>

<code>Setenv WIDTH = 1920</code>

<code>Setenv HEIGHT = 1080</code>

You can verify the output result of the environment variable by using the command "echo $CAMERAMODEL". '''Note that this environment variable is only valid for the current session.'''

=== Using the media-ctl configuration format ===

==== Use media-ctl to view the topology structure ====
By using the "media-ctl" command, the current topology structure can be clearly displayed.

<code>media-ctl -p -d /dev/media0</code>

===== Link relationship =====
gx camera->rockchip-csi2-dphy0->rockchip-mipi-csi2->stream_cif_mipi_id0 - - ->DDR(/dev/video0)

The application can obtain images through the /dev/video0 node.

===== gx camera entity information =====
Take GX-MIPI-IMX662 as an example:

<code>- entity 63: m00_b_gxcam 7-003b (1 pad, 1 link)</code>

<code>             type V4L2 subdev subtype Sensor flags 0</code>

<code>             device node name /dev/v4l-subdev2</code>

<code>        pad0: Source</code>

<code>                [fmt:UYVY8_2X8/1920x1080@10000/600000 field:none colorspace:rec709</code>

<code>                 crop:(0,0)/1920x1080]</code>

<code>                -> "rockchip-csi2-dphy0":0 [ENABLED]</code>

can see:

* The complete name of this Entity is: m00_b_gxcam 7-003b. (On the ROC-RK3566-PC, the name of this Entity is m00_b_gxcam 4-003b). 
* It is a V4L2 subdev (Sub-Device) Sensor.
* The corresponding node is /dev/v4l-subdev2. Applications (such as v4l2-ctl) can open it and configure it.
* Its output format is [UYVY8_2X8/1920x1080@10000/600000 field:none], where UYVY8_2X8 is a shorthand for an mbus-code. The supported mbus-codes will be listed in the next section. The current resolution is 1920x1080.
* The current frame interval is 10000/600000, which means the frame rate is 60.

The data format output by the camera can be modified through the "media-ctl" command.

==== Camera-supported MBUS code ====
The GX series cameras utilize the UYVY data format capability. For specific details, please refer to the data manuals of each model of the cameras.
{| class="wikitable"
|+Format correspondence relationship
!Format on datasheet
!mbus-code for media-ctl
!FourCC pixelformat for v4l2-ctl
|-
|UYVY
|UYVY8_2X8
|UYVY
|}

==== Configure the image format using media-ctl ====
The data format, resolution and frame rate of the camera can be configured through the following command.

<code>media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'</code>

Here: "m00_b_gxcam 7-003b" refers to the complete entity name of the camera. "UYVY8_2X8" is the mbus-code, and "$WIDTH"x"$HEIGHT" indicates the resolution, while 1/"$FPS" indicates the frame rate.

For example, for GX-MIPI-IMX662, the result after variable substitution of this command is:

<code>$ media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam 7-003b":0[fmt:UYVY8_2X8/1920x1080@1/60 field:none]'</code>

Not only can you configure the data format, resolution and frame rate all at once in one command, but you can also modify individual items or several items separately. 

Note that the names of each node in media-ctl can be adjusted according to the content of the aforementioned JSON file to configure different cameras.

=== veye_viewer client ===
"veye_viewer" is an open-source client software based on QT, which enables customers to conduct camera evaluations and parameter configurations conveniently. Additionally, its operation logs, register list, and open-source features facilitate customer reference and enable secondary development.

The source code can be downloaded via [https://github.com/veyeimaging/veye_viewer veye_viewer], or the corresponding platform executable program can be directly downloaded from its distribution.

=== Example of Yavta Application ===

==== Yavta Installation ====
<code>git clone <nowiki>git://git.ideasonboard.org/yavta.git</nowiki></code>

<code>$ cd yavta;make</code>

==== Save the picture to a file ====
After configuring the data format, resolution and frame rate, execute:

<code>./yavta -c1 -Fuyvy-${WIDTH}x${HEIGHT}.yuv --skip 0 -f UYVY -s ${WIDTH}x${HEIGHT} /dev/video0</code>

=== Example of importing camera data into OpenCV ===
<code>$ sudo apt install python3-opencv</code>

Please refer to the "[https://github.com/veyeimaging/rk35xx_veye_bsp/tree/main/samples samples]" directory on GitHub for details.

<code>$ python3 ./v4l2_opencv_show2.py --width 1920 --height 1080 --fps 60 --i2c 7</code>

Make sure to use the appropriate parameters to execute the above procedure.

=== GStreamer Application Example ===
We have provided several GStreamer routines that implement the preview function. Please refer to the "[https://github.com/veyeimaging/rk35xx_veye_bsp/tree/main/samples samples]" directory on GitHub for details.

=== V4L2-CTL Application Examples ===
Taking <code>/dev/v4l-subdev2</code> <code>/dev/media0</code> <code>/dev/video0</code> as an example, this article introduces the method of directly configuring the camera and obtaining images using the command line.

==== Install v4l2-utils ====
<code>sudo apt-get install v4l-utils</code>

==== Configure parameters using v4l2-ctl ====
<code>v4l2-ctl -d /dev/v4l-subdev2 -L</code>

<code>User Controls</code>

<code>                   work_mode 0x00981a01 (int)    : min=0 max=4 step=1 default=0 value=0 flags=volatile, execute-on-write</code>

<code>                    trigger_src 0x00981a02 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write</code>

<code>                    soft_trgone 0x00981a03 (button) : value=0 flags=write-only, execute-on-write</code>

<code>                      sync_role 0x00981a04 (int)    : min=0 max=1 step=1 default=0 value=0 flags=volatile, execute-on-write</code>

<code>                     frame_rate 0x00981a05 (int)    : min=0 max=60 step=1 default=60 value=60 flags=volatile, execute-on-write</code>

The parameters can be set and retrieved using the following methods.

<code>v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl [ctrl_type]=[val]</code>

<code>v4l2-ctl -d /dev/v4l-subdev2 --get-ctrl [ctrl_type]</code>All of the above functions can be achieved by using the [https://wiki.veye.cc/index.php/Gx_mipi_i2c.sh_user_guide gx_mipi_i2c.sh] script. 

It should be noted that all the above parameters cannot be modified during the image generation process. The following will provide detailed explanations for each one:

===== Configure trigger mode =====
<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>work_mode=[0-2]</small></code><br />0: Streaming Mode 

1: Ordinary Trigger Mode 

4: Multi-camera Synchronization Mode

===== Configure trigger source =====
<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>trigger_src=[0-1]</small></code>

0: Soft Trigger 

1: Hard Trigger

===== Soft trigger once =====
<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>soft_trgone=1</small></code>

===== Set frame rate =====
<code>v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl frame_rate=[1-max]</code>

As the resolution is adjusted, the maximum frame rate will be automatically updated.

==== stream mode ====

===== Set the data format, resolution, and frame rate =====
<code>media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'</code>

===== 帧率统计 =====
<code>v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to=/dev/null</code>

or

<code>./yavta -c-1 --skip 0 -f UYVY -s ${WIDTH}x${HEIGHT} /dev/video0</code>

===== Save the picture to a file =====

* UYVY format

v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-${WIDTH}x${HEIGHT}.yuv

==== Trigger mode ====

===== Set the data format, resolution, and frame rate =====
<code>media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'</code>

example:<code>$ media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam 7-003b":0[fmt:UYVY8_2X8/1920x1080@1/60]'</code>

===== Soft Trigger Mode =====

====== Setup Mode ======
<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>work_mode=1</small></code>

<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>trigger_src=0</small></code>

====== Start taking pictures ======
<code>v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-${WIDTH}x${HEIGHT}.yuv</code>

====== Perform a hard trigger operation ======
Connect the appropriate trigger signal to the trigger pin of the camera and trigger it.

==== synchronous mode ====
The following nodes and sub-devices should be differentiated according to the specific circumstances.

===== Set the data format, resolution, and frame rate =====
<code>media-ctl -d /dev/media0 --set-v4l2 '"m00_b_gxcam '"$I2C_BUS"'-003b":0[fmt:UYVY8_2X8/'"$WIDTH"'x'"$HEIGHT"'@1/'"$FPS"']'</code>

===== Set the synchronization mode =====
<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl work<small>_mode=4</small></code>

===== Set up the master and slave cameras =====
Main camera:

<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>sync_role=0</small></code>

slave camera:

<code>$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl <small>sync_role=1</small></code>

===== Start taking pictures =====
The method of taking screenshots in the synchronization mode is exactly the same as that in the video stream mode.

=== I2C Script Usage Instructions ===
We provided shell scripts to configure the parameters.

[https://wiki.veye.cc/index.php/Gx_mipi_i2c.sh_user_guide gx_mipi_i2c.sh user guide]

=== Some suggestions for client integration development ===

=== initialization phase ===
The resolution and frame rate configurations need to be completed. It is recommended that customers directly integrate the calls of media-ctl and v4l2-ctl commands during the program initialization stage to complete the configuration of resolution, frame rate, and data format. 

Do not use gx_mipi_i2c.sh to directly set the camera registers, because if you directly set the camera registers, the Rockchip Linux driver layer cannot synchronize to the set values. 

On the Rockchip platform, for a specific motherboard, the nodes detected by the script ./probe_camera_info-rk.sh remain constant. It is recommended that customers directly use the JSON file output by this script as the configuration file for their own programs. 

The customer can also use the output of this script as a reference and directly hard-code the node names in the code.

==== working stage ====
Depending on the programming language they use, customers can refer to the aforementioned code and tools in this article. 

Regarding the timestamp, the v4l2_buffer.timestamp field contains the exact time when the RK chip received a complete data frame. It can be used as a reference timestamp for camera synchronization or synchronization with other external sensors.

==== parameter configuration ====
The parameters of the GX series cameras are very rich and flexible. They are mainly achieved through the direct register configuration method using gx_mipi_i2c.sh. For details, please refer to the register documentation and the description document of gx_mipi_i2c.sh. 

The gx_mipi_i2c.sh script has the "paramsave" function, which is designed to save all the configuration parameters of the customer into the camera's flash memory. However, it is not recommended for the customer to frequently call this function within the program.

=== This document's revision history ===

* 2025-12-06

Adjust the document structure and add some sections. 

* 2025-11-28

The first version.