View source for GX series camera appnotes 4 jetson

[https://wiki.veye.cc/index.php/GX_series_camera_appnotes_4_jetson/zh 查看中文]

===overview===
The GX series cameras are designed for embedded AI applications. They offer high-performance ISP capabilities, support multiple operating modes, provide a wide range of configurable features, and are built with a robust and reliable design. The cameras use the MIPI CSI-2 interface and are well suited for embedded computing platforms. 

This article introduces how to use GX series cameras on the NVIDIA Jetson platform.

====Supported camera modules====
{| class="wikitable"
!series
!model
!status
|-
|GX Series
|GX-MIPI-IMX662
|Done
|}
====Supported Jetson Board====
{| class="wikitable"
!Jetson model
!status
|-
|Xavier NX
|Done
|-
|Orin NX
|Done
|-
|Orin Nano
|Done
|}

====Supported L4T versions====

*Jetpack5.1.4,L4T version r35.6
*Jetpack6.2.1,L4T version r36.4.4

=====How to check the current L4T version=====
On the Jetson board, check the current L4T version and try to replace it with the same version.

<code>cat /etc/nv_tegra_release</code>

As shown:

<code># R32 (release), REVISION: 7.1......</code>

This indicates that the current L4T version is 32.7.1, and the corresponding Jetpack version is Jetpack 32.6.1.
===Hardware preparation and installation===
The GX series cameras require a adapter board to be connected to the Jetson platform. The supported conditions are as shown in the table below:
{| class="wikitable"
!Camera model
!Jetson model
!FFC line
!Number of cameras
!Power supply
|-
| rowspan="2" |GX Series
|Xavier NX
|15‑pin to 22‑pin FFC (same-side contacts)
|2
|5V DC (required)
|-
|Orin Nano,Orin NX
|22‑pin FFC (opposite-side contacts)
|2
|5V DC (required)
|}
=====GX camera connected to Xavier NX=====
[[File:Gx to jetson xavier nx .png|center|thumb|800x800px|GX Camera to Xavier NX]]
<br />
=====The GX camera is connected to Orin Nano or Orin NX.=====
[[File:Gx to jetson orin.png|center|thumb|800x800px|GX camera to Orin NX/Nano]]
<br />
===Update the Jetson system===
This section describes how to update the L4T system of Jetson to support the GX camera module. For the method of operating system update, please refer to the update instructions for [https://wiki.veye.cc/index.php/How_to_upgrade_the_Jetson_system_to_support_VEYE_cameras/zh Jetson operating system]. 

Specifically, if trigger functionality is required, the Jetson kernel needs to be patched using '''veye_mv_l4t_<version>.patch'''.

This patch provides the following features:

#Adds support for monochrome camera data formats '''Y10''' and '''Y12'''.
#Adds support for '''trigger mode'''.

===Support for Trigger Mode ===
The default Jetson drivers support only video streaming mode. In the VI driver, data reception includes a timeout mechanism.

We have added a configurable option, '''vi_time_out_disable''', which allows this timeout mechanism to be dynamically enabled or disabled.

For practical usage, please refer to the example applications provided below.

===System Status Check ===
After completing the system update, reboot the Jetson board.

During the boot process, the Jetson system will detect any cameras connected to the I²C buses. If a camera is found, the corresponding '''/dev/videoX''' device node will be created.

To verify that the camera is correctly connected, run the following command on the Jetson board:

<code>dmesg | grep gxcam</code>

The output will display the camera model and version detected during Linux boot. 

For example, the following message indicates that a '''GX-MIPI-IMX662''' camera was detected on '''i2c-9''':

<code>gxcam 9-003b: camera is: GX-MIPI-IMX662</code>

===Status Check and Environment Variable Configuration===
[https://github.com/veyeimaging/nvidia_jetson_veye_bsp/tree/master/gx_i2c_tools Here], we provide two scripts that can automatically retrieve key information about the connected camera.

====probe_camera_info-jetson.sh ====
This script probes connected and successfully probed camera devices, retrieving information such as the video device node and I²C bus.

After execution, an '''auto_camera_index.json''' file will be generated in the current directory, containing the retrieved information.

Example:

<code>$ ./probe_camera_info-jetson.sh</code>

<code>cat auto_camera_index.json</code>

<code>[</code>

<code>{"i2c_bus": "9", "video_node": "/dev/video0"}</code>

<code>]</code>

Each <code>{}</code> block represents one camera. If the board supports multiple camera modules, multiple <code>{}</code> blocks will appear in the file.

The explanation of the camera information is as follows:
{| class="wikitable"
|+Camera Information
!Field
!Name
!Purpose
!Usage
|-
|video_node
|Video capture device node
|Standard V4L2 video device
|Used with <code>v4l2-ctl</code> or customer applications to capture images,and configuring certain camera parameters.
|-
|i2c_bus
|I²C bus
|Indicates the I²C bus to which the device is connected
|Used as the underlying communication channel for camera parameter configuration, e.g., by the <code>gx_mipi_i2c.sh</code> script
|}

*The video device nodes and I²C bus numbers used in the following sections can be replaced with the corresponding information obtained from the JSON file generated by the '''probe_camera_info-jetson.sh''' script.
*On the Jetson system, each camera module is mapped to a <code>/dev/videoX</code> device node.
*During system startup, cameras are probed in ascending order of their I²C bus numbers. The <code>X</code> in <code>/dev/videoX</code> is assigned sequentially starting from 0 according to the probing order.
**For example, if only one camera is connected, <code>X</code> will be 0 regardless of which I²C port the hardware is attached to.
**If five cameras are connected, <code>X</code> values will be <code>[0-4]</code>, ordered by ascending I²C bus numbers.
*In <code>v4l2-ctl</code> commands, the <code>-d /dev/videoX</code> option is used to access different cameras.

*In GStreamer, <code>v4l2src</code> can access different cameras by specifying <code>device=/dev/videoX</code>.

====gx_probe.sh====
By using the <code>gx_probe.sh</code> script, the I²C bus number corresponding to a certain camera, the camera model, width, height, frame rate, etc. can be configured into the environment variables. 

In that case, subsequent operations using v4l2-ctl will be more convenient.

The usage method is:

<code>$ source ./gx_probe.sh i2c_bus</code>

A typical output is as follows:

<code>$ source ./gx_probe.sh 9</code>

<code>Found veye_gxcam camera on i2c-9.</code>

<code>Setenv I2C_BUS = 9</code>

<code>Setenv CAMERAMODEL = GX-MIPI-IMX662</code>

<code>Setenv FPS = 60</code>

<code>Setenv WIDTH = 1920</code>

<code>Setenv HEIGHT = 1080</code>

You can verify the environment variable output using, for example:

<code>echo $CAMERAMODEL</code>

Note: These environment variables are valid only for the current shell session.

===Real-time preview===
====veye_viewer====
The <code>veye_viewer</code> is an open-source, Qt-based client application that allows users to easily evaluate cameras and configure parameters. Its operation logs, register listings, and open-source nature provide a convenient reference for users and support custom development.

The source code of <code>veye_viewer</code> can be downloaded [https://github.com/veyeimaging/veye_viewer here], or platform-specific executable programs are available directly in its [https://github.com/veyeimaging/veye_viewer/releases release packages].

====qv4l2====
First, install qv4l2

<code>sudo apt install qv4l2</code>

1. Enter <code>qv4l2</code> in the command line to start the V4L2 player. 

2. Click the ▶ (play) button to bring up the opened media window.
<br />
[[File:Play gx camera using qv4l2.png|center|thumb|1050x1050px|Play gx camera using qv4l2]]

===Gstreamer Usage===
To install the latest accelerated version of gstreamer plugins and applications, please run the following command:

<code>sudo apt-get update</code>

<code>sudo apt-get install nvidia-l4t-gstreamer</code>

<code>sudo ldconfig</code>

<code>rm -rf .cache/gstreamer-1.0/</code>

<code>export DISPLAY=:0</code>

*video preview 1080p HD

<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false</code>

<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nv3dsink sync=false</code>

*video preview 1080p HD(using xvimagesink sink if supported)

<code>export DISPLAY=:0</code>

<code>gst-launch-1.0 -e v4l2src io-mode=4 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY' ! xvimagesink sync=false</code>

*Gstreamer is embedded into OpenCV

I think maybe OpenCV do not support I420 data format input，so you should convert it to BGR format.

<code>gst-launch-1.0 nvv4l2camerasrc ! video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1920, height=(int)1080 ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)I420 !  nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink</code>

*Video recording 1080p HD

<code>gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=1 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e</code>

*Playback of saved video file

<code>gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nv3dsink -e</code>

*Capturing FHD still image  

<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jpg</code>
===yavta(Only supports streaming mode)===

====yavta install====
<code>git clone <nowiki>https://github.com/veyeimaging/yavta.git</nowiki></code>

<code>cd yavta;make</code>

====Set image format====
<code>v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY</code>
====Save the image to a file====

*UYVY type

<code>./yavta -c1 -F"uyvy-${WIDTH}x${HEIGHT}.yuv" --skip 0 -f UYVY -s "${WIDTH}x${HEIGHT}" /dev/video0</code>

*GX-MIPI-IMX662

<code>./yavta -c1 -Fuyvy-1920x1080.yuv --skip 0 -f UYVY -s 1920x1080 /dev/video0</code>
===Jetson multimedia-api samples===
On the Jetson platform, the [https://docs.nvidia.com/jetson/l4t-multimedia/index.html Jetson Linux API] is provided for developers. For installation of the '''jetson_multimedia_api''' package, please refer to the official NVIDIA documentation.

For GX series cameras, ISP functionality is already integrated within the camera module. Therefore, '''libargus''' cannot be used to acquire data. Instead, standard V4L2 interfaces can be used to capture data directly from the <code>/dev/videoX</code> device file.

After installing the Jetson system, you can navigate to the Multimedia API sample directory, compile, and run the following commands to preview the camera video:

<code>cd /usr/src/jetson_multimedia_api/samples/12_v4l2_camera_cuda</code>

<code>make</code>

<code>./v4l2_camera_cuda -d /dev/video0 -s 1920x1080 -f UYVY</code>

This command uses the <code>/dev/video0</code> device to perform real-time video preview at a resolution of 1920×1080 with the UYVY format.

===v4l2-ctl Examples===
=====Install v4l2-utils=====
<code>sudo apt-get install v4l-utils</code>
=====Configure parameters using v4l2-ctl=====

======List the data formats supported by the camera======
<code>v4l2-ctl --list-formats-ext</code>

Below is an example:

<code>ioctl: VIDIOC_ENUM_FMT</code>

<code>        Type: Video Capture</code>

<code>        [0]: 'UYVY' (UYVY 4:2:2)</code>

<code>                Size: Discrete 1920x1080</code>

<code>                        Interval: Discrete 0.017s (60.000 fps)</code>

<code>        [1]: 'YUYV' (YUYV 4:2:2)</code>

<code>        [2]: 'NV16' (Y/CbCr 4:2:2)</code>

<code>                Size: Discrete 1920x1080</code>

<code>                        Interval: Discrete 0.017s (60.000 fps)</code>

*Frame rate statistics

<code>v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to=/dev/null</code>

*Save the image to a file

<code>v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv</code>

======List the configurable parameters of the camera implemented in the driver======
<code>v4l2-ctl -L</code>

<code>User Controls</code>

<code>                   work_mode 0x00981a01 (int)    : min=0 max=4 step=1 default=0 value=0 flags=volatile, execute-on-write</code>

<code>                    trigger_src 0x00981a02 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write</code>

<code>                    soft_trgone 0x00981a03 (button) : flags=write-only, execute-on-write</code>

<code>                      sync_role 0x00981a04 (int)    : min=0 max=1 step=1 default=0 value=0 flags=volatile, execute-on-write</code>

<code>                     frame_rate 0x00981a05 (int)    : min=0 max=60 step=1 default=60 value=60 flags=volatile, execute-on-write</code>

The parameters can be set and retrieved using the following methods.

<code>v4l2-ctl --set-ctrl [ctrl_type]=[val]</code>

<code>v4l2-ctl --get-ctrl [ctrl_type]</code>

All of the above functions can be achieved by using the [https://wiki.veye.cc/index.php/Gx_mipi_i2c.sh_user_guide gx_mipi_i2c.sh] script.

It should be noted that the above parameters cannot be modified once the image capture process begins. The following will provide detailed explanations for each one:
======Configure trigger mode======
<code>v4l2-ctl --set-ctrl <small>work_mode=[0-2]</small></code>

0: Streaming Mode 

1: Trigger Mode 

2: Multi-camera Synchronization Mode

======Configure trigger source======
<code>v4l2-ctl --set-ctrl <small>trigger_src=[0-1]</small></code>

0: Soft Trigger 

1: Hard Trigger

======Soft trigger once======
<code>v4l2-ctl --set-ctrl <small>soft_trgone=1</small></code>
======Set frame rate======
<code>v4l2-ctl --set-ctrl frame_rate=[1-max]</code>

As the resolution is adjusted, the maximum frame rate will be automatically updated.
====Stream mode====

=====Set image format=====
Take the largest screen as an example:

<code>v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY</code>

<code>v4l2-ctl --set-ctrl frame_rate=$FPS</code>
=====Frame rate statistics=====
<code>v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to=/dev/null</code>
=====Save the image to a file=====

======under normal conditions======

*UYVY

<code>v4l2-ctl -d /dev/video0 --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to="uyvy-${WIDTH}x${HEIGHT}.yuv"</code>
======Use OpenCV to preview camera======
<code>sudo apt install python3-opencv</code>

*We have provided a simple [https://github.com/veyeimaging/nvidia_jetson_veye_bsp/tree/master/samples/opencv/yuv_camera sample] to demonstrate how to achieve this functionality:

<code>python3 ./v4l2_opencv_show2.py --width 1920 --height 1080 --fps 60</code>

====Trigger mode====
=====preparatory work=====
<code>v4l2-ctl --set-ctrl low_latency_mode=1</code>

<code>v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY</code>

<code>v4l2-ctl --set-ctrl frame_rate=$FPS</code>

=====Soft trigger=====

======Setup Mode======
<code>v4l2-ctl --set-ctrl <small>work_mode=1</small></code>

<code>v4l2-ctl --set-ctrl <small>trigger_src=0</small></code>

<code>v4l2-ctl --set-ctrl vi_time_out_disable=1</code>
======Start taking pictures======

<code>v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to="uyvy-${WIDTH}x${HEIGHT}.yuv"</code>

*GX-MIPI-IMX662

<code>v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to=uyvy-1920x1080.yuv</code>

======Perform soft triggering operation======
In other shell terminals, you can execute the following command multiple times to trigger it multiple times.

<code>v4l2-ctl --set-ctrl <small>soft_trgone=1</small></code>

======Stop triggering======
Due to the fact that in the kernel driver, a dead loop is used to wait for new images, it is necessary to first disable vi_time_out_disable, and then exit the acquisition operation.

<code>v4l2-ctl --set-ctrl vi_time_out_disable=0</code>

In the terminal of the image acquisition command, press Ctrl+C to exit the acquisition operation.

=====Hardware trigger=====
To perform the triggering operation using [https://github.com/NVIDIA/jetson-gpio jetson-gpio], first, install and configure jetson-gpio properly. 

Next, using Jetson GPIO40 (Board number) as the simulated trigger source and with the rising edge as the trigger condition, an example will be given. 

The rich trigger parameter settings can be carried out by using the [https://wiki.veye.cc/index.php/Gx_mipi_i2c.sh_user_guide gx_mipi_i2c.sh] script.

======hardware connection======
TBD

======Setup Mode======
<code>v4l2-ctl --set-ctrl <small>work_mode=1</small></code>

<code>v4l2-ctl --set-ctrl <small>trigger_src=1</small></code>

<code>v4l2-ctl --set-ctrl vi_time_out_disable=1</code>
======Start taking pictures======

<code>v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=UYVY --stream-mmap --stream-count=-1 --stream-to="uyvy-${WIDTH}x${HEIGHT}.yuv"</code>
======Perform a hard trigger operation======
<code>python gpio_trigger_jetson.py</code>

Note: Trigger script [https://github.com/veyeimaging/nvidia_jetson_veye_bsp/tree/master/gx_i2c_tools link].
======Stop triggering and collecting======
Due to the fact that in the kernel driver, a dead loop is used to wait for new images, it is necessary to first disable vi_time_out_disable, and then exit the acquisition operation.

<code>v4l2-ctl --set-ctrl vi_time_out_disable=0</code>

In the terminal of the image acquisition command, press Ctrl+C to exit the acquisition operation.
====Synchronous mode====
=====Set the synchronization mode=====
<code>$ v4l2-ctl -d /dev/video0 --set-ctrl work_mode=4</code>

<code>$ v4l2-ctl -d /dev/video1 --set-ctrl work_mode=4</code>

=====Set up the master and slave cameras=====
Main camera：

<code>$ v4l2-ctl -d /dev/video0 --set-ctrl <small>sync_role=0</small></code>

slave camera：

<code>$ v4l2-ctl -d /dev/video1 --set-ctrl <small>sync_role=1</small></code>

=====Start taking pictures=====
The method of taking screenshots in the synchronization mode is exactly the same as that in the video stream mode.
===I2C Script Usage Instructions===
We provided shell scripts to configure the parameters.

[https://wiki.veye.cc/index.php/Gx_mipi_i2c.sh_user_guide gx_mipi_i2c.sh user guide]


'''Document Revision History'''

*'''2025-12-20'''  Initial release.