Difference between revisions of "VEYE CS Camera for Jetson TX2"

From wiki_veye
Jump to navigation Jump to search
 
(80 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[VEYE CS Camera for Jetson TX2/zh|查看中文]]
 
[[VEYE CS Camera for Jetson TX2/zh|查看中文]]
  
−
'''<big>How to use VEYE and CS series camera module on NVIDIA Jetson Nano,TX2,AGX Xavier and Xavier NX</big>'''
+
'''<big>How to use VEYE and CS series camera module on NVIDIA Jetson Nano,TX2,Xavier and Orin</big>'''
  
 
===Overview===
 
===Overview===
−
This guide shows how to use VEYE and CS series camera modules on Jetson Nano,Jetson Nano 2GB,TX2,TX2 NX,AGX Xavier and Xavier NX.  Jetson SDK Version is
+
This guide shows how to use VEYE and CS series camera modules on Jetson boards.  Jetson SDK Version is：
  
 
*Jetpack4.2.2,L4T r32.2.1
 
*Jetpack4.2.2,L4T r32.2.1
Line 13: Line 13:
 
*Jetpack4.5.1,L4T r32.5.1
 
*Jetpack4.5.1,L4T r32.5.1
 
*Jetpack4.6,L4T r32.6.1
 
*Jetpack4.6,L4T r32.6.1
 +
 +
* Jetpack4.6.1, L4T r32.7.1
 +
*Jetpack4.6.2,L4T r32.7.2
 +
*Jetpack4.6.3,L4T r32.7.3
 +
*Jetpack4.6.4,L4T r32.7.4
 +
*Jetpack5.0.1DP, L4T r34.1.1
 +
*Jetpack5.0.2, L4T r35.1
 +
*Jetpack5.1,L4T r35.2.1
 +
*Jetpack5.1.1,L4T r35.3.1
 +
*Jetpack5.1.2,L4T r35.4.1
 +
*Jetpack5.1.3,L4T r35.5
 +
*Jetpack6.0,L4T r36.3
 +
*Jetpack6.1,L4T r36.4
  
 
We provide two ways to do so: '''Prebuilt Binaries''' or '''Source Code. Yes, It's Open Source!'''
 
We provide two ways to do so: '''Prebuilt Binaries''' or '''Source Code. Yes, It's Open Source!'''
  
 
VEYE and CS series camera modules are camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide '''V4L2 interface''' for video streaming apps , and '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly, which is called DRA(Directly Register Access).
 
VEYE and CS series camera modules are camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide '''V4L2 interface''' for video streaming apps , and '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly, which is called DRA(Directly Register Access).
 +
====Camera module list====
 +
{| class="wikitable"
 +
|+Camera module dirver status list
 +
!Series
 +
!Model
 +
!Status
 +
|-
 +
|VEYE series
 +
| VEYE-MIPI-IMX327S
 +
| Done
 +
|-
 +
|VEYE series
 +
| VEYE-MIPI-IMX385
 +
|Done
 +
|-
 +
|VEYE series
 +
|VEYE-MIPI-IMX462
 +
|Done
 +
|-
 +
|VEYE series
 +
|VEYE-MIPI-IMX335
 +
|Done
 +
|-
 +
|CS series
 +
| CS-MIPI-IMX307
 +
|Done
 +
|-
 +
|CS series
 +
|CS-MIPI-SC132
 +
|Done
 +
|-
 +
|TX2-XAVIER-nCAM Series
 +
|CS-TX2-XAVIER-'''''n'''''CAM
 +
|Done
 +
|}
 
====How to check the current L4T version====
 
====How to check the current L4T version====
 +
 +
===== Method 1 =====
 
On Jetson board
 
On Jetson board
  
Line 26: Line 76:
 
<code># R32 (release), REVISION: 4.3......</code>
 
<code># R32 (release), REVISION: 4.3......</code>
  
−
It means L4t Verion is 32.4.3
+
It means L4t Version is 32.4.3
 +
 
 +
===== Method 2 =====
 +
Refer to this [https://pypi.org/project/jetson-stats/ link] to install jetson-stats:
 +
 
 +
<code>jtop</code>
 +
 
 
===Hardware Setup===
 
===Hardware Setup===
−
We have designed a 6cam interposer board  for TX2 devkit and Xavier, It support up to 6 cameras at the same time.   
+
We have designed a 6cam interposer board  for TX2 devkit,Xavier and Orin, It support up to 6 cameras at the same time.   
 
[[File:ADP-N1-6CAM YT0.95-4I.jpg|alt=6cam interposer board connected with 6 cameras|center|thumb|800x800px|6cam interposer board connected with 6 cameras]]
 
[[File:ADP-N1-6CAM YT0.95-4I.jpg|alt=6cam interposer board connected with 6 cameras|center|thumb|800x800px|6cam interposer board connected with 6 cameras]]
−
<br />
+
 
 +
 
 +
In particular, the VEYE-MIPI-IMX335 must use a 12V DC terminal to provide auxiliary power due to its high power consumption. Refer to [[ADP-N1-V2.0 Adapter Board Data Sheet#Adapter Board Pinlist.EF.BC.88.E6.8E.A5.E5.8F.A3.E5.92.8C.E5.BC.95.E8.84.9A.E8.AF.B4.E6.98.8E.EF.BC.89|J11 of ADP-N1]].
 
====NVIDIA TX2 Developer Kit====
 
====NVIDIA TX2 Developer Kit====
 
It is connected as shown in figure:
 
It is connected as shown in figure:
Line 36: Line 94:
 
<br />'''Be sure to screw the ADP adapter board to the bottom board.'''
 
<br />'''Be sure to screw the ADP adapter board to the bottom board.'''
  
−
====NVIDIA TX2 AGX Xavier====
+
====NVIDIA AGX Xavier(the same with Orin)====
 
<br />
 
<br />
 
[[File:ADP-N1-V2.0 XAVIER 6CAM.jpg|link=http://wiki.veye.cc/index.php/File:ADP-N1-V2.0%20XAVIER%206CAM.jpg|alt=Connect VEYE cameras to AGX Xavier|center|thumb|800x800px|Connect VEYE cameras to AGX Xavier]]
 
[[File:ADP-N1-V2.0 XAVIER 6CAM.jpg|link=http://wiki.veye.cc/index.php/File:ADP-N1-V2.0%20XAVIER%206CAM.jpg|alt=Connect VEYE cameras to AGX Xavier|center|thumb|800x800px|Connect VEYE cameras to AGX Xavier]]
−
<br />'''Be sure to screw the ADP adapter board to the bottom board.'''
+
<br />'''Be sure to screw the ADP adapter board to the bottom board to ensure that the adapter plate is installed flat.'''
 
====NVIDIA Nano ，TX2 NX and Xavier NX Devkit====
 
====NVIDIA Nano ，TX2 NX and Xavier NX Devkit====
 
The Xavier NX interface is compatible with raspberry pi and Jetson Nano. The connection mode is shown in figure:
 
The Xavier NX interface is compatible with raspberry pi and Jetson Nano. The connection mode is shown in figure:
Line 46: Line 104:
 
[[File:Xavier NX connection with CS-MIPI-IMX307.jpg|alt=Xavier NX connection with CS-MIPI-IMX307|center|thumb|800x800px|Xavier NX connection with CS-MIPI-IMX307]]
 
[[File:Xavier NX connection with CS-MIPI-IMX307.jpg|alt=Xavier NX connection with CS-MIPI-IMX307|center|thumb|800x800px|Xavier NX connection with CS-MIPI-IMX307]]
 
[[File:Jetson nano to Veye327.jpg|link=http://wiki.veye.cc/index.php/File:Jetson%20nano%20to%20Veye327.jpg|alt=Jetson Nano A02 与VEYE模组连接|center|thumb|800x800px|VEYE327 connection with Jetson Nano A02]]
 
[[File:Jetson nano to Veye327.jpg|link=http://wiki.veye.cc/index.php/File:Jetson%20nano%20to%20Veye327.jpg|alt=Jetson Nano A02 与VEYE模组连接|center|thumb|800x800px|VEYE327 connection with Jetson Nano A02]]
 +
[[File:VEYE camera connection with Orin Nano and Orin NX.jpg|center|thumb|800x800px|VEYE camera  connection with Orin Nano and Orin NX]]
  
−
===Upgrade Jetson Nano, Jetson TX2, TX2 NX,AGX Xavier and Xavier NX system===
+
=== Upgrade Jetson system ===
 +
This section describes how to upgrade the Jetson system to support MV camera module. For OS update method, please refer to [[How to upgrade the Jetson system to support VEYE cameras|Update Jetson OS]].
 +
===Applications and Test===
 +
====Check system status====
 +
Run the following command to confirm whether the camera is probed.
  
−
====Overview====
+
*VEYE-MIPI-IMX327S/VEYE-MIPI-IMX462/VEYE-MIPI-IMX385(using VEYE-MIPI-CAM2M dtb)
−
This section describes how to upgrade the Jetson system to support our camera module. To support our camera module, we need to update the two parts of the L4T (Linux for Tegra) of the Jetson system, '''Image''' and '''DTB'''.
 
  
−
In the Image, we added the camera driver, while the DTB indicates the camera model used. In general, you only need to use the Image and DTB, that we have prebuilt, and you don't need to build from source  when it is not necessary.
+
<code>dmesg | grep veye</code>  
  
−
<s>In particular, for Xavier NX, we provide a burning image of the TF card.</s>
+
The output message appears as shown below：
  
−
====<s>Burning the prebuilt Image to upgrade Jetson Nano and Xavier NX Devkit</s>====
+
<code>camera id is veyecam</code>
−
'''''<s>This method is no longer recommended because in this mode, the DTB partition reverts to the default value in some cases. It is recommended to use pre-compiled Image and DTB for driver upgrade.</s>'''''
 
  
−
<s>You can use the image prebuild by us to upgrade jetson Xavier NX system, which will support our camera.</s>
+
<code>sensor is IMX327/IMX462/IMX385</code>
  
−
<s>Using whole prebuilt image</s>
+
<code>subdev veyecam [i2c_bus]-003b bound</code>
  
−
*<s>download ：</s>
+
*For CS-MIPI-IMX307
  
−
<s>baiduyun：https://pan.baidu.com/s/1sWrcfEOivjuQ7T7P5bVcng</s>
+
<code>dmesg | grep csx307</code>  
  
−
<s>google driver:https://drive.google.com/drive/folders/1KHSMTdyw-ZczR1yJPj-5y9eIv7izbgt3?usp=sharing</s>
+
The output message appears as shown below.：
  
−
<s>If Baidu cloud disk download is too slow, you can contact our company customer service ''xumm@csoneplus.com'' to obtain.</s>
+
<code>Detected CS307 sensor</code>
  
−
*<s>How to burn</s><code><nowiki/></code>
+
<code>subdev csx307 [i2c_bus]-003b bound</code>
  
−
<s>refer to [https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit#write Official document]</s>
+
*For CS-MIPI-SC132
  
−
====BSP package introduction====
+
<code>dmesg | grep cssc132</code>  
  
−
=====Path=====
+
The output message appears as shown below.：
−
[https://github.com/veyeimaging/nvidia_jetson_veye_bsp/tree/master/kernel_image https://github.com/veyeimaging/nvidia_jetson_veye_bsp]
 
−
=====bsp package includes：=====
 
  
−
*prebuild linux kernel：Image
+
<code>subdev cssc132 [i2c_bus]-003b bound</code>
−
*prebuild dtb for different boards
 
−
*driver source code
 
−
*dts source code
 
−
*i2c toolkits
 
  
−
==== DTB file name and Jetson Board ====
+
*For VEYE-MIPI-IMX335
  
−
=====Nano A02=====
+
<code>dmesg | grep imx335</code>  
−
<code>tegra210-p3448-0000-p3449-0000-a02.dtb</code>
 
  
−
===== Nano B01 =====
+
The output message appears as shown below.：
−
<code>tegra210-p3448-0000-p3449-0000-b00.dtb</code>
 
  
−
===== Nano 2GB =====
+
<code>camera id is VEYE-MIPI-IMX335</code>
−
<code>tegra210-p3448-0003-p3542-0000.dtb</code>
 
−
=====TX2 Devkit=====
 
−
<code>tegra186-quill-p3310-1000-c03-00-base.dtb</code>
 
  
−
===== TX2 NX(p3509-0000 carrier board) =====
+
<code>subdev veye_imx335 [i2c_bus]-003b bound</code>
−
use the same carrier board with XAVIER NX Devkit
 
  
−
<code>tegra186-p3636-0001-p3509-0000-a01.dtb</code>
+
*Run the following command to check the presence of video node.
−
=====AGX XAVIER=====
 
−
<code>tegra194-p2888-0001-p2822-0000.dtb</code>
 
−
=====XAVIER NX Devkit=====
 
−
<code>tegra194-p3668-all-p3509-0000.dtb</code>
 
  
−
==== DTB file name and camera module correspondence ====
+
<code>ls /dev/video*</code>
−
Under the dtb directory of each motherboard, the directory corresponding to the camera model is stored, and you can usually know the corresponding camera model directly by the directory name.
 
  
−
Specially:
+
The output message appears as shown below.
  
−
1, VEYE-MIPI-CAM2M is the new version of VEYE series dtb, which supports all VEYE series products, including VEYE-MIPI-327E, VEYE-MIPI-IMX327S, VEYE-MIPI-IMX462, VEYE-MIPI-IMX385.
+
<code>videoX</code>  
  
−
2, The ones with fpdlink as suffix correspond to fpdlink coaxial model module.
+
*For VEYE-MIPI-327(using VEYE-MIPI-327 dtb)
  
−
====Upgrade Jetson Nano and Xavier NX 、AGX Xavier without HOST PC====
+
<code>dmesg | grep veye327</code>  
−
Note: AGX Xavie Jetpack version >= 4.4 , support this upgrade mode. Earlier versions have not been verified and are not recommended.
 
  
−
=====Check the current L4T version=====
+
The output message appears as shown below：
−
First check the current L4T version and try to replace it with the same version.
 
  
−
<code>cat /etc/nv_tegra_release</code>
+
<code>Detected VEYE327 sensor</code>
  
−
If it shows:
+
<code>subdev veye327 [i2c_bus]-003b bound</code>
  
−
<code># R32 (release), REVISION: 4.3......</code>
+
The [i2c_bus] in the driver prompt message indicates the i2c bus corresponding to this camera.
  
−
It means L4t Verion is 32.4.3
+
====Video Stream Toolkits Manual====
 +
The camera appears as /dev/videoX device node,where X is between [0-5].
  
−
=====Download the bsp package to the Jetson board=====
+
During the Linux boot process, the cameras are detected in the order of i2c bus from smallest to largest.  For example, if only one camera is connected, X is 0 regardless of the hardware connection to the location. If 5 cameras are connected, X is reflected as [0-4] according to i2c bus from smallest to largest.
−
On Jetson Nano or Xavier NX:
 
  
−
<code>git clone <nowiki>https://github.com/veyeimaging/nvidia_jetson_veye_bsp.git</nowiki></code>
+
In gstreamer command ，v4l2src and nvv4l2camerasrc has a param device=/dev/videoX to identify which camera to use.
 +
=====Gstreamer Usage=====
 +
To install the latest accelerated gstreamer plugins and applications, run the following commands:
  
−
=====Upgrade Image=====
+
<code>sudo apt-get update</code>
−
First, under the bsp package directory, find the corresponding version of the Image package, decompress it, and execute:
 
  
−
<code>sudo cp <path to your Image dir>/Image /boot/Image -f</code>
+
<code>sudo apt-get install nvidia-l4t-gstreamer</code>
  
−
''For Xavier NX Devkit''
+
<code>sudo ldconfig</code>
  
−
Need to use the signed file(For example，file inside Image_l4t_r32.*.*_veyecam_signed_4_xavier),copy Image and Image.sig to /boot/ dir on the Jetson Xavier board:
+
<code>rm -rf .cache/gstreamer-1.0/</code>
  
−
<code>sudo cp <path to your Image dir>/Image /boot/Image -f</code>
 
  
−
<code>sudo cp <path to your Image dir>/Image.sig /boot/Image.sig -f</code>
+
<code>export DISPLAY=:0</code>
  
−
Effective after restart.
+
*Preview FHD
  
−
=====Upgrade dtb=====
+
<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false</code>
−
Under the bsp package directory, find the dtb file corresponding to your camera module model and L4T version number.
 
  
−
<code>sudo cp <path to your dtb dir>/<DTB file name> /boot/ -f</code>
+
<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nv3dsink sync=false</code>
  
−
Then, edit the ''/ boot/extlinux/extlinux.conf''  file and add the following line.
+
* Preview FHD(using xvimagesink sink if supported)
  
−
<code>FDT /boot/<DTB file name></code>
+
<code>export DISPLAY=:0</code>
  
−
Note: <DTB file name> should be replaced with the corresponding dtb name of the mainboard, ensure that the file specified in the FDT line actually exists correctly.
+
<code>gst-launch-1.0 -e v4l2src io-mode=4 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY' ! xvimagesink sync=false</code>
  
−
Effective after restart.
+
*Preview 720p@60 (CS-MIPI-IMX307)
  
−
====Upgrade Jetson Board using Host PC====
+
<code>gst-launch-1.0 nvv4l2camerasrc  ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false</code>
  
−
=====Burn the standard Image=====
+
*Preview 1280*1080@45 (CS-MIPI-SC132)
−
'''''Note: if your Jetson board has correctly burned the standard version Image, you can skip this step.'''''
 
  
−
You can burn the standard Image using SDK Manager or following the steps below.
+
<code>gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1280, height=(int)1080, framerate=(fraction)45/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false</code>
  
−
* Set PATH
+
*Preview 640*480p@130 (CS-MIPI-IMX307)
  
−
Assume SDK install directory is <TOPDIR>.
+
<code>gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)130/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false</code>
  
−
<code>export TOP_DIR=<absolute path to top dir accroding to your jetpack version and board name;for example /home/xumm/nvidia/nvidia_sdk/JetPack_4.5_Linux_JETSON_XAVIER_NX_DEVKIT/></code>
+
*Preview 2 cameras  1080p HD
  
−
export L4T_DIR=$TOP_DIR/Linux_for_Tegra
+
<code>WIDTH=960</code>
  
−
*Build default image, flashing to Jetson Board, prepare L4T environment
+
<code>HEIGHT=540</code>
  
−
<code>cd $L4T_DIR</code>
+
<code>CAPS="video/x-raw(memory:NVMM),format=(string)UYVY, width=1920, height=1080"</code>
  
−
<code>sudo ./apply_binaries.sh</code>  
+
<code>gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=$WIDTH sink_0::height=$HEIGHT sink_1::xpos=$WIDTH sink_1::ypos=0 sink_1::width=$WIDTH sink_1::height=$HEIGHT ! nv3dsink nvv4l2camerasrc device=/dev/video0 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp. nvv4l2camerasrc  device=/dev/video1 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp.</code>
  
−
- Nano
+
*Record FHD in H.264 format to a video file
  
−
<code><s>sudo ./flash.sh jetson-qspi-sd mmcblk0p1</s></code>
+
<code>gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=1 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e</code>
  
−
<code>sudo ./flash.sh jetson-nano-devkit mmcblk0p1</code>
+
*Playback of saved video file
  
−
<code>sudo ./flash.sh jetson-nano-2gb-devkit mmcblk0p1</code>
+
<code>gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nvoverlaysink -e</code>
  
−
''- TX2''
+
*Capturing FHD still image  
  
−
<code>sudo ./flash.sh jetson-tx2 mmcblk0p1</code>
+
<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jp</code>
  
−
''- TX2 NX(p3509-0000 carrier board)''
+
===== VEYE-MIPI-IMX335 gstreamer example =====
 +
Since VEYE-MIPI-IMX335 supports only a few specific resolution modes, its driver we use use_sensor_mode_id mode.
 +
{| class="wikitable"
 +
|+VEYE-MIPI-IMX335 mode
 +
!./veye5_mipi_i2c.sh video mode
 +
!v4l2-ctl sensor_mode
 +
!video format
 +
|-
 +
|1
 +
|0
 +
|2592x1944@20fps
 +
|-
 +
|2
 +
|1
 +
|2592x1944@12.5fps
 +
|-
 +
|3
 +
|2
 +
|2560x1440@25fps
 +
|-
 +
|4
 +
|3
 +
|2560x1440@30fps
 +
|}
  
−
<code>sudo ./flash.sh jetson-xavier-nx-devkit-tx2-nx mmcblk0p1</code>
+
* prepare
  
−
''- AGX Xavier''
+
<code>sudo apt-get install v4l-utils</code>
  
−
<code>sudo ./flash.sh jetson-xavier mmcblk0p1</code>
+
Refer to the following section and download the veye5_mipi_i2c.sh tool.
  
−
''- Xavier NX Devkit''<code><nowiki/></code>
+
* 2592x1944@20fps mode preview, framerate statistics
  
−
<code>sudo ./flash.sh jetson-xavier-nx-devkit mmcblk0p1</code>
+
<code>v4l2-ctl -d /dev/video0 --set-ctrl sensor_mode=0</code>
−
=====Download our bsp package for Jetson=====
 
−
<code>cd $L4T_DIR</code>
 
  
−
<code>git clone https://github.com/veyeimaging/nvidia_jetson_veye_bsp<nowiki/>.git</code><code><nowiki/></code>
+
<code>./veye5_mipi_i2c.sh -w -f videomode -p1 1 -b [busnum]</code>
  
−
<code>export RELEASE_PACK_DIR=$L4T_DIR/nvidia_jetson_veye_bsp</code>
+
<code>v4l2-ctl -d /dev/video0 --set-ctrl preferred_stride=5376</code>
−
=====Upgrade Image and DTB=====
 
−
prebuilt Image on github:https://github.com/veyeimaging/nvidia_jetson_veye_bsp/tree/master/kernel_image
 
  
−
*Installing the Kernel
+
Preview
  
−
Whether you use a prebuild kernel or you build it yourself
+
<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM), width=(int)2592, height=(int)1944" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false</code>
  
−
''- Nano and TX2''
+
Framerate statistics
  
−
copy Image to /boot/ dir on the Jetson Nano or TX2 board:
+
<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY,width=(int)2592,height=(int)1944" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! fpsdisplaysink video-sink=fakesink -v</code>
  
−
<code>sudo cp <path to your Image dir>/Image  /boot/Image -f</code>
+
* 2560x1440@30fps mode video encode and capture
  
−
''- AGX Xavier and Xavier NX Devkit''
+
<code>v4l2-ctl -d /dev/video0 --set-ctrl sensor_mode=3</code>
  
−
Need to use the signed file(For example，file inside Image_l4t_r32.*.*_veyecam_signed_4_xavier),copy Image and Image.sig to /boot/ dir on the Jetson Xavier board:
+
<code>./veye5_mipi_i2c.sh -w -f videomode -p1 4 -b [busnum]</code>
  
−
<code>sudo cp <path to your Image dir>/Image  /boot/Image -f</code>
+
<code>v4l2-ctl -d /dev/video0 --set-ctrl preferred_stride=5120</code>
  
−
<code>sudo cp <path to your Image dir>/Image.sig  /boot/Image -f</code>
+
Video encode
  
−
<s>on the Host PC:</s>
+
<code>gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)2560, height=(int)1440" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=0 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e</code>
  
−
<code><s>cd $L4T_DIR</s></code>
+
Video Replay
  
−
<code><s>sudo ./flash.sh -k kernel jetson-xavier mmcblk0p1</s></code>
+
<code>gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nv3dsink -e</code>
  
−
''<s>- Xavier NX</s>''
+
Capture
  
−
<s>on the Host PC：</s>
+
<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)2560, height=(int)1440" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jpg</code>
  
−
<code><s>cd $L4T_DIR</s></code>
+
===== Jetson multimedia-api samples =====
 +
The [https://docs.nvidia.com/jetson/l4t-multimedia/index.html Jetson Linux API] is available on the Jetson platform for developers to use. Please refer to the official nVidia documentation for the installation of the jetson_multimedia_api package.
  
−
<code><s>sudo ./flash.sh -k kernel jetson-xavier-nx-devkit mmcblk0p1</s></code>
+
For VEYE and CS series cameras, the ISP function is already integrated in the camera, so you cannot use libargus to get data, but can directly use the standard V4L2 interface to get data from the /dev/videoX device file.
  
−
*Flashing dtb file
+
The following two samples can be run directly.
  
−
Enter the recovery mode，and on the Host PC:
+
====== 12_camera_v4l2_cuda ======
 +
<code>./camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UYVY</code>
  
−
<code>cp <path to your dtb dir>/dtbfilename $L4T_DIR/kernel/dtb/ -f</code>
+
====== v4l2cuda ======
 +
<code>./capture-cuda -d /dev/video0 -f UYVY -m -o out.ppm -s 1920x1080 -c 1</code>
  
−
<code>cd $L4T_DIR</code>
+
===== v4l2-ctl Application Examples =====
  
−
- ''Nano''
+
======install v4l2-utils======
 +
<code>sudo apt-get install v4l-utils</code>
 +
======Configure parameters using v4l2-ctl======
  
−
<code><s>sudo ./flash.sh -r -k DTB jetson-nano-qspi-sd mmcblk0p1</s></code>
+
* List the data formats supported by the camera
  
−
<code>sudo ./flash.sh -r -k DTB  jetson-nano-devkit mmcblk0p1</code>
+
<code>v4l2-ctl -d /dev/video0 --list-formats-ext</code>
  
−
<code>sudo ./flash.sh -r -k DTB  jetson-nano-2gb-devkit mmcblk0p1</code>
+
* Frame rate statistics
  
−
''- TX2''
+
<code>v4l2-ctl -d /dev/video0  --set-fmt-video=width=1920,height=1080,pixelformat=UYVY--stream-mmap --stream-count=-1 --stream-to=/dev/null</code>
  
−
<code>sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1</code>
+
* Save image to file
  
−
''- TX2 NX(p3509-0000 carrier board)''
+
<code>v4l2-ctl -d /dev/video0  --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv</code>
  
−
<code>sudo ./flash.sh -r -k kernel-dtb jetson-xavier-nx-devkit-tx2-nx mmcblk0p1</code>
+
===== yavta =====
  
−
''- AGX Xavier''
+
======install yavta======
 +
<code>git clone <nowiki>https://github.com/veyeimaging/yavta.git</nowiki></code>
  
−
<code>sudo ./flash.sh -r -k kernel-dtb jetson-xavier mmcblk0p1</code>
+
<code>cd yavta;make</code>
  
−
''- Xavier NX''
+
====== Save image to file ======
 +
<code>./yavta -c1 -FUYVY-1920x1080.yuv --skip 0 -f UYVY -s 1920x1080 /dev/video0</code>
  
−
<code>sudo ./flash.sh -r -k kernel-dtb jetson-xavier-nx-devkit mmcblk0p1</code>
+
===== Opencv =====
 +
To import camera data from v4l2 devices to opencv, we provide several samples.
  
−
Reset Jetson Board after flashing.
+
We provide 2 samples [https://github.com/veyeimaging/nvidia_jetson_veye_bsp/tree/master/samples/opencv here].
  
−
====Are Image and DTB upgrade successful ?====
+
====Using i2c script to modify camera parameters====
  
−
=====For Image=====
+
*VEYE-MIPI-CAM2M Series（VEYE-MIPI-327、VEYE-MIPI-IMX327S、VEYE-MIPI-IMX462、VEYE-MIPI-IMX385）
−
<code>ls /sys/bus/i2c/drivers/</code>
 
  
−
There should have  veye327 veyecam csx307 cssc132 directory.
+
Video Control Toolkits Manual ：[[VEYE-MIPI-290/327 i2c/|VEYE-MIPI-327 I2C]]
−
=====For DTB=====
 
−
DTB varies from platform  and from camera module.
 
  
−
- Nano A02/ Nano 2G
+
*CS-MIPI-IMX307和CS-MIPI-SC132
  
−
<code>ls /proc/device-tree/host1x/i2c@546c0000/</code>
+
Video Control Toolkits Manual ：[[CS-MIPI-X i2c|CS-MIPI-X I2C]]
  
−
- Nano B01
+
* VEYE-MIPI-IMX335
  
−
<code>ls /proc/device-tree/cam_i2cmux/i2c@*</code>
+
Video Control Toolkits Manual ：[[5m mipi i2c user guide|VEYE-MIPI-IMX335 I2C]]
 +
===How to port the driver to a third party board===
  
−
- TX2 Devkit
+
==== driver porting ====
 +
For Image, we have added functionality to the official standard Image and have not made any deletions. In general, you can use our compiled Image directly. for special cases, please refer to the source code for integration.
  
−
<code>ls /proc/device-tree/i2c@3180000/tca9548@70/i2c@*</code>
+
For modules it is even easier to just use them. Compiling from source code and is very simple, so I won't go into details here.
  
−
or
+
==== dts porting ====
 +
We only provide dtb for some boards of Nano, TX2, Xavier, Orin. For the types not provided, it is necessary to：
  
−
<code>ls /proc/device-tree/i2c@3180000/tca9548@77/i2c@*</code>
+
1. get the dts source code of that board.
  
−
- AGX Xavier
+
2. put our camera related dts to the whole.
  
−
<code>ls /proc/device-tree/i2c@3180000/tca9548@70/i2c@*</code>
+
3. recompile to get the dtb of the corresponding board.
  
−
or
+
The operation steps are not complicated, but there are some third-party base board manufacturers do not open dts source code. This will require the cooperation of all parties to do so.
  
−
<code>ls /proc/device-tree/i2c@3180000/tca9548@77/i2c@*</code>
+
=== FAQ and bug list ===
  
−
- Xavier NX and TX2 NX(p3509-0000 carrier board)
+
==== VEYE cameras on Jetpack5.x ====
 +
Jetpack5.x has more stringent requirements for mipi signals in xavier and orin platforms. VEYE-MIPI-xxx series cameras need firmware hdver>=7 to be perfectly supported.
  
−
<code>ls /proc/device-tree/cam_i2cmux/i2c@*</code>
+
==== CS-MIPI-IMX307 combined with Orin nano and Orin NX Green Screen ====
 +
When CS-MIPI-IMX307 is powered by the 3.3V mode of the FFC cable, the MIPI receiver state machine of Orin nano and Orin NX (Jetpack5.1.1) will enter an incorrect state.
  
−
You should be able to find the camera model you used.
+
It is recommended to refer to this [[Power supply mode switching|article]] and modify the power supply mode of CS-MIPI-IMX307 to use the 5V power supply mode.
−
====Build the drivers from source====
 
−
Please refer to: [[VEYE CS Camera source for Jetson|Nvidia Jetson platform VEYE and CS series camera driver source code compilation guide]].
 
−
 
 
−
Could be used for Flashing or Upgrading.
 
−
===Applications and Test===
 
−
====Check system status====
 
−
Run the following command to confirm whether the camera is probed.
 
  
−
*For VEYE-MIPI-327(using VEYE-MIPI-327 dtb)
+
==== Jetpack5.0.1 DP ====
 +
This version has many bugs, such as not supporting nvv4l2camerasrc, not recommended to use.
  
−
<code>dmesg | grep veye327</code>  
+
==== nv3dsink ====
 +
Jetpack 5.x no longer supports nvoverlaysink, use nv3dsink instead.
 +
=== Document History ===
  
−
The output message appears as shown below：
+
* 2024-08-06
  
−
<code>Detected VEYE327 sensor</code>
+
Make Chapter 3 - Upgrading the Jetson system a separate article.
  
−
<code>subdev veye327 X-003b bound</code>
+
*2024-05-22
  
−
*VEYE-MIPI-IMX327S/VEYE-MIPI-IMX462/VEYE-MIPI-IMX385(using VEYE-MIPI-CAM2M dtb)
+
Add support for Jetpack5.1.3.
  
−
<code>dmesg | grep veye</code>  
+
*20230427
  
−
The output message appears as shown below：
+
Add support for Jetpack5.1.1.
  
−
<code>camera id is veyecam</code>
+
*20230222
  
−
<code>sensor is IMX327/IMX462/IMX385</code>
+
Add support for Jetpack5.1.
  
−
<code>subdev veyecam X-003b bound</code>
+
*20220831
  
−
*For CS-MIPI-IMX307
+
Improve the description of VEYE series and MV series under Jetpack 5.0.2.
  
−
<code>dmesg | grep csx307</code>  
+
* 20220824
  
−
The output message appears as shown below.：
+
Add support for Jetpack5.0.2.
  
−
<code>Detected CS307 sensor</code>
+
* 20220720
  
−
<code>subdev csx307 X-003b bound</code>
+
Add multimedia-api samples and v4l2 samples.
  
−
*For CS-MIPI-SC132
+
*20220714
  
−
<code>dmesg | grep cssc132</code>  
+
Add L4T32.7.2 Note。
  
−
The output message appears as shown below.：
+
* 20220629
  
−
<code>subdev cssc132 X-003b bound</code>
+
Add support for Jetpack 5.0.1DP and tune through AGX-Orin.
  
−
*Run the following command to check the presence of video node.
+
* 20220110
  
−
<code>ls /dev/video*</code>
+
Add VEYE-MIPI-IMX385 support.
−
 
 
−
The output message appears as shown below.
 
−
 
 
−
<code>videoX</code>  
 
−
 
 
−
====Video Stream Toolkits Manual====
 
−
Our camera appears as /dev/videoX device node,where X is between [0-5].
 
−
 
 
−
in gstreamer command ，v4l2src has a param device=/dev/videoX to identify which camera to use.
 
−
=====Gstreamer Usage=====
 
−
 
 
−
*Preview FHD
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvoverlaysink sync=false</code>
 
−
 
 
−
* Preview FHD(using xvimagesink sink)
 
−
 
 
−
<code>export DISPLAY=:0</code>
 
−
 
 
−
<code>gst-launch-1.0 -e v4l2src io-mode=4 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY' ! xvimagesink sync=false</code>
 
−
 
 
−
*Preview 720p@60 (CS-MIPI-IMX307)
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc  ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>
 
−
 
 
−
*Preview 1280*1080@45 (CS-MIPI-SC132)
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1280, height=(int)1080, framerate=(fraction)45/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>
 
−
 
 
−
*Preview 640*480p@130 (CS-MIPI-IMX307)
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)130/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>
 
−
 
 
−
*Preview 2 cameras  1080p HD
 
−
 
 
−
<code>WIDTH=960</code>
 
−
 
 
−
<code>HEIGHT=540</code>
 
−
 
 
−
<code>CAPS="video/x-raw(memory:NVMM),format=(string)UYVY, width=1920, height=1080"</code>
 
−
 
 
−
<code>gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=$WIDTH sink_0::height=$HEIGHT sink_1::xpos=$WIDTH sink_1::ypos=0 sink_1::width=$WIDTH sink_1::height=$HEIGHT ! nvoverlaysink nvv4l2camerasrc device=/dev/video0 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp. nvv4l2camerasrc  device=/dev/video1 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp.</code>
 
−
 
 
−
* Gstreamer cmd line for OpenCV
 
−
 
 
−
I think maybe OpenCV do not support I420 data format input，so you should convert it to BGR format.
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc ! video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1920, height=(int)1080 ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)I420 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink</code>
 
−
 
 
−
*Record FHD in H.264 format to a video file
 
−
 
 
−
<code><s>gst-launch-1.0 nvv4l2camerasrc  num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=videoname.mkv</s></code>
 
−
 
 
−
<code>gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=1 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e</code>
 
−
 
 
−
*Playback of saved video file
 
−
 
 
−
<code><s>gst-launch-1.0 filesrc location=videoname.mkv ! matroskademux ! h264parse ! omxh264dec ! nvoverlaysink</s></code>
 
−
 
 
−
<code>gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nvoverlaysink -e</code>
 
−
 
 
−
*Capturing FHD still image  
 
−
 
 
−
<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jpg</code>
 
−
====Video Control Toolkits Manual====
 
−
 
 
−
*i2c bus description：
 
−
 
 
−
For AGX-Xavier and TX2, ADP-N1 has 6 MIPI CSI-2 interface [A-F]，the corresponding i2c address is[30-35].
 
−
 
 
−
For Xavier NX, TX2 NX,the corresponding i2c address is 9 and10.
 
−
 
 
−
For Nano A02,Nano 2GB, the corresponding i2c address is 6.
 
−
 
 
−
For Nano B01, the corresponding i2c address is 7 and 8.
 
−
 
 
−
It is also the same as the X part of dmesg message:
 
−
 
 
−
<code>subdev veyeX X-003b bound</code>
 
−
 
 
−
<code>subdev csimx307 X-003b bound</code>
 
−
 
 
−
<code>subdev cssc132 X-003b bound</code>
 
−
 
 
−
using -b option to identify which bus you want to use.
 
−
 
 
−
*VEYE-MIPI-327
 
−
 
 
−
Video Control Toolkits Manual ：[[VEYE-MIPI-290/327 i2c/|VEYE-MIPI-327 I2C]]
 
−
 
 
−
*CS-MIPI-IMX307和CS-MIPI-SC132
 
−
 
 
−
Video Control Toolkits Manual ：[[CS-MIPI-X i2c|CS-MIPI-X I2C]]
 
−
===Realtimes RTSO-9001===
 
−
====Hardware Setup====
 
−
[[File:CS-MIPI-307 to ruitai 9001tx2.jpg|center|thumb|800x800px|CS-MIPI-IMX307 and RTSO-9001（TX2）]]
 
−
 
 
−
 
 
−
Matching with Realtimes RTSO-9001, you need to use RTST-RPI adapter board.
 
−
 
 
−
Note: the power supply capacity of the MIPI CSI-2 interface of the current version of RTSO-9001 is not enough to support the power requirements of our camera, which requires an additional 5V power supply to the camera. It is not shown in the picture above.
 
−
 
 
−
====Software bsp====
 
−
 
 
−
*The corresponding directory for Realtimes RTSO-9001, is：
 
−
 
 
−
nvidia_jetson_veye_bsp\TX2\JetPack_4.2.2_Linux_RTSO-9001
 
−
 
 
−
*Subject to the agreement with Ruitai, we do not release the source code
 
−
*Our code have been submitted to Realtimes, and customers are advised to contact Realtimes for a complete version of the supported image.
 
−
 
 
−
=== Document History ===
 
  
 
* 20220105
 
* 20220105
Line 509: Line 460:
 
L4T Doc:  https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3242/index.htm
 
L4T Doc:  https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3242/index.htm
  
−
TX2 user guide: [https://developer.download.nvidia.cn/embedded/L4T/r28_Release_v2.0/GA/Docs/Jetson_TX1_and_TX2_Developer_Kits_User_Guide.pdf?lumI95GW6BEyMmhP1Is8E79OYo105-tpsC2bOVQmBZ6Tw8AkLU4pSDMPCuzOaEonWMiDO2hliXbNvtkH9do81TNIXmLZGGoV7xoEehyEgfc2WO8e0Czz5chLSHoCqIuPLL1 link]
+
TX2 user guide: [https://developer.download.nvidia.cn/embedded/L4T/r28_Release_v2.0/GA/Docs/Jetson_TX1_and_TX2_Developer_Kits_User_Guide.pdf?lumI95GW6BEyMmhP1Is8E79OYo105-tpsC2bOVQmBZ6Tw8AkLU4pSDMPCuzOaEonWMiDO2hliXbNvtkH9do81TNIXmLZGGoV7xoEehyEgfc2WO8e0Czz5chLSHoCqIuPLL1 link]<br />
−
 
 
−
Realtimes website：http://www.realtimes.cn/
 
−
<br />
 

Latest revision as of 15:10, 18 November 2024

查看中文

How to use VEYE and CS series camera module on NVIDIA Jetson Nano,TX2,Xavier and Orin

1 Overview

This guide shows how to use VEYE and CS series camera modules on Jetson boards. Jetson SDK Version is：

  • Jetpack4.2.2,L4T r32.2.1
  • Jetpack4.3,L4T r32.3.1
  • Jetpack4.4,L4T r32.4.3
  • Jetpack4.4.1,L4T r32.4.4
  • Jetpack4.5,L4T r32.5
  • Jetpack4.5.1,L4T r32.5.1
  • Jetpack4.6,L4T r32.6.1
  • Jetpack4.6.1, L4T r32.7.1
  • Jetpack4.6.2,L4T r32.7.2
  • Jetpack4.6.3,L4T r32.7.3
  • Jetpack4.6.4,L4T r32.7.4
  • Jetpack5.0.1DP, L4T r34.1.1
  • Jetpack5.0.2, L4T r35.1
  • Jetpack5.1,L4T r35.2.1
  • Jetpack5.1.1,L4T r35.3.1
  • Jetpack5.1.2,L4T r35.4.1
  • Jetpack5.1.3,L4T r35.5
  • Jetpack6.0,L4T r36.3
  • Jetpack6.1,L4T r36.4

We provide two ways to do so: Prebuilt Binaries or Source Code. Yes, It's Open Source!

VEYE and CS series camera modules are camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide V4L2 interface for video streaming apps , and Video Control Toolkits (which is Shell Script) to control the camera module directly, which is called DRA(Directly Register Access).

1.1 Camera module list

Camera module dirver status list
Series Model Status
VEYE series VEYE-MIPI-IMX327S Done
VEYE series VEYE-MIPI-IMX385 Done
VEYE series VEYE-MIPI-IMX462 Done
VEYE series VEYE-MIPI-IMX335 Done
CS series CS-MIPI-IMX307 Done
CS series CS-MIPI-SC132 Done
TX2-XAVIER-nCAM Series CS-TX2-XAVIER-nCAM Done

1.2 How to check the current L4T version

1.2.1 Method 1

On Jetson board

cat /etc/nv_tegra_release

If it shows:

# R32 (release), REVISION: 4.3......

It means L4t Version is 32.4.3

1.2.2 Method 2

Refer to this link to install jetson-stats:

jtop

2 Hardware Setup

We have designed a 6cam interposer board for TX2 devkit,Xavier and Orin, It support up to 6 cameras at the same time.

6cam interposer board connected with 6 cameras
6cam interposer board connected with 6 cameras


In particular, the VEYE-MIPI-IMX335 must use a 12V DC terminal to provide auxiliary power due to its high power consumption. Refer to J11 of ADP-N1.

2.1 NVIDIA TX2 Developer Kit

It is connected as shown in figure:

TX2 Devkit 6 cameras connection
TX2 Devkit 6 cameras connection


Be sure to screw the ADP adapter board to the bottom board.

2.2 NVIDIA AGX Xavier(the same with Orin)


Connect VEYE cameras to AGX Xavier
Connect VEYE cameras to AGX Xavier


Be sure to screw the ADP adapter board to the bottom board to ensure that the adapter plate is installed flat.

2.3 NVIDIA Nano ，TX2 NX and Xavier NX Devkit

The Xavier NX interface is compatible with raspberry pi and Jetson Nano. The connection mode is shown in figure:

Xavier NX connection with VEYE-MIPI-327
Xavier NX connection with VEYE-MIPI-327


Xavier NX connection with CS-MIPI-IMX307
Xavier NX connection with CS-MIPI-IMX307
Jetson Nano A02 与VEYE模组连接
VEYE327 connection with Jetson Nano A02
VEYE camera  connection with Orin Nano and Orin NX

3 Upgrade Jetson system

This section describes how to upgrade the Jetson system to support MV camera module. For OS update method, please refer to Update Jetson OS.

4 Applications and Test

4.1 Check system status

Run the following command to confirm whether the camera is probed.

  • VEYE-MIPI-IMX327S/VEYE-MIPI-IMX462/VEYE-MIPI-IMX385(using VEYE-MIPI-CAM2M dtb)

dmesg | grep veye  

The output message appears as shown below：

camera id is veyecam

sensor is IMX327/IMX462/IMX385

subdev veyecam [i2c_bus]-003b bound

  • For CS-MIPI-IMX307

dmesg | grep csx307  

The output message appears as shown below.：

Detected CS307 sensor

subdev csx307 [i2c_bus]-003b bound

  • For CS-MIPI-SC132

dmesg | grep cssc132  

The output message appears as shown below.：

subdev cssc132 [i2c_bus]-003b bound

  • For VEYE-MIPI-IMX335

dmesg | grep imx335

The output message appears as shown below.：

camera id is VEYE-MIPI-IMX335

subdev veye_imx335 [i2c_bus]-003b bound

  • Run the following command to check the presence of video node.

ls /dev/video*

The output message appears as shown below.

videoX  

  • For VEYE-MIPI-327(using VEYE-MIPI-327 dtb)

dmesg | grep veye327  

The output message appears as shown below：

Detected VEYE327 sensor

subdev veye327 [i2c_bus]-003b bound

The [i2c_bus] in the driver prompt message indicates the i2c bus corresponding to this camera.

4.2 Video Stream Toolkits Manual

The camera appears as /dev/videoX device node,where X is between [0-5].

During the Linux boot process, the cameras are detected in the order of i2c bus from smallest to largest. For example, if only one camera is connected, X is 0 regardless of the hardware connection to the location. If 5 cameras are connected, X is reflected as [0-4] according to i2c bus from smallest to largest.

In gstreamer command ，v4l2src and nvv4l2camerasrc has a param device=/dev/videoX to identify which camera to use.

4.2.1 Gstreamer Usage

To install the latest accelerated gstreamer plugins and applications, run the following commands:

sudo apt-get update

sudo apt-get install nvidia-l4t-gstreamer

sudo ldconfig

rm -rf .cache/gstreamer-1.0/


export DISPLAY=:0

  • Preview FHD

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nv3dsink sync=false

  • Preview FHD(using xvimagesink sink if supported)

export DISPLAY=:0

gst-launch-1.0 -e v4l2src io-mode=4 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY' ! xvimagesink sync=false

  • Preview 720p@60 (CS-MIPI-IMX307)

gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false

  • Preview 1280*1080@45 (CS-MIPI-SC132)

gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1280, height=(int)1080, framerate=(fraction)45/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false

  • Preview 640*480p@130 (CS-MIPI-IMX307)

gst-launch-1.0 nvv4l2camerasrc ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)130/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false

  • Preview 2 cameras 1080p HD

WIDTH=960

HEIGHT=540

CAPS="video/x-raw(memory:NVMM),format=(string)UYVY, width=1920, height=1080"

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=$WIDTH sink_0::height=$HEIGHT sink_1::xpos=$WIDTH sink_1::ypos=0 sink_1::width=$WIDTH sink_1::height=$HEIGHT ! nv3dsink nvv4l2camerasrc device=/dev/video0 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp. nvv4l2camerasrc device=/dev/video1 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp.

  • Record FHD in H.264 format to a video file

gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=1 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e

  • Playback of saved video file

gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nvoverlaysink -e

  • Capturing FHD still image  

gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jp

4.2.2 VEYE-MIPI-IMX335 gstreamer example

Since VEYE-MIPI-IMX335 supports only a few specific resolution modes, its driver we use use_sensor_mode_id mode.

VEYE-MIPI-IMX335 mode
./veye5_mipi_i2c.sh video mode v4l2-ctl sensor_mode video format
1 0 2592x1944@20fps
2 1 2592x1944@12.5fps
3 2 2560x1440@25fps
4 3 2560x1440@30fps
  • prepare

sudo apt-get install v4l-utils

Refer to the following section and download the veye5_mipi_i2c.sh tool.

  • 2592x1944@20fps mode preview, framerate statistics

v4l2-ctl -d /dev/video0 --set-ctrl sensor_mode=0

./veye5_mipi_i2c.sh -w -f videomode -p1 1 -b [busnum]

v4l2-ctl -d /dev/video0 --set-ctrl preferred_stride=5376

Preview

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM), width=(int)2592, height=(int)1944" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nv3dsink sync=false

Framerate statistics

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY,width=(int)2592,height=(int)1944" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! fpsdisplaysink video-sink=fakesink -v

  • 2560x1440@30fps mode video encode and capture

v4l2-ctl -d /dev/video0 --set-ctrl sensor_mode=3

./veye5_mipi_i2c.sh -w -f videomode -p1 4 -b [busnum]

v4l2-ctl -d /dev/video0 --set-ctrl preferred_stride=5120

Video encode

gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)2560, height=(int)1440" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=0 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e

Video Replay

gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nv3dsink -e

Capture

gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)2560, height=(int)1440" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jpg

4.2.3 Jetson multimedia-api samples

The Jetson Linux API is available on the Jetson platform for developers to use. Please refer to the official nVidia documentation for the installation of the jetson_multimedia_api package.

For VEYE and CS series cameras, the ISP function is already integrated in the camera, so you cannot use libargus to get data, but can directly use the standard V4L2 interface to get data from the /dev/videoX device file.

The following two samples can be run directly.

4.2.3.1 12_camera_v4l2_cuda

./camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UYVY

4.2.3.2 v4l2cuda

./capture-cuda -d /dev/video0 -f UYVY -m -o out.ppm -s 1920x1080 -c 1

4.2.4 v4l2-ctl Application Examples
4.2.4.1 install v4l2-utils

sudo apt-get install v4l-utils

4.2.4.2 Configure parameters using v4l2-ctl
  • List the data formats supported by the camera

v4l2-ctl -d /dev/video0 --list-formats-ext

  • Frame rate statistics

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY--stream-mmap --stream-count=-1 --stream-to=/dev/null

  • Save image to file

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv

4.2.5 yavta
4.2.5.1 install yavta

git clone https://github.com/veyeimaging/yavta.git

cd yavta;make

4.2.5.2 Save image to file

./yavta -c1 -FUYVY-1920x1080.yuv --skip 0 -f UYVY -s 1920x1080 /dev/video0

4.2.6 Opencv

To import camera data from v4l2 devices to opencv, we provide several samples.

We provide 2 samples here.

4.3 Using i2c script to modify camera parameters

  • VEYE-MIPI-CAM2M Series（VEYE-MIPI-327、VEYE-MIPI-IMX327S、VEYE-MIPI-IMX462、VEYE-MIPI-IMX385）

Video Control Toolkits Manual ：VEYE-MIPI-327 I2C

  • CS-MIPI-IMX307和CS-MIPI-SC132

Video Control Toolkits Manual ：CS-MIPI-X I2C

  • VEYE-MIPI-IMX335

Video Control Toolkits Manual ：VEYE-MIPI-IMX335 I2C

5 How to port the driver to a third party board

5.1 driver porting

For Image, we have added functionality to the official standard Image and have not made any deletions. In general, you can use our compiled Image directly. for special cases, please refer to the source code for integration.

For modules it is even easier to just use them. Compiling from source code and is very simple, so I won't go into details here.

5.2 dts porting

We only provide dtb for some boards of Nano, TX2, Xavier, Orin. For the types not provided, it is necessary to：

1. get the dts source code of that board.

2. put our camera related dts to the whole.

3. recompile to get the dtb of the corresponding board.

The operation steps are not complicated, but there are some third-party base board manufacturers do not open dts source code. This will require the cooperation of all parties to do so.

6 FAQ and bug list

6.1 VEYE cameras on Jetpack5.x

Jetpack5.x has more stringent requirements for mipi signals in xavier and orin platforms. VEYE-MIPI-xxx series cameras need firmware hdver>=7 to be perfectly supported.

6.2 CS-MIPI-IMX307 combined with Orin nano and Orin NX Green Screen

When CS-MIPI-IMX307 is powered by the 3.3V mode of the FFC cable, the MIPI receiver state machine of Orin nano and Orin NX (Jetpack5.1.1) will enter an incorrect state.

It is recommended to refer to this article and modify the power supply mode of CS-MIPI-IMX307 to use the 5V power supply mode.

6.3 Jetpack5.0.1 DP

This version has many bugs, such as not supporting nvv4l2camerasrc, not recommended to use.

6.4 nv3dsink

Jetpack 5.x no longer supports nvoverlaysink, use nv3dsink instead.

7 Document History

  • 2024-08-06

Make Chapter 3 - Upgrading the Jetson system a separate article.

  • 2024-05-22

Add support for Jetpack5.1.3.

  • 20230427

Add support for Jetpack5.1.1.

  • 20230222

Add support for Jetpack5.1.

  • 20220831

Improve the description of VEYE series and MV series under Jetpack 5.0.2.

  • 20220824

Add support for Jetpack5.0.2.

  • 20220720

Add multimedia-api samples and v4l2 samples.

  • 20220714

Add L4T32.7.2 Note。

  • 20220629

Add support for Jetpack 5.0.1DP and tune through AGX-Orin.

  • 20220110

Add VEYE-MIPI-IMX385 support.

  • 20220105

Use nvv4l2h264enc instead of omxh264enc in gstreamer command，because omxh264enc has been deprecated.

  • 20211025

Use nvcamerasrc instead of v4l2src in gstreamer command , and the data goes directly into DMA memory.

8 References

Jetson start up： https://developer.nvidia.com/embedded/learn/getting-started-jetson

Jetson Download Center: https://developer.nvidia.com/embedded/downloads

Xavier NX: https://developer.nvidia.com/embedded/learn/get-started-jetson-xavier-nx-devkit

TX2 development kit： https://developer.nvidia.com/embedded/jetson-tx2-developer-kit

SDK Manager: https://docs.nvidia.com/sdk-manager/index.html

L4T Doc: https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3242/index.htm

TX2 user guide: link

Retrieved from "http://wiki.veye.cc/index.php?title=VEYE_CS_Camera_for_Jetson_TX2&oldid=7669"