Difference between revisions of "VEYE CS Camera for Jetson TX2"

From wiki_veye
Jump to navigation Jump to search
(Created page with "查看中文 <br />")
 
Line 1: Line 1:
 
[[VEYE CS Camera for Jetson TX2/zh|查看中文]]
 
[[VEYE CS Camera for Jetson TX2/zh|查看中文]]
  
 +
'''<big>How to use VEYE and CS series camera module on NVIDIA Jetson TX2</big>'''
 +
 +
===Overview===
 +
This guide shows how to use VEYE and CS series camera modules on Jetson TX2.  Jetson TX2 JetPack SDK Version is r32.2.1. We provide two ways to do so: '''Prebuilt Binaries''' or '''Source Code. Yes, It's Open Source!'''
 +
 +
VEYE and CS series camera modules are STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide '''V4L2 interface''' for video streaming apps , and '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly, which is called DRA(Directly Register Access).
 +
===Hardware Setup===
 +
====NVIDIA TX2 Developer Kit====
 +
We have designed a special adapter board for TX2 developwer  kit, It support up to 6 cameras at the same time.
 +
 +
It is connected as shown in figure (TODO).
 +
<br />
 +
===Upgrade Jetson TX2 system===
 +
====Setting Up the Host PC Environment====
 +
 +
*Cross-compiling Toolchain
 +
 +
Please refer to this [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fxavier_toolchain.html%23wwpID0ESHA link] to install toolchain on your Host PC.
 +
 +
*L4T source code and Rootfs
 +
 +
You could use SDK Manager or [https://developer.nvidia.com/embedded/downloads directly download] to get source code.
 +
 +
*Setting Up the Environment
 +
 +
Assume SDK install directory is <TOPDIR>，source code is in $L4T_DIR/sources directory.
 +
 +
<code>export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3310/</code>
 +
 +
<code>export L4T_DIR=$TOP_DIR/Linux_for_Tegra</code>
 +
 +
<code>export LOCALVERSION=-tegra</code>
 +
 +
<code>export LDK_ROOTFS_DIR=$TOP_DIR/Linux_for_Tegra/rootfs</code>
 +
 +
<code>export ARCH=arm64</code>
 +
 +
<code>export CROSS_COMPILE=aarch64-linux-gnu-</code>
 +
 +
<code>export CROSS32CC=arm-linux-gnueabihf-gcc</code>
 +
 +
<code>export TEGRA_KERNEL_OUT=$L4T_DIR/sources/kernel/out_kernel</code>
 +
 +
<code>export KERNEL_PATH=$L4T_DIR/sources/kernel/out_kernel</code>
 +
 +
<code>export NVIDIA_PATH=$L4T_DIR/sources/kernel/nvidia</code>
 +
 +
<code>export TX2_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t18x</code>
 +
 +
<code>cd $L4T_DIR/sources/kernel/kernel-4.9/</code>
 +
 +
*Build default image, flashing to TX2, prepare L4T environment
 +
 +
<code>cd $L4T_DIR</code>
 +
 +
<code>sudo ./apply_binaries.sh</code>  
 +
 +
<code>sudo ./flash.sh jetson-tx2 mmcblk0p1</code>
 +
====Download out bsp package for Jetson TX2====
 +
<code>cd $L4T_DIR</code>
 +
 +
<code>git clone https://github.com/veyeimaging/nvidia_jetson_veye_bsp<nowiki/>.git</code>
 +
 +
<code>export RELEASE_PACK_DIR=$L4T_DIR/nvidia_jetson_veye_bsp</code>
 +
=====bsp package includes：=====
 +
 +
*prebuild linux kernel：Image
 +
*prebuild dtb for different boards
 +
*driver source code
 +
*dts source code
 +
*i2c toolkits
 +
 +
====Using prebuild Image and DTB====
 +
 +
*Installing the Kernel
 +
 +
Whether you use a prebuild kernel or you build it yourself , you just need to copy it to /boot/ dir on the Jetson TX2 board:
 +
 +
<code>sudo cp <path to your Image dir>/Image  /boot/Image -f</code>
 +
 +
*Flash dtb file
 +
 +
Enter the recovery mode，and on the Host PC:
 +
 +
<code>cp <path to your dtb dir>/dtbfilename $L4T_DIR/kernel/dtb/ -f</code>
 +
 +
<code>cd $L4T_DIR</code>
 +
 +
<code>sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1</code>
 +
 +
Reset TX2 after flashing.
 +
====Building from Source====
 +
=====Build kernel=====
 +
 +
*patch code
 +
 +
<code>cp $RELEASE_PACK_DIR/drivers_source/cs_imx307 veye327/* $NVIDIA_PATH/drivers/media/i2c/</code>
 +
 +
<code>cp $RELEASE_PACK_DIR/drivers_source/kernel_csimx307veye327_config $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/kernel_csimx307veye327_config</code>
 +
 +
*build
 +
 +
<code>cd $L4T_DIR/sources/kernel/kernel-4.9/</code>
 +
 +
<code>make ARCH=arm64 O=$TEGRA_KERNEL_OUT kernel_csimx307veye327_config</code>
 +
 +
<code>make ARCH=arm64 O=$TEGRA_KERNEL_OUT Image -j4</code>
 +
 +
Completed Image file is $TEGRA_KERNEL_OUT/arch/arm64/boot/Image，could be used for Flashing or Upgrading.
 +
 +
Install Image for Flashing:
 +
 +
<code>sudo cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image $L4T_DIR/kernel/ -f</code>
 +
=====Build DTS=====
 +
 +
*patch code
 +
 +
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/common/t18x/* -r $TX2_DTS_PATH/</code>
 +
 +
For CS-MIPI-IMX307
 +
 +
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/CS-MIPI-IMX307/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/</code>
 +
 +
For VEYE-MIPI-327
 +
 +
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/VEYE-MIPI-327/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/</code>
 +
 +
*build
 +
 +
<code>cd $L4T_DIR/sources/kernel/kernel-4.9/</code>
 +
 +
<code>make ARCH=arm64 O=$TEGRA_KERNEL_OUT dtbs</code>
 +
 +
Could be used for Flashing or Upgrading.
 +
===Applications and Test===
 +
====Check system status====
 +
Run the following command to confirm whether the camera is probed.
 +
 +
*For VEYE-MIPI-327
 +
 +
<code>dmesg | grep veye327</code>  
 +
 +
The output message appears as shown below.：
 +
 +
<code>Detected VEYE327 sensor</code>
 +
 +
<code>subdev veye327 3X-003b bound</code>
 +
 +
此外，检查/dev/videoX设备节点，应当是存在的。
 +
 +
*For CS-MIPI-IMX307
 +
 +
<code>dmesg | grep csimx307</code>  
 +
 +
The output message appears as shown below.：
 +
 +
<code>Detected CS307 sensor</code>
 +
 +
<code>subdev csimx307 3X-003b bound</code>
 +
 +
* Run the following command to check the presence of video node.
 +
 +
<code>ls /dev/video*</code>
 +
 +
The output message appears as shown below.
 +
 +
<code>videoX</code>  
 +
 +
====Video Stream Toolkits Manual====
 +
Our camera appears as /dev/videoX device node,where X is between [0-5].
 +
 +
in gstreamer command ，v4l2src has a param device=/dev/videoX to identify which camera to use.
 +
=====Gstreamer Usage=====
 +
 +
*Preview FHD(HW accelerated)
 +
 +
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>
 +
 +
*Preview 720p@60 (CS-MIPI-IMX307,HW accelerated)
 +
 +
<code>gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>
 +
 +
*Record FHD in H.264 format to a video file（HW accelerated）
 +
 +
<code>gst-launch-1.0 v4l2src num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=videoname.mkv</code>
 +
 +
*Playback of saved video file (HW accelerated）
 +
 +
<code>gst-launch-1.0 filesrc location=videoname.mkv ! matroskademux ! h264parse ! omxh264dec ! nvoverlaysink</code>
 +
 +
*Capturing FHD still image  
 +
 +
<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! filesink location=jpgname.jpg</code>
 +
====Video Control Toolkits Manual====
 +
 +
*i2c bus description：
 +
 +
ADP-TX2-6CAM has 6 MIPI CSI-2 interface [A-F]，the corresponding i2c address is[30-35].
 +
 +
It is also the same as the 3X part of dmesg message:
 +
 +
<code>subdev veye327 3X-003b bound</code>
 +
 +
<code>subdev csimx307 3X-003b bound</code>
 +
 +
using -b option to identify which bus you want to use.
 +
 +
*VEYE-MIPI-327
 +
 +
Video Control Toolkits Manual ：[[VEYE-MIPI-290/327 i2c/|VEYE-MIPI-327 I2C]]
 +
 +
*CS-MIPI-IMX307
 +
 +
Video Control Toolkits Manual ：[[CS-MIPI-X i2c|CS-MIPI-IMX307 I2C]]
 +
===Realtimes RTSO-9001 ===
 +
====Hardware Setup====
 +
[[File:CS-MIPI-307 to ruitai 9001tx2.jpg|center|thumb|800x800px|CS-MIPI-IMX307 and RTSO-9001（TX2）|link=http://wiki.veye.cc/index.php/File:CS-MIPI-307_to_ruitai_9001tx2.jpg]]
 +
 +
 +
Matching with Realtimes RTSO-9001, you need to use RTST-RPI adapter board.
 +
 +
Note: the power supply capacity of the MIPI CSI-2 interface of the current version of RTSO-9001 is not enough to support the power requirements of our camera, which requires an additional 5V power supply to the camera. It is not shown in the picture above.
 +
====Software bsp====
 +
 +
*The corresponding directory for Realtimes RTSO-9001, is：
 +
 +
nvidia_jetson_veye_bsp\TX2\JetPack_4.2.2_Linux_RTSO-9001
 +
 +
*Subject to the agreement with Ruitai, we do not release the source code
 +
*Our code have been submitted to Realtimes, and customers are advised to contact Realtimes for a complete version of the supported image.
 +
 +
===References===
 +
Jetson start up： https://developer.nvidia.com/embedded/learn/getting-started-jetson
 +
 +
TX2 development kit： https://developer.nvidia.com/embedded/jetson-tx2-developer-kit
 +
 +
SDK Manager: https://docs.nvidia.com/sdk-manager/index.html
 +
 +
TX2 user guide: [https://developer.download.nvidia.cn/embedded/L4T/r28_Release_v2.0/GA/Docs/Jetson_TX1_and_TX2_Developer_Kits_User_Guide.pdf?lumI95GW6BEyMmhP1Is8E79OYo105-tpsC2bOVQmBZ6Tw8AkLU4pSDMPCuzOaEonWMiDO2hliXbNvtkH9do81TNIXmLZGGoV7xoEehyEgfc2WO8e0Czz5chLSHoCqIuPLL1 link]
 +
 +
Realtimes website：http://www.realtimes.cn/
 
<br />
 
<br />

Revision as of 15:38, 8 March 2020

查看中文

How to use VEYE and CS series camera module on NVIDIA Jetson TX2

1 Overview

This guide shows how to use VEYE and CS series camera modules on Jetson TX2. Jetson TX2 JetPack SDK Version is r32.2.1. We provide two ways to do so: Prebuilt Binaries or Source Code. Yes, It's Open Source!

VEYE and CS series camera modules are STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide V4L2 interface for video streaming apps , and Video Control Toolkits (which is Shell Script) to control the camera module directly, which is called DRA(Directly Register Access).

2 Hardware Setup

2.1 NVIDIA TX2 Developer Kit

We have designed a special adapter board for TX2 developwer kit, It support up to 6 cameras at the same time.

It is connected as shown in figure (TODO).

3 Upgrade Jetson TX2 system

3.1 Setting Up the Host PC Environment

  • Cross-compiling Toolchain

Please refer to this link to install toolchain on your Host PC.

  • L4T source code and Rootfs

You could use SDK Manager or directly download to get source code.

  • Setting Up the Environment

Assume SDK install directory is <TOPDIR>，source code is in $L4T_DIR/sources directory.

export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3310/

export L4T_DIR=$TOP_DIR/Linux_for_Tegra

export LOCALVERSION=-tegra

export LDK_ROOTFS_DIR=$TOP_DIR/Linux_for_Tegra/rootfs

export ARCH=arm64

export CROSS_COMPILE=aarch64-linux-gnu-

export CROSS32CC=arm-linux-gnueabihf-gcc

export TEGRA_KERNEL_OUT=$L4T_DIR/sources/kernel/out_kernel

export KERNEL_PATH=$L4T_DIR/sources/kernel/out_kernel

export NVIDIA_PATH=$L4T_DIR/sources/kernel/nvidia

export TX2_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t18x

cd $L4T_DIR/sources/kernel/kernel-4.9/

  • Build default image, flashing to TX2, prepare L4T environment

cd $L4T_DIR

sudo ./apply_binaries.sh  

sudo ./flash.sh jetson-tx2 mmcblk0p1

3.2 Download out bsp package for Jetson TX2

cd $L4T_DIR

git clone https://github.com/veyeimaging/nvidia_jetson_veye_bsp.git

export RELEASE_PACK_DIR=$L4T_DIR/nvidia_jetson_veye_bsp

3.2.1 bsp package includes：
  • prebuild linux kernel：Image
  • prebuild dtb for different boards
  • driver source code
  • dts source code
  • i2c toolkits

3.3 Using prebuild Image and DTB

  • Installing the Kernel

Whether you use a prebuild kernel or you build it yourself , you just need to copy it to /boot/ dir on the Jetson TX2 board:

sudo cp <path to your Image dir>/Image /boot/Image -f

  • Flash dtb file

Enter the recovery mode，and on the Host PC:

cp <path to your dtb dir>/dtbfilename $L4T_DIR/kernel/dtb/ -f

cd $L4T_DIR

sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1

Reset TX2 after flashing.

3.4 Building from Source

3.4.1 Build kernel
  • patch code

cp $RELEASE_PACK_DIR/drivers_source/cs_imx307 veye327/* $NVIDIA_PATH/drivers/media/i2c/

cp $RELEASE_PACK_DIR/drivers_source/kernel_csimx307veye327_config $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/kernel_csimx307veye327_config

  • build

cd $L4T_DIR/sources/kernel/kernel-4.9/

make ARCH=arm64 O=$TEGRA_KERNEL_OUT kernel_csimx307veye327_config

make ARCH=arm64 O=$TEGRA_KERNEL_OUT Image -j4

Completed Image file is $TEGRA_KERNEL_OUT/arch/arm64/boot/Image，could be used for Flashing or Upgrading.

Install Image for Flashing:

sudo cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image $L4T_DIR/kernel/ -f

3.4.2 Build DTS
  • patch code

cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/common/t18x/* -r $TX2_DTS_PATH/

For CS-MIPI-IMX307

cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/CS-MIPI-IMX307/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/

For VEYE-MIPI-327

cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/VEYE-MIPI-327/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/

  • build

cd $L4T_DIR/sources/kernel/kernel-4.9/

make ARCH=arm64 O=$TEGRA_KERNEL_OUT dtbs

Could be used for Flashing or Upgrading.

4 Applications and Test

4.1 Check system status

Run the following command to confirm whether the camera is probed.

  • For VEYE-MIPI-327

dmesg | grep veye327  

The output message appears as shown below.：

Detected VEYE327 sensor

subdev veye327 3X-003b bound

此外，检查/dev/videoX设备节点，应当是存在的。

  • For CS-MIPI-IMX307

dmesg | grep csimx307  

The output message appears as shown below.：

Detected CS307 sensor

subdev csimx307 3X-003b bound

  • Run the following command to check the presence of video node.

ls /dev/video*

The output message appears as shown below.

videoX  

4.2 Video Stream Toolkits Manual

Our camera appears as /dev/videoX device node,where X is between [0-5].

in gstreamer command ，v4l2src has a param device=/dev/videoX to identify which camera to use.

4.2.1 Gstreamer Usage
  • Preview FHD(HW accelerated)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • Preview 720p@60 (CS-MIPI-IMX307,HW accelerated)

gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • Record FHD in H.264 format to a video file（HW accelerated）

gst-launch-1.0 v4l2src num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=videoname.mkv

  • Playback of saved video file (HW accelerated）

gst-launch-1.0 filesrc location=videoname.mkv ! matroskademux ! h264parse ! omxh264dec ! nvoverlaysink

  • Capturing FHD still image  

gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! filesink location=jpgname.jpg

4.3 Video Control Toolkits Manual

  • i2c bus description：

ADP-TX2-6CAM has 6 MIPI CSI-2 interface [A-F]，the corresponding i2c address is[30-35].

It is also the same as the 3X part of dmesg message:

subdev veye327 3X-003b bound

subdev csimx307 3X-003b bound

using -b option to identify which bus you want to use.

  • VEYE-MIPI-327

Video Control Toolkits Manual ：VEYE-MIPI-327 I2C

  • CS-MIPI-IMX307

Video Control Toolkits Manual ：CS-MIPI-IMX307 I2C

5 Realtimes RTSO-9001

5.1 Hardware Setup

CS-MIPI-IMX307 and RTSO-9001（TX2）


Matching with Realtimes RTSO-9001, you need to use RTST-RPI adapter board.

Note: the power supply capacity of the MIPI CSI-2 interface of the current version of RTSO-9001 is not enough to support the power requirements of our camera, which requires an additional 5V power supply to the camera. It is not shown in the picture above.

5.2 Software bsp

  • The corresponding directory for Realtimes RTSO-9001, is：

nvidia_jetson_veye_bsp\TX2\JetPack_4.2.2_Linux_RTSO-9001

  • Subject to the agreement with Ruitai, we do not release the source code
  • Our code have been submitted to Realtimes, and customers are advised to contact Realtimes for a complete version of the supported image.

6 References

Jetson start up： https://developer.nvidia.com/embedded/learn/getting-started-jetson

TX2 development kit： https://developer.nvidia.com/embedded/jetson-tx2-developer-kit

SDK Manager: https://docs.nvidia.com/sdk-manager/index.html

TX2 user guide: link

Realtimes website：http://www.realtimes.cn/

Retrieved from "http://wiki.veye.cc/index.php?title=VEYE_CS_Camera_for_Jetson_TX2&oldid=1605"