Difference between revisions of "VEYE CS Camera for Jetson TX2"

From wiki_veye
Jump to navigation Jump to search
(add support AGX xaviar)
Line 1: Line 1:
 
[[VEYE CS Camera for Jetson TX2/zh|查看中文]]
 
[[VEYE CS Camera for Jetson TX2/zh|查看中文]]
  
−
'''<big>How to use VEYE and CS series camera module on NVIDIA Jetson TX2</big>'''
+
'''<big>How to use VEYE and CS series camera module on NVIDIA Jetson TX2 and AGX Xavier</big>'''
  
 
===Overview===
 
===Overview===
−
This guide shows how to use VEYE and CS series camera modules on Jetson TX2.  Jetson TX2 JetPack SDK Version is r32.2.1. We provide two ways to do so: '''Prebuilt Binaries''' or '''Source Code. Yes, It's Open Source!'''
+
This guide shows how to use VEYE and CS series camera modules on Jetson TX2 and AGX Xavier.  Jetson JetPack SDK Version is r32.2.1. We provide two ways to do so: '''Prebuilt Binaries''' or '''Source Code. Yes, It's Open Source!'''
  
 
VEYE and CS series camera modules are STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide '''V4L2 interface''' for video streaming apps , and '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly, which is called DRA(Directly Register Access).
 
VEYE and CS series camera modules are STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide '''V4L2 interface''' for video streaming apps , and '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly, which is called DRA(Directly Register Access).
Line 12: Line 12:
  
 
It is connected as shown in figure (TODO).
 
It is connected as shown in figure (TODO).
 +
 +
==== NVIDIA TX2 AGX Xavier ====
 
<br />
 
<br />
−
===Upgrade Jetson TX2 system===
+
===Upgrade Jetson TX2 and AGX Xavier system===
 
====Setting Up the Host PC Environment====
 
====Setting Up the Host PC Environment====
  
Line 23: Line 25:
  
 
You could use SDK Manager or [https://developer.nvidia.com/embedded/downloads directly download] to get source code.
 
You could use SDK Manager or [https://developer.nvidia.com/embedded/downloads directly download] to get source code.
 +
 +
Sync code as this if you use SDK Manager:
 +
 +
<code>./source_sync.sh -t tegra-l4t-r32.2.1</code>
  
 
*Setting Up the Environment
 
*Setting Up the Environment
  
 
Assume SDK install directory is <TOPDIR>，source code is in $L4T_DIR/sources directory.
 
Assume SDK install directory is <TOPDIR>，source code is in $L4T_DIR/sources directory.
 +
 +
''- TX2''
  
 
<code>export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3310/</code>
 
<code>export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3310/</code>
 +
 +
''- AGX Xavier''
 +
 +
<code>export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P2888/</code>
  
 
<code>export L4T_DIR=$TOP_DIR/Linux_for_Tegra</code>
 
<code>export L4T_DIR=$TOP_DIR/Linux_for_Tegra</code>
Line 49: Line 61:
  
 
<code>export TX2_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t18x</code>
 
<code>export TX2_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t18x</code>
 +
 +
<code>export XAVIER_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t19x</code>
  
 
<code>cd $L4T_DIR/sources/kernel/kernel-4.9/</code>
 
<code>cd $L4T_DIR/sources/kernel/kernel-4.9/</code>
  
−
*Build default image, flashing to TX2, prepare L4T environment
+
*Build default image, flashing to Jetson Board, prepare L4T environment
  
 
<code>cd $L4T_DIR</code>
 
<code>cd $L4T_DIR</code>
  
 
<code>sudo ./apply_binaries.sh</code>  
 
<code>sudo ./apply_binaries.sh</code>  
 +
 +
''- TX2''
  
 
<code>sudo ./flash.sh jetson-tx2 mmcblk0p1</code>
 
<code>sudo ./flash.sh jetson-tx2 mmcblk0p1</code>
−
====Download out bsp package for Jetson TX2====
+
 
 +
''- AGX Xavier''
 +
 
 +
<code>sudo ./flash.sh jetson-xavier mmcblk0p1</code>
 +
====Download out bsp package for Jetson ====
 
<code>cd $L4T_DIR</code>
 
<code>cd $L4T_DIR</code>
  
Line 77: Line 97:
 
*Installing the Kernel
 
*Installing the Kernel
  
−
Whether you use a prebuild kernel or you build it yourself , you just need to copy it to /boot/ dir on the Jetson TX2 board:
+
Whether you use a prebuild kernel or you build it yourself  
 +
 
 +
''- TX2''
 +
 
 +
copy Image to /boot/ dir on the Jetson TX2 board:
  
 
<code>sudo cp <path to your Image dir>/Image  /boot/Image -f</code>
 
<code>sudo cp <path to your Image dir>/Image  /boot/Image -f</code>
  
−
*Flash dtb file
+
''- AGX Xavier''
 +
 
 +
on the Host PC:
 +
 
 +
<code>cd $L4T_DIR</code>
 +
 
 +
<code>sudo ./flash.sh -k kernel jetson-xavier mmcblk0p1</code>
 +
 
 +
*Flashing dtb file
  
 
Enter the recovery mode，and on the Host PC:
 
Enter the recovery mode，and on the Host PC:
Line 88: Line 120:
  
 
<code>cd $L4T_DIR</code>
 
<code>cd $L4T_DIR</code>
 +
 +
<code>''- TX2''</code>
  
 
<code>sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1</code>
 
<code>sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1</code>
  
−
Reset TX2 after flashing.
+
<code>''- AGX Xavier''</code>
 +
 
 +
<code>sudo ./flash.sh -r -k kernel-dtb jetson-xavier mmcblk0p1</code>
 +
 
 +
Reset Jetson Board after flashing.
 
====Building from Source====
 
====Building from Source====
 
=====Build kernel=====
 
=====Build kernel=====
Line 116: Line 154:
 
=====Build DTS=====
 
=====Build DTS=====
  
−
*patch code
+
* patch code
 +
 
 +
''- Jetson TX2''
  
 
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/common/t18x/* -r $TX2_DTS_PATH/</code>
 
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/common/t18x/* -r $TX2_DTS_PATH/</code>
Line 127: Line 167:
  
 
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/VEYE-MIPI-327/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/</code>
 
<code>cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/VEYE-MIPI-327/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/</code>
 +
 +
''- AGX Xavier''
 +
 +
<code>cp $RELEASE_PACK_DIR/AGX-Xaviar/JetPack_4.2.2_Linux_GA_P2888/dts\ dtb/common/t19x/* -r $XAVIER_DTS_PATH/</code>
 +
 +
For CS-MIPI-IMX307
 +
 +
<code>cp $RELEASE_PACK_DIR/AGX-Xaviar/JetPack_4.2.2_Linux_GA_P2888/dts\ dtb/CS-MIPI-IMX307/tegra194-p2888-0001-p2822-0000.dts $TX2_DTS_PATH/galen/kernel-dts/</code>
 +
 +
For VEYE-MIPI-327
 +
 +
<code>cp $RELEASE_PACK_DIR/AGX-Xaviar/JetPack_4.2.2_Linux_GA_P2888/dts\ dtb/VEYE-MIPI-327/tegra194-p2888-0001-p2822-0000.dts $TX2_DTS_PATH/galen/kernel-dts/</code>
  
 
*build
 
*build

Revision as of 17:00, 15 March 2020

查看中文

How to use VEYE and CS series camera module on NVIDIA Jetson TX2 and AGX Xavier

1 Overview

This guide shows how to use VEYE and CS series camera modules on Jetson TX2 and AGX Xavier. Jetson JetPack SDK Version is r32.2.1. We provide two ways to do so: Prebuilt Binaries or Source Code. Yes, It's Open Source!

VEYE and CS series camera modules are STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide V4L2 interface for video streaming apps , and Video Control Toolkits (which is Shell Script) to control the camera module directly, which is called DRA(Directly Register Access).

2 Hardware Setup

2.1 NVIDIA TX2 Developer Kit

We have designed a special adapter board for TX2 developwer kit, It support up to 6 cameras at the same time.

It is connected as shown in figure (TODO).

2.2 NVIDIA TX2 AGX Xavier


3 Upgrade Jetson TX2 and AGX Xavier system

3.1 Setting Up the Host PC Environment

  • Cross-compiling Toolchain

Please refer to this link to install toolchain on your Host PC.

  • L4T source code and Rootfs

You could use SDK Manager or directly download to get source code.

Sync code as this if you use SDK Manager:

./source_sync.sh -t tegra-l4t-r32.2.1

  • Setting Up the Environment

Assume SDK install directory is <TOPDIR>，source code is in $L4T_DIR/sources directory.

- TX2

export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3310/

- AGX Xavier

export TOP_DIR=/home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P2888/

export L4T_DIR=$TOP_DIR/Linux_for_Tegra

export LOCALVERSION=-tegra

export LDK_ROOTFS_DIR=$TOP_DIR/Linux_for_Tegra/rootfs

export ARCH=arm64

export CROSS_COMPILE=aarch64-linux-gnu-

export CROSS32CC=arm-linux-gnueabihf-gcc

export TEGRA_KERNEL_OUT=$L4T_DIR/sources/kernel/out_kernel

export KERNEL_PATH=$L4T_DIR/sources/kernel/out_kernel

export NVIDIA_PATH=$L4T_DIR/sources/kernel/nvidia

export TX2_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t18x

export XAVIER_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t19x

cd $L4T_DIR/sources/kernel/kernel-4.9/

  • Build default image, flashing to Jetson Board, prepare L4T environment

cd $L4T_DIR

sudo ./apply_binaries.sh  

- TX2

sudo ./flash.sh jetson-tx2 mmcblk0p1

- AGX Xavier

sudo ./flash.sh jetson-xavier mmcblk0p1

3.2 Download out bsp package for Jetson

cd $L4T_DIR

git clone https://github.com/veyeimaging/nvidia_jetson_veye_bsp.git

export RELEASE_PACK_DIR=$L4T_DIR/nvidia_jetson_veye_bsp

3.2.1 bsp package includes：
  • prebuild linux kernel：Image
  • prebuild dtb for different boards
  • driver source code
  • dts source code
  • i2c toolkits

3.3 Using prebuild Image and DTB

  • Installing the Kernel

Whether you use a prebuild kernel or you build it yourself

- TX2

copy Image to /boot/ dir on the Jetson TX2 board:

sudo cp <path to your Image dir>/Image /boot/Image -f

- AGX Xavier

on the Host PC:

cd $L4T_DIR

sudo ./flash.sh -k kernel jetson-xavier mmcblk0p1

  • Flashing dtb file

Enter the recovery mode，and on the Host PC:

cp <path to your dtb dir>/dtbfilename $L4T_DIR/kernel/dtb/ -f

cd $L4T_DIR

- TX2

sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1

- AGX Xavier

sudo ./flash.sh -r -k kernel-dtb jetson-xavier mmcblk0p1

Reset Jetson Board after flashing.

3.4 Building from Source

3.4.1 Build kernel
  • patch code

cp $RELEASE_PACK_DIR/drivers_source/cs_imx307\ veye327/* $NVIDIA_PATH/drivers/media/i2c/

cp $RELEASE_PACK_DIR/drivers_source/kernel_csimx307veye327_config $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/tegra_csimx307veye327_defconfig

  • build

cd $L4T_DIR/sources/kernel/kernel-4.9/

make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_csimx307veye327_defconfig

make ARCH=arm64 O=$TEGRA_KERNEL_OUT Image -j4

Completed Image file is $TEGRA_KERNEL_OUT/arch/arm64/boot/Image，could be used for Flashing or Upgrading.

Install Image for Flashing:

sudo cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image $L4T_DIR/kernel/ -f

3.4.2 Build DTS
  • patch code

- Jetson TX2

cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/common/t18x/* -r $TX2_DTS_PATH/

For CS-MIPI-IMX307

cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/CS-MIPI-IMX307/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/

For VEYE-MIPI-327

cp $RELEASE_PACK_DIR/TX2/JetPack_4.2.2_Linux_GA_P3310/dts\ dtb/VEYE-MIPI-327/tegra186-quill-p3310-1000-a00-00-base.dts $TX2_DTS_PATH/quill/kernel-dts/

- AGX Xavier

cp $RELEASE_PACK_DIR/AGX-Xaviar/JetPack_4.2.2_Linux_GA_P2888/dts\ dtb/common/t19x/* -r $XAVIER_DTS_PATH/

For CS-MIPI-IMX307

cp $RELEASE_PACK_DIR/AGX-Xaviar/JetPack_4.2.2_Linux_GA_P2888/dts\ dtb/CS-MIPI-IMX307/tegra194-p2888-0001-p2822-0000.dts $TX2_DTS_PATH/galen/kernel-dts/

For VEYE-MIPI-327

cp $RELEASE_PACK_DIR/AGX-Xaviar/JetPack_4.2.2_Linux_GA_P2888/dts\ dtb/VEYE-MIPI-327/tegra194-p2888-0001-p2822-0000.dts $TX2_DTS_PATH/galen/kernel-dts/

  • build

cd $L4T_DIR/sources/kernel/kernel-4.9/

make ARCH=arm64 O=$TEGRA_KERNEL_OUT dtbs

Could be used for Flashing or Upgrading.

4 Applications and Test

4.1 Check system status

Run the following command to confirm whether the camera is probed.

  • For VEYE-MIPI-327

dmesg | grep veye327  

The output message appears as shown below.：

Detected VEYE327 sensor

subdev veye327 3X-003b bound

此外，检查/dev/videoX设备节点，应当是存在的。

  • For CS-MIPI-IMX307

dmesg | grep csimx307  

The output message appears as shown below.：

Detected CS307 sensor

subdev csimx307 3X-003b bound

  • Run the following command to check the presence of video node.

ls /dev/video*

The output message appears as shown below.

videoX  

4.2 Video Stream Toolkits Manual

Our camera appears as /dev/videoX device node,where X is between [0-5].

in gstreamer command ，v4l2src has a param device=/dev/videoX to identify which camera to use.

4.2.1 Gstreamer Usage
  • Preview FHD(HW accelerated)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • Preview 720p@60 (CS-MIPI-IMX307,HW accelerated)

gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • Record FHD in H.264 format to a video file（HW accelerated）

gst-launch-1.0 v4l2src num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=videoname.mkv

  • Playback of saved video file (HW accelerated）

gst-launch-1.0 filesrc location=videoname.mkv ! matroskademux ! h264parse ! omxh264dec ! nvoverlaysink

  • Capturing FHD still image  

gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! filesink location=jpgname.jpg

4.3 Video Control Toolkits Manual

  • i2c bus description：

ADP-TX2-6CAM has 6 MIPI CSI-2 interface [A-F]，the corresponding i2c address is[30-35].

It is also the same as the 3X part of dmesg message:

subdev veye327 3X-003b bound

subdev csimx307 3X-003b bound

using -b option to identify which bus you want to use.

  • VEYE-MIPI-327

Video Control Toolkits Manual ：VEYE-MIPI-327 I2C

  • CS-MIPI-IMX307

Video Control Toolkits Manual ：CS-MIPI-IMX307 I2C

5 Realtimes RTSO-9001

5.1 Hardware Setup

CS-MIPI-IMX307 and RTSO-9001（TX2）


Matching with Realtimes RTSO-9001, you need to use RTST-RPI adapter board.

Note: the power supply capacity of the MIPI CSI-2 interface of the current version of RTSO-9001 is not enough to support the power requirements of our camera, which requires an additional 5V power supply to the camera. It is not shown in the picture above.

5.2 Software bsp

  • The corresponding directory for Realtimes RTSO-9001, is：

nvidia_jetson_veye_bsp\TX2\JetPack_4.2.2_Linux_RTSO-9001

  • Subject to the agreement with Ruitai, we do not release the source code
  • Our code have been submitted to Realtimes, and customers are advised to contact Realtimes for a complete version of the supported image.

6 References

Jetson start up： https://developer.nvidia.com/embedded/learn/getting-started-jetson

TX2 development kit： https://developer.nvidia.com/embedded/jetson-tx2-developer-kit

SDK Manager: https://docs.nvidia.com/sdk-manager/index.html

TX2 user guide: link

Realtimes website：http://www.realtimes.cn/

Retrieved from "http://wiki.veye.cc/index.php?title=VEYE_CS_Camera_for_Jetson_TX2&oldid=1621"