View source for CS-MIPI-IMX307 for Jetson Nano

[[CS-MIPI-IMX307 for Jetson Nano/zh|查看中文]]

<big><big><big><big>How to use CS-MIPI-X camera module on NVIDIA Jetson Nano</big></big></big></big>
===Overview===
This guide shows how to connect CS-MIPI-X to Jetson Nano.Jetson SDK Version is(Jetpack4.2.2,L4T r32.2.1)and(Jetpack4.3,L4Tr32.3.1).. We provide two ways to do so: '''Prebuilt Binaries''' or '''Source Code. Yes, It's Open Source!'''

CS-MIPI-IMX307 is STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2，1080p@30fps mode, 720p_crop@60fps mode and VGA@130fps. 

CS-MIPI-SC132 is global shutter camera module with ISP functions build in. It output UYVY data using MIPI-CSI2，support mulity video format and trigger mode. 

We provide '''V4L2 interface''' for video streaming apps , and  '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly.

The new version of Jetson Nano (B01) is supported.
===Hardware Setup===
Jetson Nano's CSI2 interface is compatible with CS-MIPI-X. It support 5V power or 3.3V power mode. [[Power supply mode switching|power switching]].
[[File:Cs mipi imx307 jetson nao.JPG|alt=|center|thumb|600x600px|Connect CS-MIPI-IMX307 to Jetson Nano]]
[[File:Jetson nano pwr connect.jpg|center|thumb|600x600px|Jetson Nano power wire|alt=]]
[[File:Cs mipi imx307 jetson nano cameraboard.JPG|alt=|center|thumb|600x600px|CS-MIPI-IMX307 wire]]
[[File:Jetson Nano B01 001.jpg|center|thumb|600x600px|CS-MIPI-307 dual cameras connect to Jetson Nano B01]]
<br />
===Upgrade Jetson Nano system===
====Overview====
This section describes how to upgrade the Jetson system to support our camera module. To support our camera module, we need to update the two parts of the L4T (Linux for Tegra) of the Jetson system, '''Image''' and '''DTB'''.

In the Image, we added the camera driver, while the DTB indicates the camera model used. In general, you only need to use the Image and DTB, that we have prebuilt, and you don't need to build from source  when it is not necessary.

In particular, for Jetson nano, we provide a burning image of the TF card.
===Buruing the prebuilt Image to upgrade Jetson nano system===
You can use the image prebuild by us to upgrade jetson nano system, which will support our camera.

Using whole prebuilt image

*download ：

link：https://pan.baidu.com/s/1sWrcfEOivjuQ7T7P5bVcng

If Baidu cloud disk download is too slow, you can contact our company customer service ''xumm@csoneplus.com'' to obtain.

*How to burn

refer to [https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit Official document]
===Using source code to upgrade Jetson nano system===
====Reference documents====
Jetson Nano system setup, please refer to official document：[https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit get-started-jetson-nano-devkit].

Development Docs：https://docs.nvidia.com/jetson/archives/
====Setting Up the Environment====

*Host PC which runs Ubuntu 18.04/16.04(64-bit).  

*NVIDIA® provided L4T release and corresponding sample rootfs for Jetson Nano™ development kit.  

*A kernel image , device tree blob (DTB) file and module drivers for the CS-MIPI-IMX307 camera. The release package contains a kernel binary (Image),DTB files and module drivers, which you can download and rebuild from source.

*A jumper pin connected across J48 button header to enable DC power.
*A USB cable (micro USB port) to plug into the recovery port of the Jetson Nano™
*development kit.
*Power cable (5V-4A) to power the Jetson Nano™ board.
*Micro SD card must be connected to the J501 slot.

====Host PC environment prepare====

*Cross-compiling Toolchain

Please refer to [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fxavier_toolchain.html%23wwpID0ESHA this link] to install toolchain on your Host PC.

*L4T source code and Rootfs

You could use SDK Manager or [https://developer.nvidia.com/embedded/downloads directly download] to get source code.

*Setting Up the Environment

Assume SDK install directory is <TOPDIR>，source code is in $L4T_DIR/sources directory.

''-Jetpack4.2.2''

<code>export TOP_DIR=<absolute path to top dir;in my case is /home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3448/></code>

''-Jetpack4.3''

<code>export TOP_DIR=<absolute path to top dir;in my case is /home/xumm/nvidia/nvidia_sdk/JetPack_4.3_Linux_JETSON_NANO_DEVKIT/></code>

''- Common Part''

<code>export L4T_DIR=$TOP_DIR/Linux_for_Tegra</code>

<code>export LOCALVERSION=-tegra</code>

<code>export LDK_ROOTFS_DIR=$TOP_DIR/Linux_for_Tegra/rootfs</code>

<code>export ARCH=arm64</code>

<code>export CROSS_COMPILE=aarch64-linux-gnu-</code>

<code>export CROSS32CC=arm-linux-gnueabihf-gcc</code>

<code>mkdir -p $L4T_DIR/sources/kernel/out_kernel</code>

<code>export TEGRA_KERNEL_OUT=$L4T_DIR/sources/kernel/out_kernel</code>

<code>export NVIDIA_PATH=$L4T_DIR/sources/kernel/nvidia</code>

<code>export NANO_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t210/porg/kernel-dts</code>

*Build default image, prepare L4T environment

<code>cd $L4T_DIR</code>

<code>sudo ./apply_binaries.sh</code>  

*Download source code for Jetson Nano

<code>cd $L4T_DIR</code>

<code>git clone <nowiki>https://github.com/veyeimaging/nvidia_jetson_veye_bsp.git</nowiki></code>

<code>export RELEASE_PACK_DIR=$L4T_DIR/nvidia_jetson_veye_bsp</code>

====Using prebuild Image and DTB====
=====Installing the Kernel and DTS=====
- ''Jetpack4.2.2''

<code>tar -xzvf Image_l4t_r32.2.1_veyecam.tar.gz</code>

<code>sudo cp Image $L4T_DIR/kernel/ -f</code>

<code>sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f</code>

<code>sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f</code>

- ''Jetpack4.3''

<code>tar -xzvf Image_l4t_r32.3.1_veyecam.tar.gz</code>

<code>sudo cp Image $L4T_DIR/kernel/ -f</code>

*CS-MIPI-IMX307

<code>sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f</code>

<code>sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f</code>

*CS-MIPI-SC132

<code>sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-SC132/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f</code>

<code>sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-SC132/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f</code>

<code>cd $L4T_DIR</code>

====Flashing the Jetson Nano Development Kit====
The steps to flash the Jetson Nano™ development kit are as follows:

#Ensure a jumper is connected across J48 button header to enable DC power.
#Connect the micro USB cable to the Jetson Nano™ and host PC.
#Set the board to recovery mode, as mentioned in below steps:

           a. Power OFF the board.

           b. Connect the jumper pin to the pin 3 and pin 4 of the J40 button header.

           c. Power ON the Jetson Nano™ development kit.

If the board is successfully changed to recovery mode, the Jetson Nano™development kit will be enumerated as an USB device to the host PC.

Run the following command to verify whether the board is in recovery mode.

<code>lsusb</code>

The output message appears as shown below.

<code>Bus 001 Device 102: ID 0955:7f21 NVidia Corp.</code>

4. Run the following commands to flash the Jetson Nano™ development kit from

your host PC.

<code>cd $L4T_DIR</code>

<code>sudo ./flash.sh jetson-nano-qspi-sd mmcblk0p1</code>

Note: Now, the entire micro SD on the Jetson Nano™ development kit will be erased. It will take about 10-30 minutes to complete depending on the host PC configuration.

5. Reboot and connect the Jetson Nano™ board to a monitor and keyboard to complete the OS configuration, once flashing is completed.
====Building from Source====
=====Build kernel=====

*patch code

<code>cp $RELEASE_PACK_DIR/drivers_source/cam_drv_src/* $NVIDIA_PATH/drivers/media/i2c/</code>

''- r32.2.1''

<code>cp $RELEASE_PACK_DIR/drivers_source/kernel_veyecam_config_32.2.1 $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/tegra_veyecam_defconfig</code>

''- r32.3.1''

<code>cp $RELEASE_PACK_DIR/drivers_source/kernel_veyecam_config_32.3.1 $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/tegra_veyecam_defconfig</code>

*build

<code>cd $L4T_DIR/sources/kernel/kernel-4.9/</code>

<code>make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_veyecam_defconfig</code>

<code>make ARCH=arm64 O=$TEGRA_KERNEL_OUT Image -j4</code>

Completed Image file is $TEGRA_KERNEL_OUT/arch/arm64/boot/Image，could be used for Flashing or Upgrading.

Install Image for Flashing:

<code>sudo cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image $L4T_DIR/kernel/ -f</code>
=====Build DTS=====

*patch code

''- Jetpack4.2.2''

<code>cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/common/t210/* -r $NANO_DTS_PATH/</code>

<code>cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager</code>

''- Jetpack4.3''

<code>cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/common/t210/* -r $NANO_DTS_PATH/</code>

*CS-MIPI-IMX307

<code>cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager</code>

*CS-MIPI-SC132

<code>cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-SC132/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager</code>

*build

<code>make ARCH=arm64 O=$TEGRA_KERNEL_OUT dtbs</code>

<code>cp $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/</code>

<code>cp $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/</code>

Could be used for Flashing or Upgrading.
====Upgrading Kernel Image and Supplements====
First，you should copy '''Image''' file to Jetson nano somehow. Use a U disk for example.

<code>sudo cp <path to your Image dir>/Image  /boot/Image -f</code>
====Upgrading DTB File by Flashing from Host PC====
Burning process reference 4.3, burning commands is as below:

<code>cd $L4T_DIR</code>

<code>sudo ./flash.sh -r -k DTB jetson-nano-qspi-sd mmcblk0p1</code>

Power off after flashing finished.
===Applications and Test===
====Check system status====
The steps to verify the setup before testing Gstreamer pipelines are as follows:

1. Run the following commands to check the Gstreamer-1.0 version.

<code>$ gst-inspect-1.0 --version</code>

<code>gst-inspect-1.0 version 1.14.4</code>

<code>GStreamer 1.14.4</code>

Note: Make sure that CS-MIPI-X is connected and the required driversare loaded.

During booting, the module drivers for CS-MIPI-X will be loaded automatically in the Jetson Nano™ development kit.

2. Run the following command to confirm whether the camera is initialized.

*For CS-MIPI-IMX307

<code>dmesg | grep csx307</code>  

The output message appears as shown below.：

<code>Detected CS307 sensor</code>

<code>subdev csimx307 X-003b bound</code>

*For CS-MIPI-SC132

<code>dmesg | grep cssc132</code>  

The output message appears as shown below.：

<code>subdev cssc132 X-003b bound</code>

*Run the following command to check the presence of video node.

<code>ls /dev/video*</code>

The output message appears as shown below.

<code>video0(1)</code>  
====Video Stream Toolkits Manual====
=====Gstreamer Usage=====

*Preview FHD(HW accelerated)

<code>gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>

*Preview 720p@60fps HD(HW accelerated)

<code>gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>

*Preview 1280*1080@45 (CS-MIPI-SC132,HW accelerated)

<code>gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)1080, framerate=(fraction)45/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>

*Preview 640*480@130fps(HW accelerated)

<code>gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)130/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false</code>

*Preview 2 cameras  1080p HD(HW accelerated)

<code>WIDTH=960</code>

<code>HEIGHT=540</code>

<code>CAPS="video/x-raw,format=(string)UYVY, width=1920, height=1080"</code>

<code>gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=$WIDTH sink_0::height=$HEIGHT sink_1::xpos=$WIDTH sink_1::ypos=0 sink_1::width=$WIDTH sink_1::height=$HEIGHT ! nvoverlaysink v4l2src device=/dev/video0 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp. v4l2src device=/dev/video1 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp.</code>

*Record FHD in H.264 format to a video file（HW accelerated）

<code>gst-launch-1.0 v4l2src num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=videoname.mkv</code>

*Playback of saved video file (HW accelerated）

<code>gst-launch-1.0 filesrc location=videoname.mkv ! matroskademux ! h264parse ! omxh264dec ! nvoverlaysink</code>

*Capturing FHD still image  

<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! filesink location=jpgname.jpg</code>

*Capturing 720p still image  

<code>gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720" ! jpegenc ! filesink location=jpgname.jpg</code>
====Video Control Toolkits Manual====
Jetson Nano A02 use i2c-6 as camera control bus,Jetson Nano B01 use i2c-7 and i2c-8 as camera control bus.

Using -b option to identify which bus to use.

<code>cd $RELEASE_PACK_DIR/i2c_cmd/bin</code>

Video Control Toolkits Manual : [[CS-MIPI-X i2c|CS-MIPI-X i2c]]
===Others===
This article and the source code are still in the process of improving. If you have any suggestions for improvement, you are welcome to email xumm#csoneplus.com.

<br />