CS-MIPI-IMX307 for Jetson Nano/zh

From wiki_veye
Jump to navigation Jump to search

English

如何在英伟达Jetson Nano平台使用CS-MIPI-X摄像头模组

1 概述

本文讲述怎样将CS-MIPI系列 模组接入到英伟达公司的Jetson Nano开发板上。采用的Jetson Nano软件版本为

  • Jetpack4.2.2,L4T版本r32.2.1
  • Jetpack4.3,L4T版本r32.3.1
  • Jetpack4.4,L4T版本r32.4.3

我们提供了编译好的固件和源代码两种方式来方便用户的使用。

CS-MIPI-IMX307是星光级内置ISP的摄像头模组，输出为MIPI-CSI2接口UYVY流模式，支持多种分辨率模式。

CS-MIPI-SC132是全局曝光黑白摄像头模组，输出为MIPI-CSI2接口UYVY流模式，支持多种分辨率模式，支持外触发模式。

对于图像效果参数，我们并没有采用V4L2接口实现，而是采用了Shell脚本直接控制摄像头模组寄存器来实现。

已支持Jetson Nano B01新版本。

2 硬件准备及安装

Jetson Nano的CSI2接口可以直接与CS-MIPI-X连接。摄像头模组支持多种供电方式，Jetson Nano的FFC排线上3.3V功率足够，因此可以使用两种供电方式。供电模式切换方法：Power supply mode switching/zh。

CS-MIPI-SC132默认供电方式为FFC排线3.3V供电,无需插红黑电源线。

Jetson Nano 与CS-MIPI-IMX307模组连接
Jetson Nano 电源线接法
CS-MIPI-X接法
CS-MIPI-X dual cameras connect to Jetson Nano B01


3 更新Jetson Nano系统

3.1 概述

本章节描述怎样更新Jetson系统以支持我们的摄像头模组。要支持我们的摄像头模组，需要更新Jetson系统的L4T(Linux for Tegra)的两个部分，Image和DTB。

Image部分，我们增加了摄像头驱动，DTB部分则指明了使用的摄像头型号。一般情况下，只需使用我们已编译好的Image和DTB，非必要情况下，无需根据代码进行编译。

特别的，对于Xavier Nano,我们提供了TF卡的烧写镜像。

4 使用编译好的整体镜像更新Jetson nano系统

客户可选择直接使用我们编译好的镜像进行系统更新，以支持我司摄像头模组。

  • 镜像下载路径：

baiduyun：https://pan.baidu.com/s/1sWrcfEOivjuQ7T7P5bVcng

google driver:https://drive.google.com/drive/folders/1KHSMTdyw-ZczR1yJPj-5y9eIv7izbgt3?usp=sharing

下载后请先解压后烧写。

如百度云盘下载过慢，可以单独联系我公司客服xumm@csoneplus.com索取。

  • 烧写方法

参考官方链接

5 使用源码更新Jetson nano系统

5.1 参考资料

Jetson Nano的系统安装及使用请参考官方文档：get-started-jetson-nano-devkit. 详细的开发资料请参考官方文档：https://docs.nvidia.com/jetson/archives/

5.2 开发环境准备

  • Host PC which runs Ubuntu 18.04/16.04 (64-bit).  
  • NVIDIA® provided L4T release and corresponding sample rootfs for Jetson Nano™ development kit.  
  • A kernel image , device tree blob (DTB) file and module drivers for the CS-MIPI-IMX307 camera. The release package contains a kernel binary (Image),DTB files and module drivers, which you can download and rebuild from source.
  • A jumper pin connected across J48 button header to enable DC power.
  • A USB cable (micro USB port) to plug into the recovery port of the Jetson Nano™
  • development kit.
  • Power cable (5V-4A) to power the Jetson Nano™ board.
  • Micro SD card must be connected to the J501 slot.

5.3 Host PC开发环境准备

  • 编译工具链

请参考这个链接安装编译工具链。

  • L4T源码和Rootfs

可以通过SDK Manager或者直接下载两种方式来获得源码。具体参考官方文档，此处不再赘述。

  • 配置开发环境

以下步骤假设SDK安装目录为<TOPDIR>，源代码在$L4T_DIR/sources目录。

-Jetpack4.2.2

export TOP_DIR=<absolute path to top dir;in my case is /home/xumm/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_P3448/>

-Jetpack4.3

export TOP_DIR=<absolute path to top dir;in my case is /home/xumm/nvidia/nvidia_sdk/JetPack_4.3_Linux_JETSON_NANO_DEVKIT/>

-Jetpack4.4

export TOP_DIR=<absolute path to top dir;in my case is /home/xumm/nvidia/nvidia_sdk//JetPack_4.4_Linux_JETSON_NANO_DEVKIT/>

- 通用部分

export L4T_DIR=$TOP_DIR/Linux_for_Tegra

export LOCALVERSION=-tegra

export LDK_ROOTFS_DIR=$TOP_DIR/Linux_for_Tegra/rootfs

export ARCH=arm64

export CROSS_COMPILE=aarch64-linux-gnu-

export CROSS32CC=arm-linux-gnueabihf-gcc

mkdir -p $L4T_DIR/sources/kernel/out_kernel

export TEGRA_KERNEL_OUT=$L4T_DIR/sources/kernel/out_kernel

export NVIDIA_PATH=$L4T_DIR/sources/kernel/nvidia

export NANO_DTS_PATH=$L4T_DIR/sources/hardware/nvidia/platform/t210/

  • 生成默认版本烧写镜像，准备好L4T环境

cd $L4T_DIR

sudo ./apply_binaries.sh  

  • 下载程序

cd $L4T_DIR

git clone https://github.com/veyeimaging/nvidia_jetson_veye_bsp.git

export RELEASE_PACK_DIR=$L4T_DIR/nvidia_jetson_veye_bsp

5.4 使用编译好的Image和DTB

5.4.1 Installing the Kernel and DTS

cd $RELEASE_PACK_DIR/kernel_image

- Jetpack4.2.2

tar -xzvf Image_l4t_r32.2.1_veyecam.tar.gz

sudo cp Image $L4T_DIR/kernel/ -f

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f

- Jetpack4.3

tar -xzvf Image_l4t_r32.3.1_veyecam.tar.gz

sudo cp Image $L4T_DIR/kernel/ -f

  • For CS-MIPI-IMX307

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f

  • For CS-MIPI-SC132

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-SC132/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-SC132/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f

- Jetpack4.4

tar -xzvf Image_l4t_r32.4.3_veyecam.tar.gz

sudo cp Image $L4T_DIR/kernel/ -f

  • For CS-MIPI-IMX307

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/CS-MIPI-IMX307/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f

  • For CS-MIPI-SC132

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/CS-MIPI-SC132/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/ -f

sudo cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/CS-MIPI-SC132/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/ -f

cd $L4T_DIR

5.5 整体烧写 Flashing the Jetson Nano Development Kit

The steps to flash the Jetson Nano™ development kit are as follows:

  1. Ensure a jumper is connected across J48 button header to enable DC power.
  2. Connect the micro USB cable to the Jetson Nano™ and host PC.
  3. Set the board to recovery mode, as mentioned in below steps:

    a. Power OFF the board.

    b. For Jetson Nano A02: Connect the jumper pin to the pin 3 and pin 4 of the J40 button header. For Jetson Nano B01: Connect the jumper pin to the pin 9 and pin 10 of the J50 button header.

    c. Power ON the Jetson Nano™ development kit.

If the board is successfully changed to recovery mode, the Jetson Nano™development kit will be enumerated as an USB device to the host PC.

Run the following command to verify whether the board is in recovery mode.

lsusb

The output message appears as shown below.

Bus 001 Device 102: ID 0955:7f21 NVidia Corp.

4. Run the following commands to flash the Jetson Nano™ development kit from

your host PC.

cd $L4T_DIR

sudo ./flash.sh jetson-nano-qspi-sd mmcblk0p1

Note: Now, the entire micro SD on the Jetson Nano™ development kit will be erased. It will take about 10-30 minutes to complete depending on the host PC configuration.

5. Reboot and connect the Jetson Nano™ board to a monitor and keyboard to complete the OS configuration, once flashing is completed.

5.6 使用源码进行编译

5.6.1 编译kernel
  • patch代码

cp $RELEASE_PACK_DIR/drivers_source/cam_drv_src/* $NVIDIA_PATH/drivers/media/i2c/

- r32.2.1

cp $RELEASE_PACK_DIR/drivers_source/kernel_veyecam_config_32.2.1 $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/tegra_veyecam_defconfig

- r32.3.1

cp $RELEASE_PACK_DIR/drivers_source/kernel_veyecam_config_32.3.1 $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/tegra_veyecam_defconfig

- r32.4.3

cp $RELEASE_PACK_DIR/drivers_source/kernel_veyecam_config_32.4.3 $L4T_DIR/sources/kernel/kernel-4.9/arch/arm64/configs/tegra_veyecam_defconfig

  • 编译

cd $L4T_DIR/sources/kernel/kernel-4.9/

make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_veyecam_defconfig

make ARCH=arm64 O=$TEGRA_KERNEL_OUT Image -j4

编译完成的Image文件在 $TEGRA_KERNEL_OUT/arch/arm64/boot/Image，可以用于整体烧写，也可以用于动态升级。

以下用于整体烧写

sudo cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image $L4T_DIR/kernel/ -f

5.6.2 编译DTS
  • patch代码

- Jetpack4.2.2

cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/common/t210/* -r $NANO_DTS_PATH/

cp $RELEASE_PACK_DIR/Nano/JetPack_4.2.2_Linux_GA_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager

- Jetpack4.3

cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/common/t210/* -r $NANO_DTS_PATH/

对CS-MIPI-IMX307

cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-IMX307/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager

对CS-MIPI-SC132

cp $RELEASE_PACK_DIR/Nano/JetPack_4.3_Linux_P3448/dts\ dtb/CS-MIPI-SC132/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager

- Jetpack4.4

cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/common/t210/* -r $NANO_DTS_PATH/

对CS-MIPI-IMX307

cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/CS-MIPI-IMX307/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager

对CS-MIPI-SC132

cp $RELEASE_PACK_DIR/Nano/JetPack_4.4_Linux_JETSON_NANO_DEVKIT/dts\ dtb/CS-MIPI-SC132/tegra210-porg-plugin-manager.dtsi -r $NANO_DTS_PATH/porg/kernel-dts/porg-plugin-manager

  • 编译

make ARCH=arm64 O=$TEGRA_KERNEL_OUT dtbs

cp $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/tegra210-p3448-0000-p3449-0000-a02.dtb $L4T_DIR/kernel/dtb/

cp $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/tegra210-p3448-0000-p3449-0000-b00.dtb $L4T_DIR/kernel/dtb/

编译完成后，可以进行整体烧写或者动态升级。

5.7 动态升级Image和DTB

  • 升级Image文件

无论是使用预编译好的程序还是使用自行编译出的Image，只需在Jetson Nano板子上执行：

sudo cp <path to your Image dir>/Image /boot/Image -f

  • 烧写DTS分区

烧写流程参考4.3 整体烧写，烧写命令为：

cd $L4T_DIR

sudo ./flash.sh -r -k DTB jetson-nano-qspi-sd mmcblk0p1

烧写完成后，断电重启。

6 应用和测试

6.1 系统状态检测

完成系统安装后，在Jetson Nano板端执行以下命令来检测是否正确连接摄像头。

  • CS-MIPI-IMX307

dmesg | grep csimx307  

应当有类似如下提示：

Detected CS307 sensor

subdev csimx307 6/7/8-003b bound

此外，检查/dev/videoX设备节点，应当是存在的。

  • CS-MIPI-SC132

dmesg | grep cssc132  

应当有类似如下提示：

subdev cssc132 6/7/8-003b bound

此外，检查/dev/videoX设备节点，应当是存在的。

6.2 视频流软件包和测试

6.2.1 Gstreamer Usage
  • 视频预览1080p HD(硬件加速)

gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • 视频预览720p@60fps HD(硬件加速)

gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • 视频预览1280*1080@45fpsHD(硬件加速)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)1080,framerate=(fraction)45/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • 视频预览640*480@130fps(模组支持此分辨率时，硬件加速)

gst-launch-1.0 v4l2src ! "video/x-raw,format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)130/1" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false

  • 并列预览两路视频 1080p HD(硬件加速)

WIDTH=960

HEIGHT=540

CAPS="video/x-raw,format=(string)UYVY, width=1920, height=1080"

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=$WIDTH sink_0::height=$HEIGHT sink_1::xpos=$WIDTH sink_1::ypos=0 sink_1::width=$WIDTH sink_1::height=$HEIGHT ! nvoverlaysink v4l2src device=/dev/video0 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp. v4l2src device=/dev/video1 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp.

  • 视频录像1080p HD（硬件加速）

gst-launch-1.0 v4l2src num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=videoname.mkv

  • 录像回放（硬件加速）

gst-launch-1.0 filesrc location=videoname.mkv ! matroskademux ! h264parse ! omxh264dec ! nvoverlaysink

  • 抓拍图片1080p

gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! filesink location=jpgname.jpg

  • 抓拍图片720p

gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720" ! jpegenc ! filesink location=jpgname.jpg

6.3 视频控制软件包的使用

Jetson Nano A02使用i2c-6作为摄像头的控制总线。Jetson Nano B01使用i2c-7和i2c-8作为摄像头的控制总线。

执行i2c控制脚本时，需使用-b选项来指明控制的对应摄像头。

cd $RELEASE_PACK_DIR/i2c_cmd/bin

具体使用说明请见：I2C脚本使用说明。

7 其他说明

有任何的改进意见，欢迎您发邮件到xumm#csoneplus.com。

Retrieved from "http://wiki.veye.cc/index.php?title=CS-MIPI-IMX307_for_Jetson_Nano/zh&oldid=2845"