VEYE CS Camera on google Coral board

From wiki_veye
Jump to navigation Jump to search

1 Overview

The Coral Dev Board is a single-board computer made by google which use i.MX8m processer. It use Mendel linux system(4.14.98) , based on that,our camera module driver is added, and some application layer routines are provided.

In general, you only need to use the Image and dtb that we have compiled. Of course, you can also modify or recompile the driver according to our open source code.

1.1 Camera module list

Camera module dirver status list
Series Model Status Adaptor board
VEYE series VEYE-MIPI-IMX327S Done ADP-TWG
VEYE series VEYE-MIPI-IMX385 Done ADP-TWG
CS series CS-MIPI-IMX307 Done ADP-TWG
CS series CS-MIPI-SC132 Done ADP-TWG

VEYE and CS series camera modules are camera module with ISP functions build in. It output UYVY/YUYV data using MIPI-CSI2. We provide V4L2 interface for video streaming apps , and Video Control Toolkits (which is Shell Script) to control the camera module directly, which is called DRA(Directly Register Access).

Specifically, on this i.MX platform, the default driver configures the camera to YUYV format.

2 Hardware Setup

We use ADP-TWG to connnect camera to Coral Dev board. Due to insufficient power supply capacity of FFC cable, additional 5V power supply is used.

Google Coral connection overview
Google Coral connection overview(Use additional 5V power supply)
Google Coral connection details
Google Coral connection details
Google Coral connection details(2)
Google Coral connection details(2)


3 Upgrade Mendel Linux system

3.1 Overview

This section describes how to update the Coral dev board system to support our camera module. To support our camera module, we need to update two parts of the system, Image and dtb.

In the Image, we added the camera driver, while the dtb indicates the camera model used. In general, you only need to use the Image and dtb, that we have prebuilt(in deb format), and you don't need to build from source when it is not necessary.

3.2 BSP package introduction

3.2.1 i.MX platform bsp

https://github.com/veyeimaging/nxp_i.mx_veye_bsp

includes：

  • driver source code
  • i2c toolkits
  • application demo
3.2.2 google Coral Dev board bsp

https://github.com/veyeimaging/google_coral_i.mx

includes：

  • prebuild linux kernel and dtb：linux-image-4.14.98-imx_12-2_arm64.deb
  • dts source code
  • defconfig and rules file for build system

3.3 Flash the Coral standard system

Refer to:

https://coral.ai/docs/dev-board/get-started/#flash-the-board

Flash the standard system provided by google to the board.

3.4 Install MDT and connect the board via MDT

Refer to:

https://coral.ai/docs/dev-board/get-started/#install-mdt

https://coral.ai/docs/dev-board/get-started/#connect-via-mdt

3.5 Using prebuilt Image and dtb file （deb file）

On HOST PC:

git clone https://github.com/veyeimaging/google_coral_i.mx.git

  • list device

mdt devices

one coral board appears like this:

deft-kid                (192.168.0.122)

  • push deb file to coral board

cd google_coral_i.mx/linux-image-deb/

mdt push linux-image-4.14.98-imx_12-2_arm64.deb

  • Log in Coral board via mdt

mdt shell

  • Install deb file

sudo dpkg -i linux-image-4.14.98-imx_12-2_arm64.deb

Pay attention to the log information, it can be installed correctly normally.

  • Overwrite the dtb file

In coral board shell,

cd /boot/

sudo cp fsl-imx8mq-phanbell-<camera_name>.dtb fsl-imx8mq-phanbell.dtb

camera_name could be veye327,csimx307,cssc132.

  • Reboot coral dev board

4 Applications and Test

The following commands are all executed on coral board.

4.1 Check system status

Run the following command to confirm whether the camera is probed.

  • VEYE-MIPI-IMX327S

dmesg | grep veye327  

The output message appears as shown below：

camera veye327_mipi is found

Registered sensor subdevice: veye327_mipi 1-003b

  • CS-MIPI-IMX307

dmesg | grep csimx307  

The output message appears as shown below：

camera csimx307_mipi is found

Registered sensor subdevice: csimx307_mipi 1-003b

  • CS-MIPI-SC132

dmesg | grep cssc132  

The output message appears as shown below：

camera cssc132_mipi is found

Registered sensor subdevice: cssc132_mipi 1-003b

  • Run the following command to check the presence of video node.

ls /dev/video*

The output message appears as shown below.

video0

or:

v4l2-ctl --list-devices

You should see the camera listed as /dev/video0:

i.MX6S_CSI (platform:30a90000.csi1_bridge):

/dev/video0

The camera use i2c-1.

4.2 List the modes supported by the camera

v4l2-ctl --list-formats-ext -d /dev/video0

4.3 Gstreamer usage examples

4.3.1 Install gstreamer tools

sudo apt-get update

sudo apt-get install gstreamer1.0-tools gstreamer1.0-alsa gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav

If you have built the entire mendel tree, there is another way:

On Host PC:

source build/setup.sh

j product

cd packages/bsp/

mdt install ./gstreamer1.0-tools_1.14.4+imx-3_arm64.deb

4.3.2 Gstreamer test

export DISPLAY=:0

  • Preview (VEYE-MIPI-IMX327S,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1920,height=1080 ! glimagesink

  • Preview example 2 (VEYE-MIPI-IMX327S,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1920,height=1080 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v

  • Preview example 3(CS-MIPI-IMX307 @1080p mode 10fps)

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,width=1920,height=1080,framerate=(fraction)10/1' ! waylandsink

  • Preview (CS-MIPI-IMX307 @720p mode 60fps)

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1280,height=720,framerate=60/1 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v

  • Preview (CS-MIPI-SC132 @1280*1080 mode 45fps)

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1280,height=1080,framerate=45/1 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v

  • Preview (CS-MIPI-SC132 @1080*1280 mode 45fps)

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1280,height=1080,framerate=45/1 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v

  • Snap a jpeg picture (CS-MIPI-IMX307 @1080p mode)
    gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 num-buffers=1 ! video/x-raw,width=1920,height=1080 ! jpegenc ! filesink location=frame_captured_1920x1080.jpg

On host pc, you can use this cmd to pull the file to host:

mdt pull frame_captured_1920x1080.jpg .

  • Snap 3 raw yuv picture (CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 num-buffers=3 ! video/x-raw,width=1920,height=1080 ! multifilesink location=frames_captured_1920x1080_YUYV_%d.raw

  • v4l2grab snap a picture(VEYE-MIPI-327S,CS-MIPI-IMX307 @1080p mode)

./v4l2grab -d /dev/video0 -W 1920 -H 1080 -I 30 -o picture.jpg

4.4 V4l2-ctl usage example

4.4.1 install v4l2-utils

sudo apt-get install v4l-utils

4.4.2 v4l2-ctl test
  • Snap a UYVY raw data picture (1080p mode)

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv


4.5 I2C Control Toolkits Manual

Because of the high degree of freedom of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.

https://github.com/veyeimaging/nxp_i.mx_veye_bsp/tree/main/i2c_cmd

using -b option to identify which bus you want to use. (The i2c bus is 1 on coral dev board.)

  • VEYE series

Video Control Toolkits Manual ：VEYE-MIPI-327 I2C

  • CS series

Video Control Toolkits Manual ：CS-MIPI-X I2C

5 Build the drivers from source

The following operations are done on Ubuntu Host PC.

5.1 Set up the Host PC environment

Refer to this page to set up the development environment:https://coral.googlesource.com/docs/+/refs/heads/master/GettingStarted.md.

5.2 Build google standard version kernel

Complete the Build the tree steps according to the guide above.

5.3 Patch code

  • Driver source code

git clone https://github.com/veyeimaging/nxp_i.mx_veye_bsp.git

The source code path of the camera driver is: linux/drivers/media/platform/mxc/capture, replace the source code of the camera driver in the corresponding directory. The kernel version is 4.14.98 now.

Merge the Config and Makefile in the same path. Add the corresponding camera driver.

  • dts file and pakages build config file

git clone https://github.com/veyeimaging/google_coral_i.mx.git

The path of the dts file is: linux/arch/arm64/boot/dts/freescale, places the dts file in this path.

Modify the Config and Makefile in the same path. Add the corresponding dts option.

The path of packages build is :packages/linux-imx/debian/,places the defconfig and rules in this path.

5.4 Build

m docker-linux-imx

5.5 Install the new kernel

j product

cd packages/bsp/

mdt install ./linux-image-4.14.98-imx_12-2_arm64.deb

6 References

https://coral.ai/products/dev-board/

https://github.com/google-coral

https://coral.googlesource.com/docs/+/refs/heads/master/GettingStarted.md


Retrieved from "http://wiki.veye.cc/index.php?title=VEYE_CS_Camera_on_google_Coral_board&oldid=4818"