VEYE CS Camera on google Coral board
1 Overview
The Coral Dev Board is a single-board computer made by google which use i.MX8m processer. It use Mendel linux system(4.14.98) , based on that,our camera module driver is added, and some application layer routines are provided.
In general, you only need to use the Image and dtb that we have compiled. Of course, you can also modify or recompile the driver according to our open source code.
1.1 Camera module list
| Series | Model | Status | Adaptor board |
|---|---|---|---|
| VEYE series | VEYE-MIPI-IMX327S | Done | ADP-TWG |
| VEYE series | VEYE-MIPI-IMX385 | Done | ADP-TWG |
| CS series | CS-MIPI-IMX307 | Done | ADP-TWG |
| CS series | CS-MIPI-SC132 | Done | ADP-TWG |
VEYE and CS series camera modules are camera module with ISP functions build in. It output UYVY/YUYV data using MIPI-CSI2. We provide V4L2 interface for video streaming apps , and Video Control Toolkits (which is Shell Script) to control the camera module directly, which is called DRA(Directly Register Access).
Specifically, on this i.MX platform, the default driver configures the camera to YUYV format.
2 Hardware Setup
We use ADP-TWG to connnect camera to Coral Dev board. Due to insufficient power supply capacity of FFC cable, additional 5V power supply is used.
.jpg)
3 Upgrade Mendel Linux system
3.1 Overview
This section describes how to update the Coral dev board system to support our camera module. To support our camera module, we need to update two parts of the system, Image and dtb.
In the Image, we added the camera driver, while the dtb indicates the camera model used. In general, you only need to use the Image and dtb, that we have prebuilt(in deb format), and you don't need to build from source when it is not necessary.
3.2 BSP package introduction
3.2.1 i.MX platform bsp
https://github.com/veyeimaging/nxp_i.mx_veye_bsp
includes：
- driver source code
- i2c toolkits
- application demo
3.2.2 google Coral Dev board bsp
https://github.com/veyeimaging/google_coral_i.mx
includes：
- prebuild linux kernel and dtb：linux-image-4.14.98-imx_12-2_arm64.deb
- dts source code
- defconfig and rules file for build system
3.3 Flash the Coral standard system
Refer to:
https://coral.ai/docs/dev-board/get-started/#flash-the-board
Flash the standard system provided by google to the board.
3.4 Install MDT and connect the board via MDT
Refer to:
https://coral.ai/docs/dev-board/get-started/#install-mdt
https://coral.ai/docs/dev-board/get-started/#connect-via-mdt
3.5 Using prebuilt Image and dtb file （deb file）
On HOST PC:
git clone https://github.com/veyeimaging/google_coral_i.mx.git
- list device
mdt devices
one coral board appears like this:
deft-kid (192.168.0.122)
- push deb file to coral board
cd google_coral_i.mx/linux-image-deb/
mdt push linux-image-4.14.98-imx_12-2_arm64.deb
- Log in Coral board via mdt
mdt shell
- Install deb file
sudo dpkg -i linux-image-4.14.98-imx_12-2_arm64.deb
Pay attention to the log information, it can be installed correctly normally.
- Overwrite the dtb file
In coral board shell,
cd /boot/
sudo cp fsl-imx8mq-phanbell-<camera_name>.dtb fsl-imx8mq-phanbell.dtb
camera_name could be veye327,csimx307,cssc132.
- Reboot coral dev board
4 Applications and Test
The following commands are all executed on coral board.
4.1 Check system status
Run the following command to confirm whether the camera is probed.
- VEYE-MIPI-IMX327S
dmesg | grep veye327
The output message appears as shown below：
camera veye327_mipi is found
Registered sensor subdevice: veye327_mipi 1-003b
- CS-MIPI-IMX307
dmesg | grep csimx307
The output message appears as shown below：
camera csimx307_mipi is found
Registered sensor subdevice: csimx307_mipi 1-003b
- CS-MIPI-SC132
dmesg | grep cssc132
The output message appears as shown below：
camera cssc132_mipi is found
Registered sensor subdevice: cssc132_mipi 1-003b
- Run the following command to check the presence of video node.
ls /dev/video*
The output message appears as shown below.
video0
or:
v4l2-ctl --list-devices
You should see the camera listed as /dev/video0:
i.MX6S_CSI (platform:30a90000.csi1_bridge):
/dev/video0
The camera use i2c-1.
4.2 List the modes supported by the camera
v4l2-ctl --list-formats-ext -d /dev/video0
4.3 Gstreamer usage examples
4.3.1 Install gstreamer tools
sudo apt-get update
sudo apt-get install gstreamer1.0-tools gstreamer1.0-alsa gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav
If you have built the entire mendel tree, there is another way:
On Host PC:
source build/setup.sh
j product
cd packages/bsp/
mdt install ./gstreamer1.0-tools_1.14.4+imx-3_arm64.deb
4.3.2 Gstreamer test
export DISPLAY=:0
- Preview (VEYE-MIPI-IMX327S,CS-MIPI-IMX307 @1080p mode)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1920,height=1080 ! glimagesink
- Preview example 2 (VEYE-MIPI-IMX327S,CS-MIPI-IMX307 @1080p mode)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1920,height=1080 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v
- Preview example 3(CS-MIPI-IMX307 @1080p mode 10fps)
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,width=1920,height=1080,framerate=(fraction)10/1' ! waylandsink
- Preview (CS-MIPI-IMX307 @720p mode 60fps)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1280,height=720,framerate=60/1 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v
- Preview (CS-MIPI-SC132 @1280*1080 mode 45fps)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1280,height=1080,framerate=45/1 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v
- Preview (CS-MIPI-SC132 @1080*1280 mode 45fps)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=0 ! video/x-raw,width=1280,height=1080,framerate=45/1 ! fpsdisplaysink videosink=glimagesink sync=false text-overlay=false -v
- Snap a jpeg picture (CS-MIPI-IMX307 @1080p mode)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 num-buffers=1 ! video/x-raw,width=1920,height=1080 ! jpegenc ! filesink location=frame_captured_1920x1080.jpg
On host pc, you can use this cmd to pull the file to host:
mdt pull frame_captured_1920x1080.jpg .
- Snap 3 raw yuv picture (CS-MIPI-IMX307 @1080p mode)
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 num-buffers=3 ! video/x-raw,width=1920,height=1080 ! multifilesink location=frames_captured_1920x1080_YUYV_%d.raw
- v4l2grab snap a picture(VEYE-MIPI-327S,CS-MIPI-IMX307 @1080p mode)
./v4l2grab -d /dev/video0 -W 1920 -H 1080 -I 30 -o picture.jpg
4.4 I2C Control Toolkits Manual
Because of the high degree of freedom of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.
https://github.com/veyeimaging/nxp_i.mx_veye_bsp/tree/main/i2c_cmd
using -b option to identify which bus you want to use. (The i2c bus is 1 on coral dev board.)
- VEYE series
Video Control Toolkits Manual ：VEYE-MIPI-327 I2C
- CS series
Video Control Toolkits Manual ：CS-MIPI-X I2C
5 Build the drivers from source
The following operations are done on Ubuntu Host PC.
5.1 Set up the Host PC environment
Refer to this page to set up the development environment:https://coral.googlesource.com/docs/+/refs/heads/master/GettingStarted.md.
5.2 Build google standard version kernel
Complete the Build the tree steps according to the guide above.
5.3 Patch code
- Driver source code
git clone https://github.com/veyeimaging/nxp_i.mx_veye_bsp.git
The source code path of the camera driver is: linux/drivers/media/platform/mxc/capture, replace the source code of the camera driver in the corresponding directory. The kernel version is 4.14.98 now.
Merge the Config and Makefile in the same path. Add the corresponding camera driver.
- dts file and pakages build config file
git clone https://github.com/veyeimaging/google_coral_i.mx.git
The path of the dts file is: linux/arch/arm64/boot/dts/freescale, places the dts file in this path.
Modify the Config and Makefile in the same path. Add the corresponding dts option.
The path of packages build is :packages/linux-imx/debian/,places the defconfig and rules in this path.
5.4 Build
m docker-linux-imx
5.5 Install the new kernel
j product
cd packages/bsp/
mdt install ./linux-image-4.14.98-imx_12-2_arm64.deb
6 References
https://coral.ai/products/dev-board/
https://github.com/google-coral
https://coral.googlesource.com/docs/+/refs/heads/master/GettingStarted.md