VEYE MIPI 327 for Nano Pi 4 ubuntu

From wiki_veye
Jump to navigation Jump to search


How to use VEYE-MIPI-327 camera module on FriendlyARM NanoPi 4(RK3399)

1 Overview

This guide shows how to use VEYE-MIPI-290/327 on Friendlyarm Nanopi4(NanoPi M4,NanoPi NEO4,NanoPi T4),we use NanoPi M4 as an example.

RK3399 is Rockchip opensource platform. We use linuxsdk-friendlyelec-20190903 version SDK,and micro SD card version.

2 Hardware Setup

The system installation is shown as follows in the figure. We have Nanopi adapter board: VEYE-MIPI-327RK datasheet.

Connect VEYE-MIPI-327 to NanoPi M4
NanoPi M4 FFC cable details
VEYE-MIPI-327 adatpter board details

Note:NanoPi M4 board MIPI-CSI2 interface description

Interface I2C DPHY facing Reuse
MIPI-CSI1 1 RX0 back NA
MIPI-CSI2 2 RX1 front TX1

3 Using prebuilt Image to upgrade NanoPi4

  • download

You can use the image prebuild by us to upgrade nanopi system, which will support our camera.

The image is based on friendlyelec-20190903 SD card version. Download here.

  • Boot from SD Card

Extract these files(rk3399-sd-friendlydesktop-bionic-4.4-arm64-20200112.img). Insert an SD card(at least 16G) into a Windows PC and run the win32diskimager utility as administrator. On the utility's main window select your SD card's drive, the wanted image file and click on "write" to start flashing the SD card.

You can refer to Link.

4 Using source code to upgrade NanoPi4

This section provides instructions on how to update the system firmware using source code. It mainly involves two parts, kernel and dtb. Customers can refer to this section to port the driver of VEYE-MIPI-327 to the motherboard of other RK3399.

4.1 Setting Up the Environment

  • Install Cross Compiler

Please refer to link by friendlyarm to install cross complier.

  • Download kernel source code

There are three way to get kernel source code.

  1. linuxsdk-friendlyelec sdk provided by friendlyarm,Link:
  2. from friendlyarm's github.
  3. from rockchip's github.

We use the first way,download linuxsdk-friendlyelec sdk,and sync it.

tar -xvf linuxsdk-friendlyelec-20190903.tar

.repo/repo/repo sync -l

  • download bsp provided by us

The github link is :

git clone

4.2 Using prebuild Image and DTB

We provide NanoPi M4 resource.img and kernel.img.

  • Download sd-fuse_rk3399

git clone

  • Download friendlydesktop-arm64-images.tgz

Link:,find the corresponding file in the images-for-eflasher directory and download it.

tar -xzvf friendlydesktop-arm64-images.tgz -C sd-fuse_rk3399

  • Replace files and Flashing

tar -xzvf  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/kernel.img.tar.gz -C ./sd-fuse_rk3399/friendlydesktop-arm64/

tar -xzvf  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/resource.img.tar.gz -C ./sd-fuse_rk3399/friendlydesktop-arm64/  

cp  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/update_partmap.txt  ./sd-fuse_rk3399/friendlydesktop-arm64/

Insert an SD card(at least 16G) into a Windows PC. Usingthe following command, you can only upgrade kernel and dtb, instead of flashing the whole card, will be faster.

cd ./sd-fuse_rk3399/

sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt

For more information on how to use sd-fuse_rk3399, please see sd-fuse_rk3399 Readme.

4.3 Building from Source code

  • copyour driver files in the corresponding kernel directory

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/VEYE-MIPI-327/veye327.c ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Makefile ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Kconfig ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/

  • Add compilation options

cd ~/linuxsdk-friendlyelec/kernel

make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 nanopi4_linux_defconfig

make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 menuconfig


make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 nanopi4-images

cp kernel.img resource.img ~/sd-fuse_rk3399/friendlydesktop-arm64/

4.3.1 Flashing SD card

Insert an SD card(at least 16G) into a Windows PC. Usingthe following command, you can only upgrade kernel and dtb.

cd ./sd-fuse_rk3399/

sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt

5 Applications and Test

5.1 Check system status

Run the following command to confirm whether the camera is initialized.

dmesg | grep veye327  

The output message appears as shown below:

Detected VEYE06 sensor

5.2 Video Stream Toolkits Manual

5.2.1 Gstreamer Usage using

Since the officially provided by friendlyarm has a default resolution of 720p, which is not supported by VEYE-MIPI-327, we have provided a modified version ——

In this directory:


  • Preview FHD

./ -a preview

  • Record FHD in H.264 format to a video file

./ --action video -output 1.ts

  • Capturing FHD still image  

./ -a photo -o 1.jpg using gst-launch-1.0
  • Preview FHD

gst-launch-1.0 rkisp device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! rkximagesink

  • Record FHD in H.264 format to a video file

gst-launch-1.0 rkisp num-buffers=512 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! queue ! h264parse ! filesink location=/tmp/out.mp4 -e

  • Capturing FHD still image  

gst-launch-1.0 rkisp num-buffers=2 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=10/1 ! jpegenc ! multifilesink location="/tmp/isp-frame%d.jpg" -e

5.3 Video Control Toolkits Manual

We use DRA (Directly Register Access) mode to configure the camera module register. And a shell script is provided to help you use it.


  • detect camera link status

i2cdetect [value]

value description

Check whether the hardware connection is normal. The product i2C address of the VEYE327 camera module is 0x3b.

  • After confirming that the hardware connection is working, the script command is as follows,-b indicates i2c bus:

Video Control Toolkits Manual : VEYE-MIPI-290/327 i2c

6 Exsiting bugs

6.1 Bug description

  • Can not encode to H.264 stream at full framerate

Using the gstreamer instructions used in the previous section to encode the video, the full frame cannot be reached. It can only reach about 17 frames. Limiting all CPU speeds to 600MHz which gives a stable 25FPS in PAL mode.

  • Memory leak

Using rkisp as a gstreamer plug-in, there will be serious memory leaks.

6.2 Bug Analysis

Although rockchip's rk3399 supports camera module of YUV format, there are serious bugs in the gstreamer plug-in they provide.

7 Refernces

7.1 From Rockchip




7.2 From Friendlyarm



Download Center