Difference between revisions of "VEYE MIPI 327 for Nano Pi 4 ubuntu"

From wiki_veye
Jump to navigation Jump to search
 
(4 intermediate revisions by one other user not shown)
Line 7: Line 7:
 
RK3399 is Rockchip opensource platform. We use linuxsdk-friendlyelec-20190903 version SDK,and micro SD card version.
 
RK3399 is Rockchip opensource platform. We use linuxsdk-friendlyelec-20190903 version SDK,and micro SD card version.
 
===Hardware Setup===
 
===Hardware Setup===
The system installation is shown as follows in the figure. We have Nanopi adapter board:'''[[VEYE-MIPI-290RK/VEYE-MIPI-327RK Data Sheet|VEYE-MIPI-327RK datasheet]].'''[[File:Veye nano pi OverAll.jpg|center|thumb|602x602px|Connect VEYE-MIPI-327 to NanoPi M4|link=http://wiki.veye.cc/index.php/File:Veye_nano_pi_OverAll.jpg]][[File:VEYEWiring scheme00 副本12.jpg|center|thumb|602x602px|NanoPi M4 FFC cable details|link=http://wiki.veye.cc/index.php/File:VEYEWiring_scheme00_%E5%89%AF%E6%9C%AC12.jpg]][[File:Veye nano piWiring scheme01.jpg|center|thumb|596x596px|VEYE-MIPI-327 adatpter board details|link=http://wiki.veye.cc/index.php/File:Veye_nano_piWiring_scheme01.jpg]]
+
The system installation is shown as follows in the figure. We have Nanopi adapter board:
 +
'''[[VEYE-MIPI-290RK/VEYE-MIPI-327RK Data Sheet|VEYE-MIPI-327RK datasheet]].'''
 +
[[File:Veye nano pi OverAll.jpg|center|thumb|602x602px|Connect VEYE-MIPI-327 to NanoPi M4]]
 +
[[File:VEYEWiring scheme00 副本12.jpg|center|thumb|602x602px|NanoPi M4 FFC cable details]]
 +
[[File:Veye nano piWiring scheme01.jpg|center|thumb|596x596px|VEYE-MIPI-327 adatpter board details]]
  
  
Line 30: Line 34:
 
|TX1<br />
 
|TX1<br />
 
|}
 
|}
===使用编译好的整体镜像更新NanoPi4系统===
 
  
*下载系统镜像文件
+
===Using prebuilt Image to upgrade NanoPi4===
  
我们在friendlyelec-20190903版本的SD卡烧写镜像基础上,制作了支持VEYE摄像头模组的镜像。点击[https://pan.baidu.com/s/1sWrcfEOivjuQ7T7P5bVcng#list/path=%2Fveye_mipi%E8%B5%84%E6%96%99%2Fnanopim4image 链接]下载。
+
*download
  
*烧录镜像文件
+
You can use the image prebuild by us to upgrade nanopi system, which will support our camera.
  
利用SD Card Formatter软件将micro SD卡格式化(格式化前有必要将需要的文件进行备份),将下载后的压缩文件解压为rk3399-sd-friendlydesktop-bionic-4.4-arm64-20200112.img。
+
The image is based on friendlyelec-20190903 SD card version. Download [https://pan.baidu.com/s/1sWrcfEOivjuQ7T7P5bVcng#list/path=%2Fveye_mipi%E8%B5%84%E6%96%99%2Fnanopim4image here.]
  
使用win32diskimager软件将解压后的镜像文件写入micro SD卡。
+
*Boot from SD Card
  
具体烧写过程和方法,可以参考[http://wiki.friendlyarm.com/wiki/index.php/NanoPi_M4/zh#.E5.87.86.E5.A4.87.E5.B7.A5.E4.BD.9C 链接]。
+
Extract these files(rk3399-sd-friendlydesktop-bionic-4.4-arm64-20200112.img). Insert an SD card(at least 16G) into a Windows PC and run the win32diskimager utility as administrator. On the utility's main window select your SD card's drive, the wanted image file and click on "write" to start flashing the SD card.
===使用源码更新NanoPi4系统===
 
本章节为如何使用源码更新系统固件的说明。主要涉及两部分,kernel和dtb。客户可参考此部分章节将VEYE-MIPI-327的驱动移植到其他RK3399的主板上。
 
====开发环境准备====
 
  
*安装编译工具链
+
You can refer to [http://wiki.friendlyarm.com/wiki/index.php/NanoPi_M4#Boot_from_SD_Card Link].
 +
===Using source code to upgrade NanoPi4===
 +
This section provides instructions on how to update the system firmware using source code. It mainly involves two parts, kernel and dtb. Customers can refer to this section to port the driver of VEYE-MIPI-327 to the motherboard of other RK3399.
 +
====Setting Up the Environment====
  
RK3399采用的是标准linaro编译工具链。参考[http://wiki.friendlyarm.com/wiki/index.php/NanoPi_M4/zh#.E5.A6.82.E4.BD.95.E7.BC.96.E8.AF.91.E7.B3.BB.E7.BB.9F 链接]安装编译工具链。也可以参考Rockchip官方资料安装编译工具链。
+
*Install Cross Compiler
  
*下载kernel源码
+
Please refer to [http://wiki.friendlyarm.com/wiki/index.php/NanoPi_M4#Boot_from_SD_Card link by friendlyarm] to install cross complier.
  
有三种方案进行kernel源码的获取。
+
*Download kernel source code
  
#使用friendlyarm提供的linuxsdk-friendlyelec完整开发包。下载链接:http://download.friendlyarm.com/NanoPiM4<nowiki/>。
+
There are three way to get kernel source code.
#从friendlyarm的官方github获取。
 
#从rockchip的官方github获取。
 
  
我们采用第一种方案。
+
#linuxsdk-friendlyelec sdk provided by friendlyarm,Link:[http://download.friendlyarm.com/NanoPiM4 http://download.friendlyarm.com/N]<nowiki/>[http://download.friendlyarm.com/NanoPiM4 anoPiM4]。
 +
#from friendlyarm's github.
 +
#from rockchip's github.
 +
 
 +
We use the first way,download linuxsdk-friendlyelec sdk,and sync it.
  
 
<code>tar -xvf linuxsdk-friendlyelec-20190903.tar</code>
 
<code>tar -xvf linuxsdk-friendlyelec-20190903.tar</code>
Line 65: Line 70:
 
<code>.repo/repo/repo sync -l</code>
 
<code>.repo/repo/repo sync -l</code>
  
*下载我们提供的驱动
+
*download bsp provided by us
 +
 
 +
The github link is :https://github.com/veyeimaging/rk3399_veye_dimaging_bsp
 +
 
  
从我们提供的github地址下载源码,下载链接:https://github.com/veyeimaging/rk3399_veye_dimaging_bsp<nowiki/>。
 
  
 
<code>git clone [https://github.com/veyeimaging/rk3399_veye_dimaging_bsp https://github.com/veyeimaging/rk3399_veye_dimaging_bsp.git]</code>
 
<code>git clone [https://github.com/veyeimaging/rk3399_veye_dimaging_bsp https://github.com/veyeimaging/rk3399_veye_dimaging_bsp.git]</code>
====使用编译好的Image和DTB====
+
====Using prebuild Image and DTB====
我们提供了NanoPi M4主板的resource.img和kernel.img,可以分区烧写提高速度。sd-fuse_rk3399详细方法请参考Readme。
+
We provide NanoPi M4 resource.img and kernel.img.  
  
使用方法如下:
+
*Download sd-fuse_rk3399
  
*下载sd-fuse_rk3399
+
<code>git clone https://github.com/friendlyarm/sd-fuse_rk3399<nowiki/>.git</code>
  
<code>git clone https://github.com/friendlyarm/sd-fuse_rk3399<nowiki/>.git</code>
+
*Download friendlydesktop-arm64-images.tgz
 +
 
 +
Link:http://download.friendlyarm.com/NanoPiM4<nowiki/>,find the corresponding file in the images-for-eflasher directory and download it.
  
*下载friendlydesktop-arm64-images.tgz
 
  
下载链接:http://download.friendlyarm.com/NanoPiM4<nowiki/>下,images-for-eflasher目录找到对应文件,下载。
 
  
 
<code>tar -xzvf friendlydesktop-arm64-images.tgz -C sd-fuse_rk3399</code>
 
<code>tar -xzvf friendlydesktop-arm64-images.tgz -C sd-fuse_rk3399</code>
  
*替换文件和动态烧写
+
*Replace files and Flashing
  
 
<code>tar -xzvf  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/kernel.img.tar.gz -C ./sd-fuse_rk3399/friendlydesktop-arm64/</code>
 
<code>tar -xzvf  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/kernel.img.tar.gz -C ./sd-fuse_rk3399/friendlydesktop-arm64/</code>
Line 93: Line 100:
 
<code>cp  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/update_partmap.txt  ./sd-fuse_rk3399/friendlydesktop-arm64/</code>
 
<code>cp  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/update_partmap.txt  ./sd-fuse_rk3399/friendlydesktop-arm64/</code>
  
将micro SD卡通过TF卡转USB工具,插到开发Host PC上。通过以下命令,可以只进行升级,而不是整个卡进行烧写,速度比较快。
+
Insert an SD card(at least 16G) into a Windows PC. Usingthe following command, you can only upgrade kernel and dtb, instead of flashing the whole card, will be faster.
  
 
<code>cd ./sd-fuse_rk3399/</code>
 
<code>cd ./sd-fuse_rk3399/</code>
  
 
<code>sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt</code>
 
<code>sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt</code>
====使用源码进行编译====
 
  
*将我们的驱动文件放到对应kernel目录下
+
For more information on how to use sd-fuse_rk3399, please see sd-fuse_rk3399 Readme.
 +
====Building from Source code====
 +
 
 +
*copyour driver files in the corresponding kernel directory
  
 
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/VEYE-MIPI-327/veye327.c ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/</code>
 
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/VEYE-MIPI-327/veye327.c ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/</code>
Line 108: Line 117:
 
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Kconfig ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/</code>
 
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Kconfig ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/</code>
  
*增加编译选项
+
*Add compilation options
  
 
cd <code>~/linuxsdk-friendlyelec/kernel</code>
 
cd <code>~/linuxsdk-friendlyelec/kernel</code>
Line 116: Line 125:
 
<code>make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 menuconfig</code>
 
<code>make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 menuconfig</code>
  
增加CONFIG_VIDEO_VEYE327选项。
+
Add CONFIG_VIDEO_VEYE327 option
  
 
<code>make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 nanopi4-images</code>
 
<code>make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 nanopi4-images</code>
  
 
<code>cp kernel.img resource.img ~/sd-fuse_rk3399/friendlydesktop-arm64/</code>
 
<code>cp kernel.img resource.img ~/sd-fuse_rk3399/friendlydesktop-arm64/</code>
====动态升级====
+
=====Flashing SD card=====
将micro SD卡通过TF卡转USB工具,插到开发Host PC上。通过以下命令,可以只进行升级,而不是整个卡进行烧写,速度比较快。
+
Insert an SD card(at least 16G) into a Windows PC. Usingthe following command, you can only upgrade kernel and dtb.
  
 
<code>cd ./sd-fuse_rk3399/</code>
 
<code>cd ./sd-fuse_rk3399/</code>
  
 
<code>sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt</code>
 
<code>sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt</code>
===应用和测试===
+
===Applications and Test===
====系统状态检测====
+
====Check system status====
完成系统安装后,在板端执行以下命令来检测是否正确连接摄像头。
+
Run the following command to confirm whether the camera is initialized.
  
 
<code>dmesg | grep veye327</code>  
 
<code>dmesg | grep veye327</code>  
  
应当有如下提示:
+
The output message appears as shown below:
  
 
<code>Detected VEYE06 sensor</code>
 
<code>Detected VEYE06 sensor</code>
====视频流测试====
+
====Video Stream Toolkits Manual====
=====使用gstreamer=====
+
=====Gstreamer Usage=====
======使用gst-camera-veye.sh======
+
======using gst-camera-veye.sh======
由于友善之臂官方提供的gst-camera.sh默认分辨率为720p,VEYE-MIPI-327不支持,我们提供了 修改版本的gst-camera-veye.sh。
+
Since the gst-camera.sh officially provided by friendlyarm has a default resolution of 720p, which is not supported by VEYE-MIPI-327, we have provided a modified version —— gst-camera-veye.sh.
  
目录为:
+
In this directory:
  
 
<code>rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/gst-camera-sh/</code>
 
<code>rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/gst-camera-sh/</code>
  
*视频预览1080p HD
+
*Preview FHD
  
 
<code>./gst-camera-veye.sh -a preview</code>
 
<code>./gst-camera-veye.sh -a preview</code>
  
*视频录像1080p HD
+
*Record FHD in H.264 format to a video file
  
 
<code>./gst-camera-veye.sh --action video -output 1.ts</code>
 
<code>./gst-camera-veye.sh --action video -output 1.ts</code>
  
*抓拍图片
+
*Capturing FHD still image  
  
 
<code>./gst-camera-veye.sh -a photo -o 1.jpg</code>
 
<code>./gst-camera-veye.sh -a photo -o 1.jpg</code>
======直接使用gst-launch-1.0======
+
======using gst-launch-1.0======
  
*视频预览1080p HD
+
*Preview FHD
  
 
<code>gst-launch-1.0 rkisp device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! rkximagesink</code>
 
<code>gst-launch-1.0 rkisp device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! rkximagesink</code>
  
*视频录像1080p HD
+
*Record FHD in H.264 format to a video file
  
 
<code>gst-launch-1.0 rkisp num-buffers=512 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! queue ! h264parse ! filesink location=/tmp/out.mp4 -e</code>
 
<code>gst-launch-1.0 rkisp num-buffers=512 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! queue ! h264parse ! filesink location=/tmp/out.mp4 -e</code>
  
*抓拍图片
+
*Capturing FHD still image  
  
 
<code>gst-launch-1.0 rkisp num-buffers=2 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=10/1 ! jpegenc ! multifilesink location="/tmp/isp-frame%d.jpg" -e</code>
 
<code>gst-launch-1.0 rkisp num-buffers=2 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=10/1 ! jpegenc ! multifilesink location="/tmp/isp-frame%d.jpg" -e</code>
====视频控制软件包的使用====
+
====Video Control Toolkits Manual====
我们使用DRA(Directly Register Access)模式进行摄像头模组寄存器的配置。并且提供了一个shell脚本帮助使用。
+
We use DRA (Directly Register Access) mode to configure the camera module register. And a shell script is provided to help you use it.
  
 
<code>rk3399_veye_dimaging_bsp/ubuntu/i2c_cmd</code>
 
<code>rk3399_veye_dimaging_bsp/ubuntu/i2c_cmd</code>
  
*Ubuntu系统执行命令
+
*detect camera link status
  
 
<code>i2cdetect [value]</code>
 
<code>i2cdetect [value]</code>
Line 186: Line 195:
 
|2
 
|2
 
|MIPI-CSI2
 
|MIPI-CSI2
|}检测硬件连接是否正常,VEYE327摄像头模组的产品i2c地址为0x3b。
+
|}Check whether the hardware connection is normal. The product i2C address of the VEYE327 camera module is 0x3b.
  
*确定硬件连接正常后,脚本命令如下,-b指明i2c bus:
+
*After confirming that the hardware connection is working, the script command is as follows,-b indicates i2c bus:
  
 
<code>veye_mipi_i2c_rk.sh</code>
 
<code>veye_mipi_i2c_rk.sh</code>
  
视频控制接口软件包选项具体使用操作详见:[[VEYE-MIPI-290/327 i2c/zh|I2C脚本使用说明]]
+
Video Control Toolkits Manual  : [[VEYE-MIPI-290/327 i2c/|VEYE-MIPI-290/327 i2c]]
===参考资料===
+
===Exsiting bugs===
====RK3399介绍====
+
====Bug description====
RK3399是[http://www.rock-chips.com/ 瑞芯微]出品的一款高性价比的运算平台SoC。在业内拥有广泛的应用。
+
 
 +
*Can not encode to H.264 stream at full framerate
 +
 
 +
Using the gstreamer instructions used in the previous section to encode the video, the full frame cannot be reached. It can only reach about 17 frames. Limiting all CPU speeds to 600MHz which gives a stable 25FPS in PAL mode.
 +
 
 +
*Memory leak
  
RK3399的资料详细,开放度高,备受欢迎。 重点资料列举如下:
+
Using rkisp as a gstreamer plug-in, there will be serious memory leaks.
  
 +
====Bug Analysis====
 +
Although rockchip's rk3399 supports camera module of YUV format, there are serious bugs in the gstreamer plug-in they provide.
 +
===Refernces===
 +
====From Rockchip====
 
[http://opensource.rock-chips.com/wiki_Main_Page wiki]
 
[http://opensource.rock-chips.com/wiki_Main_Page wiki]
  
Line 204: Line 222:
  
 
[https://github.com/rockchip-linux/docs docs]
 
[https://github.com/rockchip-linux/docs docs]
 +
====From Friendlyarm====
 +
[http://wiki.friendlyarm.com/wiki/index.php/NanoPi_M4/zh wiki]
  
采用RK3399的主板也非常丰富,可供选择。各家软件均为基于瑞芯微资料制作,所以本文的内容也具有一定的普适性,不局限于NanoPi4系列。
+
[https://github.com/friendlyarm github]
====NanoPi M4介绍====
 
NanoPi M4是[https://www.friendlyarm.com/ 友善之臂]出品的基于RK3399的开发板。在爱好者和行业内拥趸极多。其资料开放,易于开发,[http://wiki.friendlyarm.com/wiki/index.php/NanoPi_M4/zh wiki]详细周到。
 
 
 
友善之臂也维护了自己[https://github.com/friendlyarm github]
 
 
 
我们在开发的时候,采用的是友善之臂的[http://download.friendlyarm.com/NanoPiM4 官方定版版本],建议采用google drive云盘下载,速度更快。
 
 
 
 
 
Nanopi的4系列均采用RK3399芯片,本文均可适用。
 
  
<br />
+
[http://download.friendlyarm.com/NanoPiM4 Download Center]

Latest revision as of 09:20, 12 June 2020

查看中文

How to use VEYE-MIPI-327 camera module on FriendlyARM NanoPi 4(RK3399)

1 Overview

This guide shows how to use VEYE-MIPI-290/327 on Friendlyarm Nanopi4(NanoPi M4,NanoPi NEO4,NanoPi T4),we use NanoPi M4 as an example.

RK3399 is Rockchip opensource platform. We use linuxsdk-friendlyelec-20190903 version SDK,and micro SD card version.

2 Hardware Setup

The system installation is shown as follows in the figure. We have Nanopi adapter board: VEYE-MIPI-327RK datasheet.

Connect VEYE-MIPI-327 to NanoPi M4
NanoPi M4 FFC cable details
VEYE-MIPI-327 adatpter board details


Note:NanoPi M4 board MIPI-CSI2 interface description

Interface I2C DPHY facing Reuse
MIPI-CSI1 1 RX0 back NA
MIPI-CSI2 2 RX1 front TX1

3 Using prebuilt Image to upgrade NanoPi4

  • download

You can use the image prebuild by us to upgrade nanopi system, which will support our camera.

The image is based on friendlyelec-20190903 SD card version. Download here.

  • Boot from SD Card

Extract these files(rk3399-sd-friendlydesktop-bionic-4.4-arm64-20200112.img). Insert an SD card(at least 16G) into a Windows PC and run the win32diskimager utility as administrator. On the utility's main window select your SD card's drive, the wanted image file and click on "write" to start flashing the SD card.

You can refer to Link.

4 Using source code to upgrade NanoPi4

This section provides instructions on how to update the system firmware using source code. It mainly involves two parts, kernel and dtb. Customers can refer to this section to port the driver of VEYE-MIPI-327 to the motherboard of other RK3399.

4.1 Setting Up the Environment

  • Install Cross Compiler

Please refer to link by friendlyarm to install cross complier.

  • Download kernel source code

There are three way to get kernel source code.

  1. linuxsdk-friendlyelec sdk provided by friendlyarm,Link:http://download.friendlyarm.com/NanoPiM4
  2. from friendlyarm's github.
  3. from rockchip's github.

We use the first way,download linuxsdk-friendlyelec sdk,and sync it.

tar -xvf linuxsdk-friendlyelec-20190903.tar

.repo/repo/repo sync -l

  • download bsp provided by us

The github link is :https://github.com/veyeimaging/rk3399_veye_dimaging_bsp


git clone https://github.com/veyeimaging/rk3399_veye_dimaging_bsp.git

4.2 Using prebuild Image and DTB

We provide NanoPi M4 resource.img and kernel.img.

  • Download sd-fuse_rk3399

git clone https://github.com/friendlyarm/sd-fuse_rk3399.git

  • Download friendlydesktop-arm64-images.tgz

Link:http://download.friendlyarm.com/NanoPiM4,find the corresponding file in the images-for-eflasher directory and download it.


tar -xzvf friendlydesktop-arm64-images.tgz -C sd-fuse_rk3399

  • Replace files and Flashing

tar -xzvf  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/kernel.img.tar.gz -C ./sd-fuse_rk3399/friendlydesktop-arm64/

tar -xzvf  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/resource.img.tar.gz -C ./sd-fuse_rk3399/friendlydesktop-arm64/  

cp  rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/images/update_partmap.txt  ./sd-fuse_rk3399/friendlydesktop-arm64/

Insert an SD card(at least 16G) into a Windows PC. Usingthe following command, you can only upgrade kernel and dtb, instead of flashing the whole card, will be faster.

cd ./sd-fuse_rk3399/

sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt

For more information on how to use sd-fuse_rk3399, please see sd-fuse_rk3399 Readme.

4.3 Building from Source code

  • copyour driver files in the corresponding kernel directory

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/VEYE-MIPI-327/veye327.c ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Makefile ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Kconfig ~/linuxsdk-friendlyelec/kernel/drivers/media/i2c/

  • Add compilation options

cd ~/linuxsdk-friendlyelec/kernel

make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 nanopi4_linux_defconfig

make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 menuconfig

Add CONFIG_VIDEO_VEYE327 option

make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 nanopi4-images

cp kernel.img resource.img ~/sd-fuse_rk3399/friendlydesktop-arm64/

4.3.1 Flashing SD card

Insert an SD card(at least 16G) into a Windows PC. Usingthe following command, you can only upgrade kernel and dtb.

cd ./sd-fuse_rk3399/

sudo ./tools/sd_update -d /dev/sdb -p ./friendlydesktop-arm64/update_partmap.txt

5 Applications and Test

5.1 Check system status

Run the following command to confirm whether the camera is initialized.

dmesg | grep veye327  

The output message appears as shown below:

Detected VEYE06 sensor

5.2 Video Stream Toolkits Manual

5.2.1 Gstreamer Usage
5.2.1.1 using gst-camera-veye.sh

Since the gst-camera.sh officially provided by friendlyarm has a default resolution of 720p, which is not supported by VEYE-MIPI-327, we have provided a modified version —— gst-camera-veye.sh.

In this directory:

rk3399_veye_dimaging_bsp/ubuntu/platform/friendlyarm_nanopi4/gst-camera-sh/

  • Preview FHD

./gst-camera-veye.sh -a preview

  • Record FHD in H.264 format to a video file

./gst-camera-veye.sh --action video -output 1.ts

  • Capturing FHD still image  

./gst-camera-veye.sh -a photo -o 1.jpg

5.2.1.2 using gst-launch-1.0
  • Preview FHD

gst-launch-1.0 rkisp device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! rkximagesink

  • Record FHD in H.264 format to a video file

gst-launch-1.0 rkisp num-buffers=512 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! queue ! h264parse ! filesink location=/tmp/out.mp4 -e

  • Capturing FHD still image  

gst-launch-1.0 rkisp num-buffers=2 device=/dev/video0 io-mode=1 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=10/1 ! jpegenc ! multifilesink location="/tmp/isp-frame%d.jpg" -e

5.3 Video Control Toolkits Manual

We use DRA (Directly Register Access) mode to configure the camera module register. And a shell script is provided to help you use it.

rk3399_veye_dimaging_bsp/ubuntu/i2c_cmd

  • detect camera link status

i2cdetect [value]

value description
1 MIPI-CSI1
2 MIPI-CSI2

Check whether the hardware connection is normal. The product i2C address of the VEYE327 camera module is 0x3b.

  • After confirming that the hardware connection is working, the script command is as follows,-b indicates i2c bus:

veye_mipi_i2c_rk.sh

Video Control Toolkits Manual : VEYE-MIPI-290/327 i2c

6 Exsiting bugs

6.1 Bug description

  • Can not encode to H.264 stream at full framerate

Using the gstreamer instructions used in the previous section to encode the video, the full frame cannot be reached. It can only reach about 17 frames. Limiting all CPU speeds to 600MHz which gives a stable 25FPS in PAL mode.

  • Memory leak

Using rkisp as a gstreamer plug-in, there will be serious memory leaks.

6.2 Bug Analysis

Although rockchip's rk3399 supports camera module of YUV format, there are serious bugs in the gstreamer plug-in they provide.

7 Refernces

7.1 From Rockchip

wiki

github

docs

7.2 From Friendlyarm

wiki

github

Download Center