Difference between revisions of "VEYE MIPI 327 for KHADAS EDGE ubuntu"

From wiki_veye
Jump to navigation Jump to search
(Created blank page)
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
[[VEYE MIPI 327 for KHADAS EDGE ubuntu/zh|查看中文]]
  
 +
[[VEYE MIPI 327 for KHADAS EDGE ubuntu|<big>How to use VEYE-MIPI-327 camera module on KHADAS EDGE(RK3399 ubuntu)</big>]]
 +
 +
===Overview===
 +
This guide shows how to use VEYE-MIPI-290/327 on [https://www.khadas.com/ KHADAS Edge Board]. We use fenix_0.8.2(20200110) version source code.
 +
 +
VEYE-MIPI-290/327 is STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide '''V4L2 interface''' for video streaming apps , and '''Video Control Toolkits (which is Shell Script)''' to control the camera module directly.
 +
===Hardware Setup===
 +
The system installation is shown as follows in the figure. We have a special Edge adapter board.For Details：（TODO）.[[File:KHADAS EDGE VEYE327.jpg|alt=|center|thumb|602x602px|VEYE-MIPI-327 connect to Khadas Edge-V|link=http://wiki.veye.cc/index.php/File:KHADAS_EDGE_VEYE327.jpg]][[File:KHADAS EDGE FFC.jpg|alt=|center|thumb|602x602px|Khadas Edge-V FFC cable details（Contact facing outward）|link=http://wiki.veye.cc/index.php/File:KHADAS_EDGE_FFC.jpg]][[File:VEYE-MIPI-327KH FFC.jpg|alt=|center|thumb|596x596px|VEYE-MIPI-327  FFC cable details（Contact facing outward）|link=http://wiki.veye.cc/index.php/File:VEYE-MIPI-327KH_FFC.jpg]]
 +
 +
 +
Note：Khadas Edge-V board use i2c-6 for both CAM0 and CAM1,We just support CAM0 now.
 +
===Download bsp package we provide===
 +
Download source code and bsp form link below：https://github.com/veyeimaging/rk3399_veye_dimaging_bsp
 +
 +
<code>git clone [https://github.com/veyeimaging/rk3399_veye_dimaging_bsp https://github.com/veyeimaging/rk3399_veye_dimaging_bsp.git]</code>
 +
===Using the prebuilt Image to upgrade Khadas Edge===
 +
 +
*Install Official Image first
 +
 +
Download Offical Image：https://docs.khadas.com/edge/FirmwareUbuntu.html<nowiki/>，We use Edge_Ubuntu-lxde-bionic_Linux-4.4_arm64_EMMC_V20190830 as an example.
 +
 +
Flashing the Edge Board follow manual：https://docs.khadas.com/edge/HowtoBootIntoUpgradeMode.html<nowiki/>。
 +
 +
*Upgrading kernel and dtb
 +
 +
The upgrade package path is:
 +
 +
<code>~/rk3399_veye_dimaging_bsp/ubuntu/platform/khadas_edge-v</code>
 +
 +
copy kernel-debs.tgz and rkisp-engine-2.0.1_arm64.tgz to Edge Board.
 +
 +
*Installing kernel and dtb
 +
 +
<code>tar -xzvf kernel-debs.tgz</code>
 +
 +
<code>cd kernel-debs</code>
 +
 +
<code>sudo dpkg -i linux-dtb-rockchip-4.4_0.8.2_arm64.deb</code>
 +
 +
<code>sudo dpkg -i linux-headers-rockchip-4.4_0.8.2_arm64.deb</code>
 +
 +
<code>sudo dpkg -i linux-image-rockchip-4.4_0.8.2_arm64.deb</code>
 +
 +
*Installing rkisp librarys(as needed)
 +
 +
<code>tar -xzvf rkisp-engine-2.0.1_arm64.tgz</code>
 +
 +
<code>sudo cp -arf rkisp-engine-2.0.1_arm64/* /</code>
 +
===Using the source code to upgrade Khadas Edge===
 +
====Setting Up the Environment====
 +
 +
*Installing Fenix and download source code
 +
 +
According to the steps of Khadas：https://docs.khadas.com/zh-cn/edge/FenixScript.html<nowiki/>，The latest code is automatically downloaded.
 +
 +
<code>make kernel</code>
 +
====Building from Source====
 +
 +
*patch code
 +
 +
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/VEYE-MIPI-327/veye327.c ~/project/fenix/linux/drivers/media/i2c/</code>
 +
 +
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Makefile ~/project/fenix/linux/drivers/media/i2c/</code>
 +
 +
<code>cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Kconfig ~/project/fenix/linux//drivers/media/i2c/</code>
 +
 +
*add compile option
 +
 +
<code>cd ~/project/fenix/linux/</code>
 +
 +
<code>make ARCH=arm64 kedge_defconfig</code>
 +
 +
<code>make ARCH=arm64 menuconfig</code>
 +
 +
add CONFIG_VIDEO_VEYE327 option.
 +
 +
or you can edit .config to add CONFIG_VIDEO_VEYE327 option.
 +
 +
*build
 +
 +
<code>make kernel && make kernel-deb</code>
 +
 +
the output debs is in ./build/images/debs/0.8.2/ directory.
 +
===Applications and Test===
 +
====Check system status====
 +
Run the following command to confirm whether the camera is correctly connected.
 +
 +
<code>dmesg | grep veye327</code>  
 +
 +
The output message appears as shown below：
 +
 +
<code>Detected VEYE06 sensor</code>
 +
====Video Stream Toolkits Manual====
 +
=====gst-launch-1.0=====
 +
 +
*Preview 1080p HD
 +
 +
*Record 1080p HD in H.264 format to a video file
 +
 +
<code>gst-launch-1.0 rkv4l2src num-buffers=512 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! queue ! mpph264enc ! queue ! h264parse ! mpegtsmux ! filesink location=/tmp/veye_video.ts</code>
 +
 +
*Capturing FHD still image  
 +
 +
====Video Control Toolkits Manual====
 +
We use DRA(Directly Register Access) mode to set camera paramters. And we provide shell script for using. Location：
 +
 +
<code>rk3399_veye_dimaging_bsp/ubuntu/i2c_cmd</code>
 +
 +
*Check hardware connection
 +
 +
<code>i2cdetect 6</code>
 +
 +
VEYE327camera i2c address is 0x3b.
 +
 +
*use veye_mipi_i2c_rk.sh to setting paramters,-b is i2c bus：
 +
 +
<code>veye_mipi_i2c_rk.sh -b 6</code>
 +
 +
Video Control Toolkits Manual : [[VEYE-MIPI-290/327 i2c/|VEYE-MIPI-290/327 i2c]]
 +
===Exsiting bugs===
 +
====Bug description====
 +
 +
*Can not encode to H.264 stream at full framerate
 +
 +
Using the gstreamer instructions used in the previous section to encode the video, the full frame cannot be reached. It can only reach about 17 frames.  Limiting all CPU speeds to 600MHz which gives a stable 25FPS in PAL mode.
 +
====Bug Analysis====
 +
Although rockchip's rk3399 supports camera module input in yuv format, there is a serious bug in the gstreamer plug-in they provide.
 +
===References===
 +
 +
====From Khadas====
 +
[https://docs.khadas.com/ docs]
 +
 +
====From Rockchip====
 +
[http://opensource.rock-chips.com/wiki_Main_Page wiki]
 +
 +
[https://github.com/rockchip-linux/ github]
 +
 +
[https://github.com/rockchip-linux/docs docs]
 +
<br />

Revision as of 13:39, 10 March 2020

查看中文

How to use VEYE-MIPI-327 camera module on KHADAS EDGE(RK3399 ubuntu)

1 Overview

This guide shows how to use VEYE-MIPI-290/327 on KHADAS Edge Board. We use fenix_0.8.2(20200110) version source code.

VEYE-MIPI-290/327 is STARVIS camera module with ISP functions build in. It output UYVY data using MIPI-CSI2. We provide V4L2 interface for video streaming apps , and Video Control Toolkits (which is Shell Script) to control the camera module directly.

2 Hardware Setup

The system installation is shown as follows in the figure. We have a special Edge adapter board.For Details：（TODO）.

VEYE-MIPI-327 connect to Khadas Edge-V
Khadas Edge-V FFC cable details（Contact facing outward）
VEYE-MIPI-327 FFC cable details（Contact facing outward）


Note：Khadas Edge-V board use i2c-6 for both CAM0 and CAM1,We just support CAM0 now.

3 Download bsp package we provide

Download source code and bsp form link below：https://github.com/veyeimaging/rk3399_veye_dimaging_bsp

git clone https://github.com/veyeimaging/rk3399_veye_dimaging_bsp.git

4 Using the prebuilt Image to upgrade Khadas Edge

  • Install Official Image first

Download Offical Image：https://docs.khadas.com/edge/FirmwareUbuntu.html，We use Edge_Ubuntu-lxde-bionic_Linux-4.4_arm64_EMMC_V20190830 as an example.

Flashing the Edge Board follow manual：https://docs.khadas.com/edge/HowtoBootIntoUpgradeMode.html。

  • Upgrading kernel and dtb

The upgrade package path is:

~/rk3399_veye_dimaging_bsp/ubuntu/platform/khadas_edge-v

copy kernel-debs.tgz and rkisp-engine-2.0.1_arm64.tgz to Edge Board.

  • Installing kernel and dtb

tar -xzvf kernel-debs.tgz

cd kernel-debs

sudo dpkg -i linux-dtb-rockchip-4.4_0.8.2_arm64.deb

sudo dpkg -i linux-headers-rockchip-4.4_0.8.2_arm64.deb

sudo dpkg -i linux-image-rockchip-4.4_0.8.2_arm64.deb

  • Installing rkisp librarys(as needed)

tar -xzvf rkisp-engine-2.0.1_arm64.tgz

sudo cp -arf rkisp-engine-2.0.1_arm64/* /

5 Using the source code to upgrade Khadas Edge

5.1 Setting Up the Environment

  • Installing Fenix and download source code

According to the steps of Khadas：https://docs.khadas.com/zh-cn/edge/FenixScript.html，The latest code is automatically downloaded.

make kernel

5.2 Building from Source

  • patch code

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/VEYE-MIPI-327/veye327.c ~/project/fenix/linux/drivers/media/i2c/

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Makefile ~/project/fenix/linux/drivers/media/i2c/

cp rk3399_veye_dimaging_bsp/ubuntu/drivers_source/Kconfig ~/project/fenix/linux//drivers/media/i2c/

  • add compile option

cd ~/project/fenix/linux/

make ARCH=arm64 kedge_defconfig

make ARCH=arm64 menuconfig

add CONFIG_VIDEO_VEYE327 option.

or you can edit .config to add CONFIG_VIDEO_VEYE327 option.

  • build

make kernel && make kernel-deb

the output debs is in ./build/images/debs/0.8.2/ directory.

6 Applications and Test

6.1 Check system status

Run the following command to confirm whether the camera is correctly connected.

dmesg | grep veye327  

The output message appears as shown below：

Detected VEYE06 sensor

6.2 Video Stream Toolkits Manual

6.2.1 gst-launch-1.0
  • Preview 1080p HD
  • Record 1080p HD in H.264 format to a video file

gst-launch-1.0 rkv4l2src num-buffers=512 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! queue ! mpph264enc ! queue ! h264parse ! mpegtsmux ! filesink location=/tmp/veye_video.ts

  • Capturing FHD still image  

6.3 Video Control Toolkits Manual

We use DRA(Directly Register Access) mode to set camera paramters. And we provide shell script for using. Location：

rk3399_veye_dimaging_bsp/ubuntu/i2c_cmd

  • Check hardware connection

i2cdetect 6

VEYE327camera i2c address is 0x3b.

  • use veye_mipi_i2c_rk.sh to setting paramters,-b is i2c bus：

veye_mipi_i2c_rk.sh -b 6

Video Control Toolkits Manual : VEYE-MIPI-290/327 i2c

7 Exsiting bugs

7.1 Bug description

  • Can not encode to H.264 stream at full framerate

Using the gstreamer instructions used in the previous section to encode the video, the full frame cannot be reached. It can only reach about 17 frames. Limiting all CPU speeds to 600MHz which gives a stable 25FPS in PAL mode.

7.2 Bug Analysis

Although rockchip's rk3399 supports camera module input in yuv format, there is a serious bug in the gstreamer plug-in they provide.

8 References

8.1 From Khadas

docs

8.2 From Rockchip

wiki

github

docs

Retrieved from "http://wiki.veye.cc/index.php?title=VEYE_MIPI_327_for_KHADAS_EDGE_ubuntu&oldid=1608"