Changes

Jump to navigation Jump to search

MV Camera on Firfly Boards (view source)

Revision as of 20:14, 12 April 2023

8,394 bytes added ,  20:14, 12 April 2023
no edit summary
Line 1: Line 1:  
[[MV Camera on Firfly Boards/zh|查看中文]]
 
[[MV Camera on Firfly Boards/zh|查看中文]]
 +
 +
=== Overview ===
 +
The MV series and RAW series cameras are cameras designed for AI applications in the industrial field. They use the MIPI CSI-2 interface and are particularly suitable for use with embedded computing platforms. They have rich data formats and triggering features, extremely low latency, high bandwidth, and reliable stability.
 +
 +
This article takes Firefly's ROC-RK3588S-PC motherboard as an example to introduce how to connect MV and RAW series cameras to the RK3588S/RK3588 system.
 +
 +
We provide drivers for the Linux operating system (using Ubuntu as an example).
 +
 +
==== Camera Module List ====
 +
{| class="wikitable"
 +
!Series
 +
!Model
 +
!Status
 +
|-
 +
|MV series
 +
|MV-MIPI-IMX178M
 +
|Done
 +
|-
 +
|MV series
 +
|MV-MIPI-SC130M
 +
|Done
 +
|-
 +
|MV series
 +
|MV-MIPI-IMX296M
 +
|Done
 +
|-
 +
|MV series
 +
|MV-MIPI-IMX287M
 +
|Done
 +
|-
 +
|MV series
 +
|MV-MIPI-IMX265M
 +
|Done
 +
|-
 +
|MV series
 +
|MV-MIPI-IMX264M
 +
|Done
 +
|-
 +
|RAW series
 +
|RAW-MIPI-SC132M
 +
|Done
 +
|}
 +
 +
=== Hardware Setup ===
 +
The MV series and RAW series cameras require an [[ADP-MV2 Adapter Board Data Sheet/zh|ADP-MV2]] adapter board to connect to the ROC-RK35xx-PC motherboard.
 +
 +
==== Connection of MV-MIPI-CAM and ADP-MV2 ====
 +
The two are connected using 0.5 mm pitch*30P FFC cable with opposite direction. The cable must be inserted with the silver contacts facing outside.
 +
{| class="wikitable"
 +
!TOP
 +
!BOTTOM
 +
|-
 +
|[[File:ADP-MV2-MV-MIPI 01.jpg|alt=|center|thumb|600x600px|ADP-MV2 to MV-MIPI-X|link=http://wiki.veye.cc/index.php/File:ADP-MV2-MV-MIPI_01.jpg]]
 +
|[[File:ADP-MV2-MV-MIPI 02.jpg|alt=|center|thumb|600x600px|ADP-MV2 to MV-MIPI-X|link=http://wiki.veye.cc/index.php/File:ADP-MV2-MV-MIPI_02.jpg]]
 +
|}
 +
 +
==== Connection of RAW-MIPI-CAM and ADP-MV2 ====
 +
The two are connected using 1.0 mm pitch*15P FFC cable with opposite direction. The cable must be inserted with the silver contacts facing outside.
 +
{| class="wikitable"
 +
!TOP
 +
!BOTTOM
 +
|-
 +
|[[File:ADP-MV2-RAW-MIPI 02.jpg|alt=|center|thumb|600x600px|ADP-MV2 to RAW-MIPI-X|link=http://wiki.veye.cc/index.php/File:ADP-MV2-RAW-MIPI_02.jpg]]
 +
|[[File:ADP-MV2-RAW-MIPI 01.jpg|alt=|center|thumb|600x600px|ADP-MV2 to RAW-MIPI-X|link=http://wiki.veye.cc/index.php/File:ADP-MV2-RAW-MIPI_01.jpg]]
 +
|}
 +
 +
==== Connection with Main board using ADP-MV2 ====
 +
[[File:RK-ADP-MV2-MV-MIPI 01.jpg|center|thumb|800x800px|RK to ADP-MV2 and MV cam|link=http://wiki.veye.cc/index.php/File:RK-ADP-MV2-MV-MIPI_01.jpg]]
 +
===Introduction to github repositories===
 +
https://github.com/veyeimaging/rk35xx_firefly
 +
 +
includes：
 +
 +
*driver source code
 +
*i2c toolkits
 +
*application demo
 +
 +
In addition, a compiled linux kernel installation package and Android image is provided in the [https://github.com/veyeimaging/rk356x_firefly/releases releases].
 +
 +
=== Upgrade Firefly Ubuntu system ===
 +
For the ROC-RK3588S-PC, we have provided an image of the release system.
 +
 +
Download the latest rk358x_firefly_ubuntu.tar.gz from https://github.com/veyeimaging/rk35xx_firefly/releases/ .
 +
 +
Refer to the [https://wiki.t-firefly.com/en/ROC-RK3588S-PC/upgrade_bootmode.html Firefly documentation] to burn in a standard system.
 +
===Check system status===
 +
 +
==== Whether the camera is correctly recognized ====
 +
After system update, reboot the main board.
 +
 +
Execute the following command on the main board to check if the camera is properly connected.
 +
 +
<code>dmesg | grep mvcam</code>
 +
 +
You can see the camera model and the camera version number probed.
 +
 +
A prompt as below indicates that the MV-MIPI-IMX296M camera is detected on the i2c-7 bus.
 +
 +
<code>mvcam 7-003b: camera is：MV-MIPI-IMX296M</code>
 +
 +
<code>mvcam 7-003b: firmware version: 0x1290133</code>
 +
 +
* Check the video0 device node:
 +
 +
<code>ls /dev/video0</code>
 +
 +
You should see:
 +
 +
<code>video0</code>
 +
 +
After successfully identifying the camera, the camera will be recognized as /dev/video0.
 +
 +
Use <code>media-ctl</code> command to view the current topology structure
 +
 +
==== Using media-ctl to view topology ====
 +
Using the media-ctl command can clearly display the current topography structure.
 +
 +
<code>media-ctl -p -d /dev/media0</code>
 +
 +
===== Link relationship =====
 +
mv camera->rockchip-csi2-dphy0->rockchip-mipi-csi2->stream_cif_mipi_id0 - - ->DDR(/dev/video0)
 +
 +
The application can obtain images through the /dev/video0 node.
 +
 +
===== mv camera entity information =====
 +
Taking the MV-MIPI-IMX296M as an example:
 +
 +
<code>- entity 63: m00_b_mvcam 7-003b (1 pad, 1 link)</code>
 +
 +
<code>             type V4L2 subdev subtype Sensor flags 0</code>
 +
 +
<code>             device node name /dev/v4l-subdev2</code>
 +
 +
<code>        pad0: Source</code>
 +
 +
<code>                [fmt:Y8_1X8/1456x1088@100/6000 field:none]</code>
 +
 +
<code>                -> "rockchip-csi2-dphy0":0 [ENABLED]</code>
 +
 +
You can see that:
 +
 +
* The complete name of this entity is: <code>m00_b_mvcam 7-003b</code>.
 +
* It is a V4L2 subdev (Sub-Device) Sensor.
 +
* Its corresponding node is <code>/dev/v4l-subdev2</code>, which can be opened and configured by applications (such as <code>v4l2-ctl</code>).
 +
* Its output format is <code>[fmt:Y8_1X8/1456x1088@100/6000 field:none]</code>, where <code>Y8_1X8</code> is a shorthand for a mbus-code, which will be listed in the next section of this article.
 +
* The current resolution is <code>1456x1088</code>.
 +
* The current frame interval is <code>100/6000</code>, which means the frame rate is 60.
 +
* The data format output by the camera can be modified using the media-ctl command.
 +
 +
===== mbus-code list =====
 +
MV series and RAW series cameras have different data format capabilities, which can be found in the data manual for each camera model.
 +
{| class="wikitable"
 +
!Format on datasheet
 +
!mbus-code for media-ctl
 +
!FourCC pixelformat for v4l2-ctl
 +
|-
 +
|RAW8
 +
|Y8_1X8
 +
|GREY
 +
|-
 +
|RAW10
 +
|Y10_1X10
 +
|'Y10 '
 +
|-
 +
|RAW12
 +
|Y12_1X12
 +
|'Y12 '
 +
|-
 +
|UYVY
 +
|UYVY8_2X8
 +
|UYVY
 +
|}
 +
 +
=== Raw data format ===
 +
The VICAP module of RK3588 supports two data saving formats, Compact and Noncompact RAW. You can modify the mode using the RKCIF_CMD_SET_CSI_MEMORY_MODE ioctl command of RKCIF. By default, the output is in Compact RAW format.[[File:Compact raw and noncompact raw of rk3588 vicap.png|center|thumb|800x800px|Compact raw and noncompact raw of rk3588 VICAP|link=http://wiki.veye.cc/index.php/File:Compact_raw_and_noncompact_raw_of_rk3588_vicap.png]]
 +
 +
==== Noncompact RAW ====
 +
For pixel data with 10-bit depth or 12-bit depth, two bytes are always used to store one pixel. This storage method is convenient for software processing, but it has the disadvantage of occupying a large amount of space.
 +
 +
Depending on whether the effective data is stored in the high bits or low bits, it can be further divided into two types: high align and low align.
 +
 +
=====Noncompact RAW(high align)=====
 +
Data is saved to the high bits, and the unused low bits are filled with 0. This is one of the data formats supported by RK VICAP.
 +
 +
===== Noncompact RAW(low align) =====
 +
In Noncompact RAW (low align) format, data is saved to the low bits, and the unused high bits are filled with 0. The V4L2 standard 'Y10' (10-bit Greyscale) and 'Y12' (12-bit Greyscale) formats are both stored in this way.
 +
 +
The pixel_layer_convert conversion tool mentioned later in the article also converts Compact RAW to this storage format for easy display using image players.
 +
 +
==== Compact RAW ====
 +
As shown above，there is no bit padding between pixels in this storage format.
 +
 +
==== Line stride ====
 +
To facilitate fast operations on images, the system usually provides row-aligned buffer sizes for each line of data. RK3588 uses 256-byte alignment for this purpose.
 +
 +
line_stride = ALIGN_UP（image_width*bits_per_pixel/8,256）
 +
 +
For example, when the image width is 1456:
 +
 +
8bit depth，line_stride=1536
 +
 +
10bit depth，preferred_stride=2048
 +
 +
12bit depth，preferred_stride=2304
 +
====Format convert tool====
 +
We have written a small tool: [https://github.com/veyeimaging/pixel_layer_convert pixel_layer_convert], which can easily convert Compact images to Noncompact (low align) images.
 +
 +
For example, the following command can convert a Compact RAW10 image with a width of 1456 to Noncompact RAW10 format:
 +
 +
<code>./pixel_layer_convert -I R10C -i y10-1456x1088_0001.raw -o y10-1456x1088_0001_new.raw -w 1456</code>
 +
====Raw data image player====
 +
We recommend using [https://www.offminor.de/ vooya] as the player, which supports GREY, and unpacked image formats.
 +
 +
Also, y8 file can be used with this player: [https://yuv-player-deluxe.software.informer.com/2.6/ YUV Displayer Deluxe].
Retrieved from "http://wiki.veye.cc/index.php/Special:MobileDiff/6582"

Navigation menu