Difference between revisions of "Mv series camera appnotes 4 jetson"

From wiki_veye
Jump to navigation Jump to search
Line 215: Line 215:
 
|}
 
|}
 
Also, we provide tool software in order to convert these special data formats to standard formats: https://github.com/veyeimaging/pixel_layer_convert.
 
Also, we provide tool software in order to convert these special data formats to standard formats: https://github.com/veyeimaging/pixel_layer_convert.
 +
 +
We recommend using [https://www.offminor.de/ vooya] as the player.
  
 
===== Support for trigger mode =====
 
===== Support for trigger mode =====
Line 249: Line 251:
 
In gstreamer, v4l2src can access different cameras by specifying device=/dev/videoX.
 
In gstreamer, v4l2src can access different cameras by specifying device=/dev/videoX.
  
 +
=== v4l2-ctl Application Examples ===
 +
 +
=====Install v4l2-utils=====
 +
<code>sudo apt-get install v4l-utils</code>
 +
=====Install yavta=====
 +
<code>git clone <nowiki>git://git.ideasonboard.org/yavta.git</nowiki></code>
 +
 +
<code>cd yavta;make</code>
 +
=====Configure parameters using v4l2-ctl=====
 +
======List the data formats supported by the camera======
 +
<code>v4l2-ctl --list-formats-ext</code>
 +
 +
The following is an example in the XAVIER system：
 +
 +
<code>ioctl: VIDIOC_ENUM_FMT</code>
 +
 +
<code>        Index       : 0</code>
 +
 +
<code>        Type        : Video Capture</code>
 +
 +
<code>        Pixel Format: 'UYVY'</code>
 +
 +
<code>        Name        : UYVY 4:2:2</code>
 +
 +
<code>                Size: Discrete 3088x2064</code>
 +
 +
<code>                        Interval: Discrete 0.045s (22.000 fps)</code>
 +
 +
<code>        Index       : 1</code>
 +
 +
<code>        Type        : Video Capture</code>
 +
 +
<code>        Pixel Format: 'NV16'</code>
 +
 +
<code>        Name        : Y/CbCr 4:2:2</code>
 +
 +
<code>                Size: Discrete 3088x2064</code>
 +
 +
<code>                        Interval: Discrete 0.045s (22.000 fps)</code>
 +
 +
<code>        Index       : 2</code>
 +
 +
<code>        Type        : Video Capture</code>
 +
 +
<code>        Pixel Format: 'GREY'</code>
 +
 +
<code>        Name        : 8-bit Greyscale</code>
 +
 +
<code>                Size: Discrete 3088x2064</code>
 +
 +
<code>                        Interval: Discrete 0.045s (22.000 fps)</code>
 +
 +
<code>        Index       : 3</code>
 +
 +
<code>        Type        : Video Capture</code>
 +
 +
<code>        Pixel Format: 'XY10'</code>
 +
 +
<code>        Name        : XAVIER 10-bit/16-bit Greyscale</code>
 +
 +
<code>                Size: Discrete 3088x2064</code>
 +
 +
<code>                        Interval: Discrete 0.045s (22.000 fps)</code>
 +
 +
<code>        Index       : 4</code>
 +
 +
<code>        Type        : Video Capture</code>
 +
 +
<code>        Pixel Format: 'XY12'</code>
 +
 +
<code>        Name        : XAVIER 12-bit/16-bit Greyscale</code>
 +
 +
<code>                Size: Discrete 3088x2064</code>
 +
 +
                        <code>Interval: Discrete 0.045s (22.000 fps)</code>
 +
 +
It should be noted that the UYVY and NV16 modes provided by the MV series cameras are for debugging purposes only and the maximum width cannot exceed 2880.
 +
 +
====== List the configurable parameters of the camera implemented in the driver ======
 +
<code>v4l2-ctl -L</code>
 +
 +
<code>User Controls</code>
 +
 +
<code>                horizontal_flip 0x00980914 (bool)   : default=0 value=0</code>
 +
 +
<code>                  vertical_flip 0x00980915 (bool)   : default=0 value=0</code>
 +
 +
<code>                   trigger_mode 0x00981901 (int)    : min=0 max=2 step=1 default=0 value=0 flags=volatile, execute-on-write</code>
 +
 +
<code>                    trigger_src 0x00981902 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write</code>
 +
 +
<code>                    soft_trgone 0x00981903 (button) : flags=write-only, execute-on-write</code>
 +
 +
<code>                     frame_rate 0x00981904 (int)    : min=1 max=22 step=1 default=22 value=22 flags=volatile, execute-on-write</code>
 +
 +
<code>                          roi_x 0x00981905 (int)    : min=0 max=2712 step=8 default=0 value=0</code>
 +
 +
<code>                          roi_y 0x00981906 (int)    : min=0 max=1744 step=4 default=0 value=0</code>
 +
 +
<code>Camera Controls</code>
 +
 +
<code>low_latency_mode 0x009a206d (bool)   : default=0 value=0</code>
 +
 +
<code>               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0</code>
 +
 +
<code>            vi_time_out_disable 0x009a2078 (bool)   : default=0 value=0</code>
 +
 +
Parameters can be set and get using the following methods.
 +
 +
<code>v4l2-ctl --set-ctrl [ctrl_type]=[val]</code>
 +
 +
<code>v4l2-ctl --get-ctrl [ctrl_type]</code>
 +
 +
All the above functions can be implemented using [http://wiki.veye.cc/index.php/Mv_mipi_i2c.sh_user_guide mv_mipi_i2c.sh].
 +
 +
None of the above parameters can be modified in the state when the image acquisition is started.
 +
 +
Each of them is described below:
 +
======horizontal and <small>vertical</small> flip======
 +
 +
*horizontal flip
 +
 +
<code>v4l2-ctl --set-ctrl horizontal_flip=1</code>
 +
 +
*<small>vertical flip</small>
 +
 +
<code>v4l2-ctl --set-ctrl <small>vertical_flip</small>=1</code>
 +
======Trigger Mode======
 +
<code>v4l2-ctl --set-ctrl <small>trigger_mode=[0-2]</small></code>
 +
 +
0：Video streaming mode
 +
 +
1：Normal trigger mode.
 +
 +
2：High-speed continuous trigger mode.
 +
======Trigger Source======
 +
<code>v4l2-ctl --set-ctrl <small>trigger_src=[0-1]</small></code>
 +
 +
0: Software trigger mode.
 +
 +
1: Hardware trigger mode.
 +
======Software trigger command======
 +
<code>v4l2-ctl --set-ctrl <small>soft_trgone=1</small></code>
 +
======Set frame rate======
 +
<code>v4l2-ctl --set-ctrl frame_rate=[1-max]</code>
 +
 +
The maximum frame rate is automatically updated as the resolution changed.
 +
 +
======Set ROI and pixel format======
 +
For example, for MV-MIPI-IMX178M:
 +
 +
<code>v4l2-ctl --set-ctrl roi_x=0</code>
 +
 +
<code>v4l2-ctl --set-ctrl roi_y=0</code>
 +
 +
<code>v4l2-ctl --set-fmt-video=width=3088,height=2064,pixelformat=GREY</code>
 +
 +
The maximum frame rate will be adjusted automatically after setting ROI. Please note that the camera roi parameters need to comply with the requirements in the camera manual.
 +
 +
====== preferred_stride ======
 
<br />
 
<br />

Revision as of 12:07, 19 May 2022

查看中文

1 Overview

The MV series cameras are cameras introduced for AI applications in the industrial field. It uses the MIPI CSI-2 interface, which is especially suitable for embedded computing platforms.

It features rich data formats and trigger characteristics, extremely low latency, extremely high bandwidth and reliable stability.

This article describes how to use the MV series camera on the NVIDIA Jetson platform.

1.1 Camera Module List

Series Model Status
MV series MV-MIPI-IMX178M Done

1.2 Jetson Board List

Jetson Board Status
Nano A02 Done
Nano B01 Done
Nano 2GB Done
TX2 NX Done
XAVIER NX Done
TX2 Devkit Coming soon
AGX Xavier Coming soon
Orin Series Coming soon

1.3 Supported L4T versions

  • Jetpack4.6.1,L4T r32.7.1
1.3.1 How to check the current L4T version

On Jetson board

cat /etc/nv_tegra_release

If it shows:

# R32 (release), REVISION: 7.1......

It means L4t Version is 32.7.1, and the Jetpack version is 4.6.1.

2 Hardware Setup

MV series cameras require an adapter board to access the Jetson platform. The following table shows the support status.

Jetson Board adapter board camera number Power
Nano A02 ADP-MV1 1 5V DC（Required）
Nano B01 ADP-MV1 1 5V DC（Required）
Nano 2GB ADP-MV1 2 5V DC（Required）
TX2 NX ADP-MV1 2 5V DC（Required）
XAVIER NX ADP-MV1 2 5V DC（Required）
TX2 Devkit ADP-N4 6 12V DC (Optional)
AGX Xavier ADP-N4 6 12V DC (Optional)
Orin series TODO TODO TODO

2.1 Connection of MV-MIPI-IMX178M and ADP-MV1

The two are connected using 0.5 mm pitch*30P FFC cable with opposite direction. The cable must be inserted with the silver contacts facing outside.

ADP-MV1 to MV-MIPI-X
ADP-MV1 to MV-MIPI-X


2.2 Connection using ADP-MV1 (using Xavier NX as an example)

2.2.1 Power supply

The ADP-MV1 requires a separate 5V power supply and can be powered directly from the Jetson board using a Dupont cable.

ADP-MV1 power supply
ADP-MV1 power supply


2.3 Connection using ADP-MV1

The Nano Series and NX Series are connected in this way.

MV camera and Xavier NX connection
MV camera and Xavier NX connection


2.4 Connection using ADP-N4

The AGX Series and TX2 Devkit are connected in this way.

TODO

3 Upgrade Jetson system

This section describes how to upgrade the Jetson system to support MV camera module. For OS update method, please refer to Update Jetson OS.

Specially, for MV series camera, besides adding the camera driver in Linux system, we also add a kernel patch - veye_mv_l4t_32.7.1.patch.

This patch has two features：

  1. Added support for both Y10 and Y12 data formats for Mono cameras.
  2. Added support for trigger mode.

3.1 Description of raw data image format

On TX2 and XAVIER, 10 bit depth and 12 bit depth raw data is stored in memory in a format that is not compliant with the V4L2 standard. We have extended the definition in the linux kernel to support this case.

TY10, TY12, XY10 and XY12 are the new types we have added.

3.1.1 Nano
Depth Bit order FourCC Enumerator
8 B7 B6 B5 B4 B3 B2 B1 B0 GREY V4L2_PIX_FMT_GREY
10 0 0 0 0 0 0 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 'Y10 ' V4L2_PIX_FMT_Y10
12 0 0 0 0 B11 B10 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 'Y12 ' V4L2_PIX_FMT_Y12
3.1.2 TX2
Depth Bit order (X is undefined) FourCC Enumerator
8 B7 B6 B5 B4 B3 B2 B1 B0 GREY V4L2_PIX_FMT_GREY
10 0 0 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X X X TY10 V4L2_PIX_FMT_TX2_Y10
12 0 0 B11 B10 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X TY12 V4L2_PIX_FMT_TX2_Y12
3.1.3 XAVIER
Depth Bit order (X is undefined) FourCC Enumerator
8 B7 B6 B5 B4 B3 B2 B1 B0 GREY V4L2_PIX_FMT_GREY
10 0 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X X X X XY10 V4L2_PIX_FMT_XAVIER_Y10
12 0 B11 B10 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X X XY12 V4L2_PIX_FMT_XAVIER_Y12

Also, we provide tool software in order to convert these special data formats to standard formats: https://github.com/veyeimaging/pixel_layer_convert.

We recommend using vooya as the player.

3.1.4 Support for trigger mode

The default driver for Jetson systems only supports video streaming mode. In its VI driver, the data reception function has a timeout mechanism. We have added a settable vi_time_out_disable option to dynamically turn this timeout mechanism on and off.

Refer to the following application example for specific applications.

4 Check system status

After system update, reboot the Jetson board.

During the Jetson system boot process, it detects the presence of cameras on all i2c buses and generates the /dev/videoX device node if it exists.

Execute the following command on the Jetson board to check if the camera is properly connected.

dmesg | grep mvcam

You can see the camera model and the camera version number probed.

A prompt as below indicates that the MV-MIPI-IMX178M camera is detected on the i2c-10 bus.

mvcam 10-003b: camera is: MV-MIPI-IMX178M

mvcam 10-003b: firmware version: 0x1080103

4.1 /dev/videoX node

The camera module is mapped as /dev/videoX device node in the Jetson system.

During the OS boot process, the cameras are detected in the order of the i2c bus. The X value is incremented from 0 according to the logical order of detection.

For instance, if only one camera is connected, X is 0 regardless of the location to which the hardware is connected. If 5 cameras are connected, X is [0-4] according to i2c bus from smallest to largest.

The v4l2-ctl command uses -d /dev/videoX to access different cameras.

In gstreamer, v4l2src can access different cameras by specifying device=/dev/videoX.

5 v4l2-ctl Application Examples

5.1 Install v4l2-utils

sudo apt-get install v4l-utils

5.2 Install yavta

git clone git://git.ideasonboard.org/yavta.git

cd yavta;make

5.3 Configure parameters using v4l2-ctl
5.3.1 List the data formats supported by the camera

v4l2-ctl --list-formats-ext

The following is an example in the XAVIER system：

ioctl: VIDIOC_ENUM_FMT

        Index       : 0

        Type        : Video Capture

        Pixel Format: 'UYVY'

        Name        : UYVY 4:2:2

                Size: Discrete 3088x2064

                        Interval: Discrete 0.045s (22.000 fps)

        Index       : 1

        Type        : Video Capture

        Pixel Format: 'NV16'

        Name        : Y/CbCr 4:2:2

                Size: Discrete 3088x2064

                        Interval: Discrete 0.045s (22.000 fps)

        Index       : 2

        Type        : Video Capture

        Pixel Format: 'GREY'

        Name        : 8-bit Greyscale

                Size: Discrete 3088x2064

                        Interval: Discrete 0.045s (22.000 fps)

        Index       : 3

        Type        : Video Capture

        Pixel Format: 'XY10'

        Name        : XAVIER 10-bit/16-bit Greyscale

                Size: Discrete 3088x2064

                        Interval: Discrete 0.045s (22.000 fps)

        Index       : 4

        Type        : Video Capture

        Pixel Format: 'XY12'

        Name        : XAVIER 12-bit/16-bit Greyscale

                Size: Discrete 3088x2064

                        Interval: Discrete 0.045s (22.000 fps)

It should be noted that the UYVY and NV16 modes provided by the MV series cameras are for debugging purposes only and the maximum width cannot exceed 2880.

5.3.2 List the configurable parameters of the camera implemented in the driver

v4l2-ctl -L

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0

                  vertical_flip 0x00980915 (bool)   : default=0 value=0

                   trigger_mode 0x00981901 (int)    : min=0 max=2 step=1 default=0 value=0 flags=volatile, execute-on-write

                    trigger_src 0x00981902 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write

                    soft_trgone 0x00981903 (button) : flags=write-only, execute-on-write

                     frame_rate 0x00981904 (int)    : min=1 max=22 step=1 default=22 value=22 flags=volatile, execute-on-write

                          roi_x 0x00981905 (int)    : min=0 max=2712 step=8 default=0 value=0

                          roi_y 0x00981906 (int)    : min=0 max=1744 step=4 default=0 value=0

Camera Controls

low_latency_mode 0x009a206d (bool)   : default=0 value=0

               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0

            vi_time_out_disable 0x009a2078 (bool)   : default=0 value=0

Parameters can be set and get using the following methods.

v4l2-ctl --set-ctrl [ctrl_type]=[val]

v4l2-ctl --get-ctrl [ctrl_type]

All the above functions can be implemented using mv_mipi_i2c.sh.

None of the above parameters can be modified in the state when the image acquisition is started.

Each of them is described below:

5.3.3 horizontal and vertical flip
  • horizontal flip

v4l2-ctl --set-ctrl horizontal_flip=1

  • vertical flip

v4l2-ctl --set-ctrl vertical_flip=1

5.3.4 Trigger Mode

v4l2-ctl --set-ctrl trigger_mode=[0-2]

0：Video streaming mode

1：Normal trigger mode.

2：High-speed continuous trigger mode.

5.3.5 Trigger Source

v4l2-ctl --set-ctrl trigger_src=[0-1]

0: Software trigger mode.

1: Hardware trigger mode.

5.3.6 Software trigger command

v4l2-ctl --set-ctrl soft_trgone=1

5.3.7 Set frame rate

v4l2-ctl --set-ctrl frame_rate=[1-max]

The maximum frame rate is automatically updated as the resolution changed.

5.3.8 Set ROI and pixel format

For example, for MV-MIPI-IMX178M:

v4l2-ctl --set-ctrl roi_x=0

v4l2-ctl --set-ctrl roi_y=0

v4l2-ctl --set-fmt-video=width=3088,height=2064,pixelformat=GREY

The maximum frame rate will be adjusted automatically after setting ROI. Please note that the camera roi parameters need to comply with the requirements in the camera manual.

5.3.9 preferred_stride


Retrieved from "http://wiki.veye.cc/index.php?title=Mv_series_camera_appnotes_4_jetson&oldid=5475"