Difference between revisions of "Mv series camera appnotes 4 jetson"
| Line 136: | Line 136: | ||
This section describes how to upgrade the Jetson system to support MV camera module. For OS update method, please refer to [[VEYE CS Camera for Jetson TX2#Upgrade Jetson Nano.2C Jetson TX2.2C TX2 NX.2CAGX Xavier and Xavier NX system|Update Jetson OS]]. | This section describes how to upgrade the Jetson system to support MV camera module. For OS update method, please refer to [[VEYE CS Camera for Jetson TX2#Upgrade Jetson Nano.2C Jetson TX2.2C TX2 NX.2CAGX Xavier and Xavier NX system|Update Jetson OS]]. | ||
| − | Specially, for MV series camera, besides adding the camera driver in | + | Specially, for MV series camera, besides adding the camera driver in Linux system, we also add a kernel patch - veye_mv_l4t_32.7.1.patch. |
This patch has two features： | This patch has two features： | ||
| Line 224: | Line 224: | ||
After system update, reboot the Jetson board. | After system update, reboot the Jetson board. | ||
| − | During the Jetson system boot process, it detects the presence of cameras on all i2c buses and generates the /dev/ | + | During the Jetson system boot process, it detects the presence of cameras on all i2c buses and generates the /dev/videoX device node if it exists. |
Execute the following command on the Jetson board to check if the camera is properly connected. | Execute the following command on the Jetson board to check if the camera is properly connected. | ||
| Line 230: | Line 230: | ||
<code>dmesg | grep mvcam</code> | <code>dmesg | grep mvcam</code> | ||
| − | You can see the camera model | + | You can see the camera model and the camera version number probed. |
A prompt as below indicates that the MV-MIPI-IMX178M camera is detected on the i2c-10 bus. | A prompt as below indicates that the MV-MIPI-IMX178M camera is detected on the i2c-10 bus. | ||
| Line 237: | Line 237: | ||
<code>mvcam 10-003b: firmware version: 0x1080103</code> | <code>mvcam 10-003b: firmware version: 0x1080103</code> | ||
| + | |||
| + | ==== /dev/videoX node ==== | ||
| + | The camera module is mapped as /dev/videoX device node in the Jetson system. | ||
| + | |||
| + | During the OS boot process, the cameras are detected in the order of the i2c bus. The X value is incremented from 0 according to the logical order of detection. | ||
| + | |||
| + | For instance, if only one camera is connected, X is 0 regardless of the location to which the hardware is connected. If 5 cameras are connected, X is [0-4] according to i2c bus from smallest to largest. | ||
| + | |||
| + | The v4l2-ctl command uses -d /dev/videoX to access different cameras. | ||
| + | |||
| + | In gstreamer, v4l2src can access different cameras by specifying device=/dev/videoX. | ||
| + | |||
<br /> | <br /> | ||
Revision as of 17:06, 18 May 2022
1 Overview
The MV series cameras are cameras introduced for AI applications in the industrial field. It uses the MIPI CSI-2 interface, which is especially suitable for embedded computing platforms.
It features rich data formats and trigger characteristics, extremely low latency, extremely high bandwidth and reliable stability.
This article describes how to use the MV series camera on the NVIDIA Jetson platform.
1.1 Camera Module List
| Series | Model | Status |
|---|---|---|
| MV series | MV-MIPI-IMX178M | Done |
1.2 Jetson Board List
| Jetson Board | Status |
|---|---|
| Nano A02 | Done |
| Nano B01 | Done |
| Nano 2GB | Done |
| TX2 NX | Done |
| XAVIER NX | Done |
| TX2 Devkit | Coming soon |
| AGX Xavier | Coming soon |
| Orin Series | Coming soon |
1.3 Supported L4T versions
- Jetpack4.6.1,L4T r32.7.1
1.3.1 How to check the current L4T version
On Jetson board
cat /etc/nv_tegra_release
If it shows:
# R32 (release), REVISION: 7.1......
It means L4t Version is 32.7.1, and the Jetpack version is 4.6.1.
2 Hardware Setup
MV series cameras require an adapter board to access the Jetson platform. The following table shows the support status.
| Jetson Board | adapter board | camera number | Power |
|---|---|---|---|
| Nano A02 | ADP-MV1 | 1 | 5V DC（Required） |
| Nano B01 | ADP-MV1 | 1 | 5V DC（Required） |
| Nano 2GB | ADP-MV1 | 2 | 5V DC（Required） |
| TX2 NX | ADP-MV1 | 2 | 5V DC（Required） |
| XAVIER NX | ADP-MV1 | 2 | 5V DC（Required） |
| TX2 Devkit | ADP-N4 | 6 | 12V DC (Optional) |
| AGX Xavier | ADP-N4 | 6 | 12V DC (Optional) |
| Orin series | TODO | TODO | TODO |
2.1 Connection of MV-MIPI-IMX178M and ADP-MV1
The two are connected using 0.5 mm pitch*30P FFC cable with opposite direction. The cable must be inserted with the silver contacts facing outside.
2.2 Connection using ADP-MV1 (using Xavier NX as an example)
2.2.1 Power supply
The ADP-MV1 requires a separate 5V power supply and can be powered directly from the Jetson board using a Dupont cable.
2.3 Connection using ADP-MV1
The Nano Series and NX Series are connected in this way.
2.4 Connection using ADP-N4
The AGX Series and TX2 Devkit are connected in this way.
TODO
3 Upgrade Jetson system
This section describes how to upgrade the Jetson system to support MV camera module. For OS update method, please refer to Update Jetson OS.
Specially, for MV series camera, besides adding the camera driver in Linux system, we also add a kernel patch - veye_mv_l4t_32.7.1.patch.
This patch has two features：
- Added support for both Y10 and Y12 data formats for Mono cameras.
- Added support for trigger mode.
3.1 Description of raw data image format
On TX2 and XAVIER, 10 bit depth and 12 bit depth raw data is stored in memory in a format that is not compliant with the V4L2 standard. We have extended the definition in the linux kernel to support this case.
TY10, TY12, XY10 and XY12 are the new types we have added.
3.1.1 Nano
| Depth | Bit order | FourCC | Enumerator |
|---|---|---|---|
| 8 | B7 B6 B5 B4 B3 B2 B1 B0 | GREY | V4L2_PIX_FMT_GREY |
| 10 | 0 0 0 0 0 0 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 | 'Y10 ' | V4L2_PIX_FMT_Y10 |
| 12 | 0 0 0 0 B11 B10 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 | 'Y12 ' | V4L2_PIX_FMT_Y12 |
3.1.2 TX2
| Depth | Bit order (X is undefined) | FourCC | Enumerator |
|---|---|---|---|
| 8 | B7 B6 B5 B4 B3 B2 B1 B0 | GREY | V4L2_PIX_FMT_GREY |
| 10 | 0 0 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X X X | TY10 | V4L2_PIX_FMT_TX2_Y10 |
| 12 | 0 0 B11 B10 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X | TY12 | V4L2_PIX_FMT_TX2_Y12 |
3.1.3 XAVIER
| Depth | Bit order (X is undefined) | FourCC | Enumerator |
|---|---|---|---|
| 8 | B7 B6 B5 B4 B3 B2 B1 B0 | GREY | V4L2_PIX_FMT_GREY |
| 10 | 0 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X X X X | XY10 | V4L2_PIX_FMT_XAVIER_Y10 |
| 12 | 0 B11 B10 B9 B8 B7 B6 B5 B4 B3 B2 B1 B0 X X X | XY12 | V4L2_PIX_FMT_XAVIER_Y12 |
Also, we provide tool software in order to convert these special data formats to standard formats: https://github.com/veyeimaging/pixel_layer_convert.
3.1.4 Support for trigger mode
The default driver for Jetson systems only supports video streaming mode. In its VI driver, the data reception function has a timeout mechanism. We have added a settable vi_time_out_disable option to dynamically turn this timeout mechanism on and off.
Refer to the following application example for specific applications.
4 Check system status
After system update, reboot the Jetson board.
During the Jetson system boot process, it detects the presence of cameras on all i2c buses and generates the /dev/videoX device node if it exists.
Execute the following command on the Jetson board to check if the camera is properly connected.
dmesg | grep mvcam
You can see the camera model and the camera version number probed.
A prompt as below indicates that the MV-MIPI-IMX178M camera is detected on the i2c-10 bus.
mvcam 10-003b: camera is: MV-MIPI-IMX178M
mvcam 10-003b: firmware version: 0x1080103
4.1 /dev/videoX node
The camera module is mapped as /dev/videoX device node in the Jetson system.
During the OS boot process, the cameras are detected in the order of the i2c bus. The X value is incremented from 0 according to the logical order of detection.
For instance, if only one camera is connected, X is 0 regardless of the location to which the hardware is connected. If 5 cameras are connected, X is [0-4] according to i2c bus from smallest to largest.
The v4l2-ctl command uses -d /dev/videoX to access different cameras.
In gstreamer, v4l2src can access different cameras by specifying device=/dev/videoX.