Mv series camera appnotes 4 rpi

查看中文

1 Overview

The MV series cameras are cameras introduced for AI applications in the industrial field. It uses the MIPI CSI-2 interface, which is especially suitable for embedded computing platforms.

It features rich data formats and trigger characteristics, extremely low latency, extremely high bandwidth and reliable stability.

This article describes how to use the MV series camera on the Raspberry Pi platform.

1.1 Camera module list

Series	Model	Status
MV series	MV-MIPI-IMX178M	Done

2 Hardware Setup

MV series cameras need to use ADP-MV1 adapter board in order to connect to Raspberry Pi.

We take MV-MIPI-IMX178M as an example to introduce the hardware installation method.

2.1 Connection of MV-MIPI-IMX178M and ADP-MV1

The two are connected using 0.5 mm pitch*30P FFC cable with opposite direction. The cable must be inserted with the silver contacts facing outside.

ADP-MV1 to MV-MIPI-X

2.2 Connection of ADP-MV1 to Raspberry Pi

2.2.1 Power supply

The ADP-MV1 requires a separate 5V power supply and can be powered directly from the Raspberry Pi motherboard using a Dupont cable.

ADP-MV1 power supply

2.2.2 Raspberry Pi Model B and B+

The two are connected using 1.0 mm pitch*15P FFC cable with opposite direction. Pay attention to the silver contacts facing side.

MV camera and RPI connection

2.2.3 Raspberry Pi Zero, Zero W and Computer Module

The two are connected using 15Pto22P FFC cable with same direction. Pay attention to the silver contacts facing side.

MV camera and Raspberry Pi zero connection

3 piOS Configuration

For details on how to install the Raspberry Pi system, please refer to the official documentation at Install raspberry pi guide.

Initially, the Raspberry Pi system I2C is not enabled. After booting, we need to turn them on manually by executing the following command:

sudo raspi-config

raspi-config bullseye

Enter Interface Options, enable I2C, and then restart the Raspberry Pi.

It is recommended to enable the ssh service and samba service of Raspberry Pi system, here we will not go into the details of how to enable ssh and samba service of Raspberry Pi system.

4 Legacy mode and V4L2 mode introduction

The difference between these two modes is described in detail on the Raspberry Pi website. The libcamera-stack mode mentioned on the Raspberry Pi website is architecturally identical to the V4L2 mode we are talking about.

4.1 Legacy mode

Traditional mode, relying on Broadcom's GPU for image processing. The traditional raspicam software set uses this model.

The disadvantage of this model is closed. The GPU side is closed source, can not freely access sensor.

The Raspberry Pi organization has switched to the libcamera camera stack, but this model still has value:

1. More use of GPU resources and lower CPU load. This is useful for earlier versions of the Raspberry Pi where CPU performance is poorer.
2. It is possible to simply and directly fetch image data to the application layer without the support of the driver layer. This is especially useful for cameras that do not rely on the Raspberry Pi's isp.
3. libcamera lacks some feature support now.

There are two ways to use Legacy mode.

1. Use the Legacy version of piOS.
2. For the bullseye or newer version of piOS, turn on the Legacy Camera option in raspi-config.

Since the two modes cannot co-exist, Legacy mode needs to be turned off when using V4L2 mode.

4.2 libcamera and V4L2 mode

Now piOS has switched to libcamera-stack mode.

Libcamera is essentially centered on implementing isp functionality, something that is not needed for the MV series cameras. Therefore, we have adopted the V4L2 mode instead of using libcamera-stack.

As with libcamera-stack, our V4L2 model implements the standard V4L2 driver for the linux driver layer too. Based on this driver, the application layer can directly develop programs to acquire images and perform further processing.

5 V4L2 mode demo

We have saved the code for v4l2 mode in this github repository.

5.1 Download the driver package

wget https://github.com/veyeimaging/raspberrypi_v4l2/releases/latest/download/raspberrypi_v4l2.tgz

5.2 Install driver

tar -xzvf raspberrypi_v4l2.tgz

cd raspberrypi_v4l2/release/

chmod +x *

sudo ./install_driver.sh veye_mvcam

Then reboot Raspberry Pi.

Note: If you are prompted that the corresponding version of the driver cannot be found, it means that we do not provide a compiled driver corresponding to your piOS version. Please try to compile from the source code.

5.3 Uninstall the driver

If you need to change to Legacy mode, or want to change to another camera module model driver, you must uninstall the current driver first.

sudo ./uninstall_driver.sh veye_mvcam

5.4 Check system status

Take MV-MIPI-IMX178M as an example.

dmesg | grep mvcam

You can see the camera model and the version of the camera probed during the Linux boot phase.

camera is: MV-MIPI-IMX178M

firmware version: 0x1080102

And the /dev/video0 node exists, which proves that the camera status is normal.

5.5 v4l2-ctl application examples

5.5.1 Install v4l2-utils

sudo apt-get install v4l-utils

5.5.2 Install yavta

git clone git://git.ideasonboard.org/yavta.git

cd yavta;make

5.5.3 Configure parameters using v4l2-ctl

5.5.3.1 List the data formats supported by the camera

v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT

        Type: Video Capture

        [0]: 'GREY' (8-bit Greyscale)

                Size: Discrete 3088x2064

        [1]: 'Y10P' (10-bit Greyscale (MIPI Packed))

                Size: Discrete 3088x2064

        [2]: 'Y10 ' (10-bit Greyscale)

                Size: Discrete 3088x2064

        [3]: 'Y12P' (12-bit Greyscale (MIPI Packed))

                Size: Discrete 3088x2064

        [4]: 'Y12 ' (12-bit Greyscale)

                Size: Discrete 3088x2064

        [5]: 'UYVY' (UYVY 4:2:2)

                Size: Discrete 3088x2064

Note: UYVY data format is for debugging use only.

5.5.3.2 List the configurable parameters of the camera implemented in the driver

v4l2-ctl -L

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0

                  vertical_flip 0x00980915 (bool)   : default=0 value=0

                   trigger_mode 0x00981901 (int)    : min=0 max=2 step=1 default=0 value=0 flags=volatile

                    trigger_src 0x00981902 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile

                    soft_trgone 0x00981903 (button) : flags=write-only, execute-on-write

                     frame_rate 0x00981904 (int)    : min=0 max=22 step=1 default=22 value=22 flags=volatile

Parameters can be set and get using the following methods.

v4l2-ctl --set-ctrl [ctrl_type]=[val]

v4l2-ctl --get-ctrl [ctrl_type]

All the above functions can be implemented using mv_mipi_i2c.sh.

None of the above parameters can be modified in the state when the image acquisition is started.

Each of them is described below:

5.5.3.3 horizontal and vertical flip

• horizontal flip

v4l2-ctl --set-ctrl horizontal_flip=1

• vertical flip

v4l2-ctl --set-ctrl vertical_flip=1

5.5.3.4 Trigger Mode

v4l2-ctl --set-ctrl trigger_mode=[0-2]

0：Video streaming mode

1：Normal trigger mode.

2：High-speed continuous trigger mode.

5.5.3.5 Trigger Source

v4l2-ctl --set-ctrl trigger_src=[0-1]

0: Software trigger mode.

1: Hardware trigger mode.

5.5.3.6 Software trigger command

v4l2-ctl --set-ctrl soft_trgone=1

5.5.3.7 Set frame rate

v4l2-ctl --set-ctrl frame_rate=[1-max]

The maximum frame rate is automatically updated as the resolution changed.

5.5.3.8 Set ROI

For example, for MV-MIPI-IMX178M:

v4l2-ctl --set-selection=target=crop,top=32,left=64,width=2592,height=1944

The maximum frame rate will be adjusted automatically after setting ROI.

5.6 Video Streaming mode

5.6.1 Set ROI

Take MV-MIPI-IMX178M, 3088*2064 as an example.

v4l2-ctl --set-selection=target=crop,top=0,left=0,width=3088,height=2064

5.6.2 Frame rate statistics

In streaming mode, the following commands can be used for frame rate statistics.

v4l2-ctl --set-fmt-video=width=3088,height=2064,pixelformat=GREY --stream-mmap --stream-count=-1 --stream-to=/dev/null

5.6.3 Save image to file

v4l2-ctl --set-fmt-video=width=3088,height=2064,pixelformat=GREY --stream-mmap --stream-count=1 --stream-to=y8-3104x2064.yuv

Or

./yavta -c1 -Fy8-3104x2064.yuv --skip 0 -f Y8 -s 3088x2064 /dev/video0

PS. This y8 file can be used with this player: YUV Displayer Deluxe.

Since the memory requested by Raspberry Pi for the image, the width is 32-align and the height is 16-align, the 3088*2064 image will be saved as 3104*2064 size.

5.7 Trigger mode

5.7.1 Set ROI

Take MV-MIPI-IMX178M, 3088*2064 as an example.

v4l2-ctl --set-selection=target=crop,top=0,left=0,width=3088,height=2064

5.7.2 Software trigger mode

5.7.2.1 Set mode

v4l2-ctl --set-ctrl trigger_mode=1

v4l2-ctl --set-ctrl trigger_src=0

5.7.2.2 Start acquisition

v4l2-ctl --set-fmt-video=width=3088,height=2064,pixelformat=GREY --stream-mmap --stream-count=5 --stream-to=y8-3104x2064.yuv

5.7.2.3 Perform soft trigger operation

In other shell terminals, you can execute the following command multiple times for multiple triggers.

v4l2-ctl --set-ctrl soft_trgone=1

5.7.3 Hardware trigger mode

The following is an example of using Raspberry Pi GPIO21 as an trigger source with rising edge trigger.

You can use the mv_mipi_i2c.sh script for other trigger parameter setting.

5.7.3.1 Hardware Connection

MV camera hardware trigger connection

5.7.3.2 Set mode

v4l2-ctl --set-ctrl trigger_mode=1

v4l2-ctl --set-ctrl trigger_src=1

5.7.3.3 Start acquisition

v4l2-ctl --set-fmt-video=width=3088,height=2064,pixelformat=GREY --stream-mmap --stream-count=5 --stream-to=y8-3104x2064.yuv

5.7.3.4 Perform hardware trigger operation

python gpio_trigger.py

Note: script link.

5.8 Compiling drivers from source code

Please refer to: Build drivers from source for rpi.

6 Legacy mode manual

We store the code for legacy patterns in this github repository.

6.1 Download the software package

git clone https://github.com/veyeimaging/mvcam_raspberrypi.git

Add executable permissions:

chmod +x -R ./mvcam_raspberrypi/

6.2 Software package main content introduction

camera_i2c_config：Automatically recognizes different Raspberry Pi board types and controls the power on pins to power the camera.

mv_raspicam：Realize image preview, image saving and other functions.

mv_mipi_i2c.sh：Scripts for reading and setting camera parameters via i2c protocol.

veye_i2c_upgrade：Camera firmware upgrade tool.

6.3 Camera power on

./camera_i2c_config

When the camera is powered on, try:

i2cdetect -y 10

The presence of a 0x3b device indicates that the camera is on and the i2c channel is working properly.

6.4 Preview

./mv_raspicam -t -1 -roi 0,0,3088,2064 -md 0

where -md specifies the pixel data format.

-md option	data type
0	8bit Y
1	10bit Y
2	12bit Y

-roi specifies the desired ROI size.

For sizes larger than 1920 in width, mv_raspicam will first reduce to half the original size of the image and then send to render.

6.5 Save image

./mv_raspicam -t 1000 -roi 0,0,3088,2064 -md 0 -sr 10 -o /dev/shm/y8-3104x2064_%04d.yuv

-sr is the interval for storing images, one image is saved every 10 images here.

-t 1000, in ms, it means 1 second here.

Several files will be saved in the /dev/shm/ directory after completion.

6.6 Trigger mode

This section requires the use of mv_mipi_i2c.sh, please refer to the related i2c parameter configuration section below.

6.6.1 Software trigger mode

6.6.1.1 Set mode

./mv_mipi_i2c.sh -w -f trgmode -p1 1

./mv_mipi_i2c.sh -w -f trgsrc -p1 0

6.6.1.2 Set trigger properties

You can set the number of triggers, trigger interval, etc.

The following settings are for a single trigger of 5 images with an interval of 1 second each.

./mv_mipi_i2c.sh -w -f trgnum -p1 5

./mv_mipi_i2c.sh -w -f trginterval -p1 1000

6.6.1.3 Start acquisition

./mv_raspicam -t -1 -roi 0,0,3088,2064 -md 0

6.6.1.4 Perform soft trigger operation

In other shell terminals, you can execute the following command multiple times for multiple triggers.

./mv_mipi_i2c.sh -w -f trgone

6.6.2 Hardware trigger mode

The following is an example of using Raspberry Pi GPIO21 as an trigger source with rising edge trigger.

You can use the mv_mipi_i2c.sh script for other trigger parameter setting.

6.6.2.1 Hardware Connection

MV camera hardware trigger connection

6.6.2.2 Set mode

This time, we use high-speed continuous trigger as an example.

./mv_mipi_i2c.sh -w -f trgmode -p1 2

./mv_mipi_i2c.sh -w -f trgsrc -p1 1

6.6.2.3 Set trigger properties

The trigger interval is invalid under high-speed continuous triggering. This time trigger 2 images.

./mv_mipi_i2c.sh -w -f trgnum -p1 2

Set rising edge trigger

./mv_mipi_i2c.sh -w -f trgedge -p1

In addition, rich trigger attributes can be set for hard triggering, such as trigger delay, signal filtering, etc.

6.6.2.4 Start acquisition

./mv_raspicam -t -1 -roi 0,0,3088,2064 -md 0

6.6.2.5 Perform hardware trigger operation

python gpio_trigger.py

Note: script link.

6.8 Unpacked image format(padded)

For 10bit depth and 12bit depth pixel data, two bytes are always used to store one pixel. The V4L2 standard 'Y10 ', 'Y12 ' formats are stored in this way.

This storage method is convenient for software processing, but has the disadvantage of taking up more space.

6.9 Packed image format

The data format saved by mv_raspicam is the data format on the MIPS bus, with no empty bit padding between pixels.

Packed raw data layer

6.10 Packed image size

Buffersize= ALIGN_UP（width*bit_depth/8,32）*ALIGN_UP(height,16)

The row buffer size should be 32 bytes aligned; the column buffer size should be 16 bytes aligned.

For example, the image size of 3088*2064@8bit is 6406656; the image size of 3088*2064@10bit is 7991808; the image size of 3088*2064@12bit is 9576960.

6.11 Format convert tool

We have written a small tool: pixel_layer_convert, which can easily convert packed images to unpacked images.

For example, the following command converts a packed raw10 image of 3088 width to an unpacked raw10.

. /pixel_layer_convert -I Y10P -i y10-3088x2064_0001.raw -o y10-3088x2064_0001_new.raw -w 3088

6.12 Raw data image player

We recommend using vooya as the player, which supports GREY, and unpacked image formats.

Also, y8 file can be used with this player: YUV Displayer Deluxe.

Since the memory requested by Raspberry Pi for the image, the width is 32-align and the height is 16-align, the 3088*2064 image will be saved as 3104*2064 size.

7 Shell script for parameter configuration

We provide shell scripts to configure the parameters.

mv_mipi_i2c.sh user guide

8 References

• Installation of the Raspberry Pi system
• The difference between Raspberry Pi legacy camera stack and libcamera mode

9 Document History

• 2022-05-16

Provide image format convert tool.

• 2022-04-14

Release 1st version.