Difference between revisions of "V4L2 mode for Raspberry Pi"

From wiki_veye
Jump to navigation Jump to search
(60 intermediate revisions by 2 users not shown)
Line 9: Line 9:
  
 
This article describes how to call the camera modules of VEYE series and CS series through V4L2 on the raspberry Pi platform.
 
This article describes how to call the camera modules of VEYE series and CS series through V4L2 on the raspberry Pi platform.
 +
==== Camera module list====
 +
{| class="wikitable"
 +
|+Camera module dirver status list
 +
!Series
 +
! Model
 +
! Status
 +
|-
 +
|VEYE series
 +
|VEYE-MIPI-IMX327S
 +
|Done
 +
|-
 +
|VEYE series
 +
|VEYE-MIPI-IMX385
 +
|Done
 +
|-
 +
|VEYE series
 +
|VEYE-MIPI-IMX462
 +
|Done
 +
|-
 +
|CS series
 +
|CS-MIPI-IMX307
 +
|Done
 +
|-
 +
|CS series
 +
|CS-MIPI-SC132
 +
|Done
 +
|}
 +
In addition, FPD-Link3 mode connection is now supported.
  
 
=== Hardware Setup ===
 
=== Hardware Setup ===
Line 19: Line 47:
  
 
=== Driver Installation ===
 
=== Driver Installation ===
 +
We have saved the code for v4l2 driver mode in this [https://github.com/veyeimaging/raspberrypi_v4l2 github repository].
  
 
==== Download the driver ====
 
==== Download the driver ====
−
<code>git clone <nowiki>https://github.com/veyeimaging/raspberrypi_v4l2.git</nowiki></code>
+
<code>wget <nowiki>https://github.com/veyeimaging/raspberrypi_v4l2/releases/latest/download/raspberrypi_v4l2.tgz</nowiki></code>
  
 
==== Install the driver ====
 
==== Install the driver ====
 +
<code>tar -xzvf raspberrypi_v4l2.tgz</code>
 +
 
<code>cd raspberrypi_v4l2/release/</code>
 
<code>cd raspberrypi_v4l2/release/</code>
  
 
<code>chmod +x *</code>
 
<code>chmod +x *</code>
  
−
<code>sudo ./install_driver.sh [camera module]</code>
+
* For Raspberry Pi 5
 +
 
 +
<code>sudo ./install_driver_rpi5.sh camera_module_name</code>
 +
 
 +
It will install both CAM1 and CAM0 overlays in /boot/config.txt.
 +
 
 +
* For other Raspberry Pi
 +
 
 +
<code>sudo ./install_driver.sh camera_module_name</code>
 +
 
 +
camera_module_name：could be veye327,csimx307,cssc132,veyecam2m.
 +
 
 +
'''Note: veyecam2m is a new version driver that can replace veye327 and supports all modules of the VEYE series with 200W resolution.'''
 +
 
 +
'''please use veyecam2m instead of veye327 which will be discarded.'''
 +
 
 +
Note: For VEYE-MIPI-IMX327S or VEYE-MIPI-IMX462, please make sure '''hdver''' must >= 0x5.
  
−
camera module：could be veye327,csimx307,cssc132.
+
http://wiki.veye.cc/index.php/VEYE-MIPI-IMX327S_version_log
  
 
==== Uninstall the driver ====
 
==== Uninstall the driver ====
 
If you need to change to using the camera by veye_raspicam software, or if you want to change to a driver of another camera module model, you must first uninstall the current driver.
 
If you need to change to using the camera by veye_raspicam software, or if you want to change to a driver of another camera module model, you must first uninstall the current driver.
  
−
<code>sudo ./uninstall_driver.sh [camera module]</code>
+
<code>sudo ./uninstall_driver.sh camera_module_name</code>
 +
 
 +
camera_module_name：could be veye327,csimx307,cssc132,veyecam2m.
  
−
camera module：could be veye327,csimx307,cssc132.
+
==== Camera in FPD-Link3 Mode ====
 +
For a camera operating in FPD-Link3 transmission mode, it is necessary to ensure that the ds90ub954 driver is loaded and initialized before loading the camera driver and performing probe.
 +
 
 +
Take the VEYE-MIPI-IMX327S camera in FPD-Link3 mode as an example:
 +
 
 +
Install the drivers correctly:
 +
 
 +
<code>sudo ./install_driver.sh ds90ub954</code>
 +
 
 +
<code>sudo ./install_driver.sh veyecam2m</code>
  
 
=== Check and Test the Camera ===
 
=== Check and Test the Camera ===
Line 73: Line 131:
 
==== List the available video modes ====
 
==== List the available video modes ====
 
<code>v4l2-ctl --list-formats-ext</code>
 
<code>v4l2-ctl --list-formats-ext</code>
 +
===Raspberry Pi 5 Setting===
 +
On Raspberry Pi 5,the drivers now use the media controller API, and we must setup the media graph correctly first. This includes setting up the media pad formats correctly and correctly linking them together.
 +
 +
We provide a series of scripts to implement this functionality, saved in the rpi5_scripts directory.
 +
 +
*./find_entity.sh
 +
 +
<code>$ ./find_entity.sh</code>
 +
 +
<code>Found veyecam2m @ i2c-6 entity on /dev/media0</code>
 +
 +
<code>Plese get frame from /dev/video0 and use /dev/v4l-subdev2 for camera setting.</code>
 +
 +
After a reboot of Raspberry Pi 5, the media node and video node of the camera may change. Therefore, it is recommended to execute <code>./find_entity.sh</code> before performing subsequent operations to identify the device nodes.
 +
 +
The above prompt shows that the system has detected two cameras and their corresponding device nodes.
 +
 +
<code>i2c-4</code> corresponds to the CAM1 port on the board, and <code>i2c-6</code> corresponds to the CAM0 port on the board.
 +
 +
*media_setting_rpi5.sh
 +
 +
<code>./media_setting_rpi5.sh</code>
 +
 +
<code>Usage: ./media_setting_rpi5.sh veyecam2m/csimx307/cssc132/mvcam -fmt [UYVY/RAW8/RAW10/RAW12] -w [width] -h [height]</code>
 +
 +
<code>This shell script is designed to detect the connection of a camera on Raspberry Pi 5.</code>
 +
 +
<code>    It utilizes media-ctl and v4l2-ctl commands to configure the linking relationships and data formats of the media pad.</code>
 +
 +
<code>    Once completed, you can directly use /dev/video0 or /dev/video8 to obtain image data</code>
 +
 +
'''Note: This script must be executed after each startup to complete the configuration of the parameters before proceeding with further operations.'''
 +
 +
For instance:
 +
 +
<code>./media_setting_rpi5.sh veyecam2m</code>
 +
===Preview===
 +
On a raspberry pi 5, VLC for preview is now having problems and not working.
 +
==== Preview using VLC ====
 +
1. Open VLC with command line <code>vlc</code> , or click the icon to launch.
 +
 +
2. Hit the <code>▶</code>(Play) button to call the open media window.
 +
 +
3. In <code>Capture Device</code> >> <code>Device Selection</code> >> <code>Video device name</code>, select the camera video node.
 +
 +
4. Hit <code>Advanced Options...</code> button
 +
 +
5. In Advanced Options window >> <code>Video input chroma format</code>, type <code>UYUV</code>.
 +
 +
6. Type in the <code>width</code> and <code>height</code>, for example, 1920 and 1080.
  
−
=== Gstreamer usage Demo ===
+
7. Hit <code>OK</code> to save the settings and see the video feed.
 +
[[File:Play camera using VLC on RPI.png|alt=Play camera using VLC on RPI|center|thumb|1000x1000px|Play camera using VLC on RPI]]
 +
<br />
 +
======Preview using qv4l2======
 +
Install qv4l2，
 +
 
 +
<code>sudo apt install qv4l2</code>
 +
 
 +
1. Open VLC with command line <code>vlc</code> to launch.
 +
 
 +
2. Hit the <code>▶</code>(Play) button to call the open media window.
 +
=== Gstreamer usage samples ===
 
<code>export DISPLAY=:0</code>
 
<code>export DISPLAY=:0</code>
  
−
* Snap a picture (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
+
==== install gstreamer ====
 +
<code>sudo apt-get install gstreamer1.0-tools</code>
  
 +
<code>sudo apt-get install libx264-dev libjpeg-dev</code>
 +
 +
<code>sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-ugly gstreamer1.0-tools gstreamer1.0-gl gstreamer1.0-gtk3</code>
 +
 +
==== Run gstreamer videotest command ====
 +
<code>gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink</code>
 +
 +
==== Snap a picture (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
 
<code>gst-launch-1.0 v4l2src num-buffers=1 device=/dev/video0 ! 'video/x-raw, format=(string)UYVY, width=1920,height=1080' ! jpegenc ! filesink location=test_image.jpg</code>
 
<code>gst-launch-1.0 v4l2src num-buffers=1 device=/dev/video0 ! 'video/x-raw, format=(string)UYVY, width=1920,height=1080' ! jpegenc ! filesink location=test_image.jpg</code>
  
−
* Preview (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
+
==== Frame rate test (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
 +
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink video-sink=fakesink -v</code>
 +
 
 +
==== Preview (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
 +
<code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code>
 +
 
 +
Or：
  
−
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink sync=false -v</code>
+
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink  sync=false -v</code>
  
−
* Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode)
+
==== Preview and Scale with timestamp on (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
 +
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1"  ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! video/x-raw, width=640, height=360 ! autovideosink sync=false -v</code>
  
 +
==== Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode) ====
 
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720,framerate=(fraction)60/1" ! videoconvert ! autovideosink sync=false -v</code>
 
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720,framerate=(fraction)60/1" ! videoconvert ! autovideosink sync=false -v</code>
  
−
* Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @vga mode)
+
==== Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @vga mode) ====
 +
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)640, height=(int)480" ! videoconvert ! autovideosink sync=false -v</code>
  
−
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)640, height=(int)480" ! videoconvert ! autovideosink sync=false -v</code>
+
==== Preview (CS-MIPI-SC132 @1280*1080@45 mode) ====
 +
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)1080,framerate=(fraction)45/1" ! videoconvert ! autovideosink sync=false -v</code>
 +
 
 +
==== Streaming to file，save as mkv file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ====
 +
<code>gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! matroskamux ! filesink location=output.mkv</code>
 +
 
 +
==== Streaming to file，save as mp4 file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ====
 +
<code>gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! mp4mux ! filesink location=video.mp4</code>
 +
==== TCP streaming====
 +
'''''raspberrypi（sender）'''''
 +
 
 +
<code>gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=-1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc extra-controls="controls, h264_profile=4, video_bitrate=4000000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink  host=x.x.x.x port=5000</code>
 +
 
 +
The bandwidth of the stream is 4Mbps, continuous transmission, and the listening port is 5000.
 +
 
 +
'''''Client'''''
 +
 
 +
<code>gst-launch-1.0 -v tcpclientsrc host=x.x.x.x port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! autovideosink sync=false</code>
 +
 
 +
Where x.x.x.x is the IP address of the Raspberry Pi (Server).
 +
 
 +
It is recommended to use powershell if client is a windows system. '''''gstreamer''''' windows version download [https://gstreamer.freedesktop.org/documentation/installing/on-windows.html?gi-language=c here].
 +
 
 +
=== Stream to OpenCV ===
 +
<code>pip uninstall opencv-python</code>
 +
 
 +
<code>sudo apt install python3-opencv</code>
 +
 
 +
To import camera data from v4l2 devices to opencv, we provide several [https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/samples/opencv samples].
 +
===V4l2-ctl usage examples===
 +
====install v4l2-utils====
 +
<code>sudo apt-get install v4l-utils</code>
 +
====v4l2-ctl test====
  
−
* Preview (CS-MIPI-SC132 @1280*1080@45 mode)
+
*Snap one UYVY raw data picture (1080p mode)
  
−
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)1080,framerate=(fraction)45/1" ! videoconvert ! autovideosink sync=false -v</code>
+
<code>v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv</code>
  
−
* Framerate test (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
+
* Snap 10 UYVY raw data pictures (1080p mode)
  
−
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink video-sink=fakesink -v</code>
+
<code>v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=10 --stream-to=uyvy-1920x1080_stream.yuv</code>
  
−
=== Video Control Toolkits Manual ===
+
=== yavta usage examples ===
−
Because of the high flexibility of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.
 
  
−
<nowiki>https://github.com/veyeimaging/raspberrypi/tree/master/i2c_cmd</nowiki>
+
==== download yavta ====
 +
<code>git clone <nowiki>git://git.ideasonboard.org/yavta.git</nowiki></code>  
  
−
[[VEYE-MIPI-290/327 for Raspberry Pi#Video Control Toolkits Manual|VEYE Series Video Control Toolkits Manual]]
+
<code>cd yavta;make</code>
  
−
[[CS-MIPI-X for Raspberry Pi#Video Control Toolkits Manual|CS Series Video Control Toolkits Manual]]
+
==== yavta test ====
 +
<code>./yavta -c8 -Fuyvy_1920X1080.yuv --skip 0 -f UYVY -s 1920x1080 /dev/video0</code>
  
−
=== Source code cross compilation ===
+
PS, UYVY file can be played by [https://yuv-player-deluxe.software.informer.com/2.6/ YUV Displayer Deluxe].
−
The main resources in this section are the [https://www.raspberrypi.org/documentation/linux/kernel/building.md official piOS building method]. We use the method of cross-compilation, and the 5.4.72 version of 32bitOS is taken as an example.
 
  
−
The following operations are done on ubuntu PC.
+
=== Video Control Toolkits Manual ===
 +
Because of the high flexibility of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.
  
−
==== Build cross-compilation environment on Ubuntu 64-bitoperating system) ====
+
https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/i2c_cmd
−
<code>sudo apt install git bc bison flex libssl-dev make libc6-dev libncurses5-dev</code>
 
  
−
<code>sudo apt install crossbuild-essential-armhf</code>
+
[[VEYE-MIPI-290/327 for Raspberry Pi#Video Control Toolkits Manual|VEYE Series Video Control Toolkits Manual]]
  
−
==== Download the standard version of the piOS source code ====
+
[[CS-MIPI-X for Raspberry Pi#Video Control Toolkits Manual|CS Series Video Control Toolkits Manual]]
  
−
===== Confirm the piOS version of your raspberry Pi =====
+
=== Notes for CM4 ===
 +
Cm4 supports the use of two cameras at the same time. Following above steps will only be able to use CAM1. To use two cameras, follow the following steps:
  
−
* <code>Release version</code>
+
===== Hardware Connection =====
 +
Refer to [https://www.raspberrypi.com/documentation/computers/compute-module.html this Raspberry Pi instruction] ， add jumper to J6.
  
−
<code>$ uname -a</code>
+
==== Upgrade dt-blob.bin ====
 +
<code>sudo wget <nowiki>https://datasheets.raspberrypi.com/cmio/dt-blob-dualcam.bin</nowiki> -O /boot/dt-blob.bin</code>
  
−
<code>Linux raspberrypi 5.4.72-v7l+ #1356 SMP Thu Oct 22 13:57:51 BST 2020 armv7l GNU/Linux</code>
+
==== For Buster OS（kernel5.10） ====
 +
Upgrade dual camera version dtbo file, Use csimx307 as an example.
  
−
* <code>Code tag</code>
+
<code>sudo cp raspberrypi_v4l2/release/driver_bin/$(uname -r)/csimx307-dual-cm4.dtbo /boot/overlays/csimx307.dtbo</code>
  
−
<code>$ cp /usr/share/doc/raspberrypi-bootloader/changelog.Debian.gz ./</code>
+
<code>sudo reboot</code>
  
−
<code>gunzip changelog.Debian.gz</code>
+
==== For Bullseye OS（kernel5.15） ====
 +
Edit file<code>/boot/config.txt</code>，Add one line<code>dtoverlay=[camera],cam0</code>.
  
−
Check the top lines and you will know the tag.
+
Take veyecam2m as an example.
−
===== There are two ways to get the source code of the version you want: =====
 
  
−
# using git to clone the corresponding branch and checkout the corresponding tag.
+
<code>[all]</code>
  
−
<code>git clone --branch rpi-5.4.y <nowiki>https://github.com/raspberrypi/linux</nowiki></code>
+
<code>dtparam=i2c_vc=on</code>
  
−
<code>git checkout raspberrypi-kernel_1.20201022-1</code>
+
<code>dtoverlay=veyecam2m</code>
  
−
2. Manually download the version code of the corresponding tag directly from the link below.
+
<code>dtoverlay=veyecam2m,cam0</code>
  
−
https://github.com/raspberrypi/linux/tags
+
==== Device node description ====
 +
The CM4 module uses two I2C channels to communicate with the two cameras, respectively.
 +
{| class="wikitable"
 +
! description
 +
!i2c bus num
 +
!video node
 +
|-
 +
|CAM0
 +
|0
 +
|video0
 +
|-
 +
|CAM1
 +
|10
 +
|video2（Buster），video1（Bullseye）
 +
|}
 +
Note: If there is only one camera, video node is always video0.
  
−
==== Patch our  code to kernel ====
+
==== Gstreamer usage samples ====
 +
<code>export DISPLAY=:0</code>
  
−
*Driver source code
+
*CAM0 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
  
−
camera driver path is ： linux/drivers/media/i2c，copy our camera module drivers to this path.
+
<code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code>
  
−
*Modify the the Makefile and Kconfig files
+
*CAM1 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
  
−
Modify the Config and Makefile in the same path，add the corresponding camera driver.
+
<code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code>
 +
=== Source code compilation ===
 +
Please refer to: [[Build drivers from source for rpi]].
  
−
*dts file
+
=== Document History ===
  
−
dts file path is ：linux/arch/arm/boot/dts/overlays，copy our [camera]-overlay.dts files to this path.
+
* 2024-01-01
  
−
*Modify dts Makefile
+
Add support for Raspberrypi 5.
  
−
Modify the Makefile in the same path，add the corresponding dts compilation option.
+
* 20230607
  
−
==== Building ====
+
Add FPD-Link3 Driver and Its Description.
  
−
==== Preparing ====
+
* 20230304
−
<code>export ARCH=arm</code>
 
  
−
<code>export CROSS_COMPILE=arm-linux-gnueabihf-</code>
+
Add the instructions related to CM4 in Bullseye system.
  
−
* Raspberry Pi 1, Pi Zero, Pi Zero W, and Compute Module default build configuration
+
* 20220706
  
−
<code>KERNEL=kernel</code>
+
To optimize gstreamer's pipeline, it is recommended to use v4l2convert instead of videoconvert and try to use the capture-io-mode output-io-mode io-mode option.
  
−
<code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcmrpi_defconfig</code>
+
* 20220505
  
−
* Raspberry Pi 2, Pi 3, Pi 3+, and Compute Module 3 default build configuration
+
Refine the description of CM4.
  
−
<code>KERNEL=kernel7</code>
+
*20220411
  
−
<code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig</code>
+
The source code compilation method is separate to a separate article.
  
−
* Raspberry Pi 4 default build configuration
+
* 20220302
  
−
<code>KERNEL=kernel7l</code>
+
Add v4l2-ctl and yavta examples.
  
−
<code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2711_defconfig</code>
+
* 20220301
  
−
==== Add compilation options ====
+
Support 64-bit piOS
−
<code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig</code>
 
  
−
Add the compilation options by the corresponding camera module, for 5.4 version kernel the path is driver-- > multimedia-- > i2C.
+
* 20220225
  
−
For 5.10 version kernel,the path is Device Drivers --> Multimedia Support --> Media ancillary drivers --> Camera sensor devices.
+
Add the application samples of gstreamer.
  
−
===== Build =====
+
*20211125
−
<code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs -j4</code>
 
  
−
=== References ===
+
Add VLC preview section.<br />
−
<nowiki>https://www.raspberrypi.org/documentation/linux/kernel/building.md</nowiki>
 

Revision as of 09:29, 20 March 2024

查看中文

How to use VEYE and CS series camera module on Raspberry Pi(V4L2 mode)

1 Introduction

For the use of VEYE series and CS series camera modules on the raspberry Pi platform, we provide veye_raspicam software, an application layer open source software similar to the raspicam. This series of software, without driver support, has good compatibility with different versions of piOS.

However, we think that the V4L2 driver mode also has a wide range of applications. The two modes cannot be used at the same time, which is embodied in whether the driver is installed or not. It will be described in detail later.

This article describes how to call the camera modules of VEYE series and CS series through V4L2 on the raspberry Pi platform.

1.1 Camera module list

Camera module dirver status list
Series Model Status
VEYE series VEYE-MIPI-IMX327S Done
VEYE series VEYE-MIPI-IMX385 Done
VEYE series VEYE-MIPI-IMX462 Done
CS series CS-MIPI-IMX307 Done
CS series CS-MIPI-SC132 Done

In addition, FPD-Link3 mode connection is now supported.

2 Hardware Setup

VEYE series camera module setup on rpi

CS series camera module setup on rpi

3 RaspberryPi System Setup and Configuration

RaspberryPi System Setup and Configuration

4 Driver Installation

We have saved the code for v4l2 driver mode in this github repository.

4.1 Download the driver

wget https://github.com/veyeimaging/raspberrypi_v4l2/releases/latest/download/raspberrypi_v4l2.tgz

4.2 Install the driver

tar -xzvf raspberrypi_v4l2.tgz

cd raspberrypi_v4l2/release/

chmod +x *

  • For Raspberry Pi 5

sudo ./install_driver_rpi5.sh camera_module_name

It will install both CAM1 and CAM0 overlays in /boot/config.txt.

  • For other Raspberry Pi

sudo ./install_driver.sh camera_module_name

camera_module_name：could be veye327,csimx307,cssc132,veyecam2m.

Note: veyecam2m is a new version driver that can replace veye327 and supports all modules of the VEYE series with 200W resolution.

please use veyecam2m instead of veye327 which will be discarded.

Note: For VEYE-MIPI-IMX327S or VEYE-MIPI-IMX462, please make sure hdver must >= 0x5.

http://wiki.veye.cc/index.php/VEYE-MIPI-IMX327S_version_log

4.3 Uninstall the driver

If you need to change to using the camera by veye_raspicam software, or if you want to change to a driver of another camera module model, you must first uninstall the current driver.

sudo ./uninstall_driver.sh camera_module_name

camera_module_name：could be veye327,csimx307,cssc132,veyecam2m.

4.4 Camera in FPD-Link3 Mode

For a camera operating in FPD-Link3 transmission mode, it is necessary to ensure that the ds90ub954 driver is loaded and initialized before loading the camera driver and performing probe.

Take the VEYE-MIPI-IMX327S camera in FPD-Link3 mode as an example:

Install the drivers correctly:

sudo ./install_driver.sh ds90ub954

sudo ./install_driver.sh veyecam2m

5 Check and Test the Camera

Check the camera probe status

  • VEYE-MIPI-327

dmesg | grep veye

You can see:

veye327 camera probed

  • CS-MIPI-IMX307

dmesg | grep 307

You can see:

camera id is cs-mipi-imx307

  • CS-MIPI-SC132

dmesg | grep 132

You can see:

camera id is cs-mipi-sc132

5.1 List the video device

ls /dev/video0

The device node exsit.

5.2 List the available video modes

v4l2-ctl --list-formats-ext

6 Raspberry Pi 5 Setting

On Raspberry Pi 5,the drivers now use the media controller API, and we must setup the media graph correctly first. This includes setting up the media pad formats correctly and correctly linking them together.

We provide a series of scripts to implement this functionality, saved in the rpi5_scripts directory.

  • ./find_entity.sh

$ ./find_entity.sh

Found veyecam2m @ i2c-6 entity on /dev/media0

Plese get frame from /dev/video0 and use /dev/v4l-subdev2 for camera setting.

After a reboot of Raspberry Pi 5, the media node and video node of the camera may change. Therefore, it is recommended to execute ./find_entity.sh before performing subsequent operations to identify the device nodes.

The above prompt shows that the system has detected two cameras and their corresponding device nodes.

i2c-4 corresponds to the CAM1 port on the board, and i2c-6 corresponds to the CAM0 port on the board.

  • media_setting_rpi5.sh

./media_setting_rpi5.sh

Usage: ./media_setting_rpi5.sh veyecam2m/csimx307/cssc132/mvcam -fmt [UYVY/RAW8/RAW10/RAW12] -w [width] -h [height]

This shell script is designed to detect the connection of a camera on Raspberry Pi 5.

    It utilizes media-ctl and v4l2-ctl commands to configure the linking relationships and data formats of the media pad.

    Once completed, you can directly use /dev/video0 or /dev/video8 to obtain image data

Note: This script must be executed after each startup to complete the configuration of the parameters before proceeding with further operations.

For instance:

./media_setting_rpi5.sh veyecam2m

7 Preview

On a raspberry pi 5, VLC for preview is now having problems and not working.

7.1 Preview using VLC

1. Open VLC with command line vlc , or click the icon to launch.

2. Hit the ▶(Play) button to call the open media window.

3. In Capture Device >> Device Selection >> Video device name, select the camera video node.

4. Hit Advanced Options... button

5. In Advanced Options window >> Video input chroma format, type UYUV.

6. Type in the width and height, for example, 1920 and 1080.

7. Hit OK to save the settings and see the video feed.

Play camera using VLC on RPI
Play camera using VLC on RPI


7.1.1 Preview using qv4l2

Install qv4l2，

sudo apt install qv4l2

1. Open VLC with command line vlc to launch.

2. Hit the ▶(Play) button to call the open media window.

8 Gstreamer usage samples

export DISPLAY=:0

8.1 install gstreamer

sudo apt-get install gstreamer1.0-tools

sudo apt-get install libx264-dev libjpeg-dev

sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-ugly gstreamer1.0-tools gstreamer1.0-gl gstreamer1.0-gtk3

8.2 Run gstreamer videotest command

gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

8.3 Snap a picture (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src num-buffers=1 device=/dev/video0 ! 'video/x-raw, format=(string)UYVY, width=1920,height=1080' ! jpegenc ! filesink location=test_image.jpg

8.4 Frame rate test (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink video-sink=fakesink -v

8.5 Preview (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v

Or：

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink  sync=false -v

8.6 Preview and Scale with timestamp on (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1"  ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! video/x-raw, width=640, height=360 ! autovideosink sync=false -v

8.7 Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)720,framerate=(fraction)60/1" ! videoconvert ! autovideosink sync=false -v

8.8 Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @vga mode)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)640, height=(int)480" ! videoconvert ! autovideosink sync=false -v

8.9 Preview (CS-MIPI-SC132 @1280*1080@45 mode)

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1280, height=(int)1080,framerate=(fraction)45/1" ! videoconvert ! autovideosink sync=false -v

8.10 Streaming to file，save as mkv file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! matroskamux ! filesink location=output.mkv

8.11 Streaming to file，save as mp4 file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! mp4mux ! filesink location=video.mp4

8.12 TCP streaming

raspberrypi（sender）

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=-1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc extra-controls="controls, h264_profile=4, video_bitrate=4000000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink  host=x.x.x.x port=5000

The bandwidth of the stream is 4Mbps, continuous transmission, and the listening port is 5000.

Client

gst-launch-1.0 -v tcpclientsrc host=x.x.x.x port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! autovideosink sync=false

Where x.x.x.x is the IP address of the Raspberry Pi (Server).

It is recommended to use powershell if client is a windows system. gstreamer windows version download here.

9 Stream to OpenCV

pip uninstall opencv-python

sudo apt install python3-opencv

To import camera data from v4l2 devices to opencv, we provide several samples.

10 V4l2-ctl usage examples

10.1 install v4l2-utils

sudo apt-get install v4l-utils

10.2 v4l2-ctl test

  • Snap one UYVY raw data picture (1080p mode)

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv

  • Snap 10 UYVY raw data pictures (1080p mode)

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=10 --stream-to=uyvy-1920x1080_stream.yuv

11 yavta usage examples

11.1 download yavta

git clone git://git.ideasonboard.org/yavta.git

cd yavta;make

11.2 yavta test

./yavta -c8 -Fuyvy_1920X1080.yuv --skip 0 -f UYVY -s 1920x1080 /dev/video0

PS, UYVY file can be played by YUV Displayer Deluxe.

12 Video Control Toolkits Manual

Because of the high flexibility of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.

https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/i2c_cmd

VEYE Series Video Control Toolkits Manual

CS Series Video Control Toolkits Manual

13 Notes for CM4

Cm4 supports the use of two cameras at the same time. Following above steps will only be able to use CAM1. To use two cameras, follow the following steps:

13.1 Hardware Connection

Refer to this Raspberry Pi instruction ， add jumper to J6.

13.2 Upgrade dt-blob.bin

sudo wget https://datasheets.raspberrypi.com/cmio/dt-blob-dualcam.bin -O /boot/dt-blob.bin

13.3 For Buster OS（kernel5.10）

Upgrade dual camera version dtbo file, Use csimx307 as an example.

sudo cp raspberrypi_v4l2/release/driver_bin/$(uname -r)/csimx307-dual-cm4.dtbo /boot/overlays/csimx307.dtbo

sudo reboot

13.4 For Bullseye OS（kernel5.15）

Edit file/boot/config.txt，Add one linedtoverlay=[camera],cam0.

Take veyecam2m as an example.

[all]

dtparam=i2c_vc=on

dtoverlay=veyecam2m

dtoverlay=veyecam2m,cam0

13.5 Device node description

The CM4 module uses two I2C channels to communicate with the two cameras, respectively.

description i2c bus num video node
CAM0 0 video0
CAM1 10 video2（Buster），video1（Bullseye）

Note: If there is only one camera, video node is always video0.

13.6 Gstreamer usage samples

export DISPLAY=:0

  • CAM0 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v

  • CAM1 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v

14 Source code compilation

Please refer to: Build drivers from source for rpi.

15 Document History

  • 2024-01-01

Add support for Raspberrypi 5.

  • 20230607

Add FPD-Link3 Driver and Its Description.

  • 20230304

Add the instructions related to CM4 in Bullseye system.

  • 20220706

To optimize gstreamer's pipeline, it is recommended to use v4l2convert instead of videoconvert and try to use the capture-io-mode output-io-mode io-mode option.

  • 20220505

Refine the description of CM4.

  • 20220411

The source code compilation method is separate to a separate article.

  • 20220302

Add v4l2-ctl and yavta examples.

  • 20220301

Support 64-bit piOS

  • 20220225

Add the application samples of gstreamer.

  • 20211125

Add VLC preview section.

Retrieved from "http://wiki.veye.cc/index.php?title=V4L2_mode_for_Raspberry_Pi&oldid=7320"