2,021
edits
Changes
Jump to navigation
Jump to search
Line 9:
Line 9: + + + + + + + + + + + + + + + + + + + + + + + + + + + + Line 19:
Line 47: + − + + + − + + + + + + + − camera module：could be veye327,csimx307,cssc132,veyecam2m.+ + + Line 43:
Line 82: − + − camera module：could be veye327,csimx307,cssc132,veyecam2m.+ + + + + + + + + + + + Line 81:
Line 131: + + + + + + + + + + + + + + − + + + + + + + + + + + + + + + + + + + + + + + + Line 98:
Line 185: + + + + + + + Line 107:
Line 201: − + Line 119:
Line 213: − + + + + + − + Line 134:
Line 232: − + − + Line 153:
Line 251: − ==== Stream to OpenCV ====+ − + + + + + Line 183:
Line 285: − <nowiki>https://github.com/veyeimaging/raspberrypi/tree/master/i2c_cmd</nowiki>+ Line 191:
Line 293: + + + − + + + + + + + + + + + + + − + − + − Use csimx307,5.10.17-v7l+ kernel as an example： − + − + − + Line 217:
Line 333: − + Line 226:
Line 342: − + − + − + − The main resources in this section are the [https://www.raspberrypi.org/documentation/linux/kernel/building.md official piOS building method]. We use the method of cross-compilation, and the 5.4.72 version of 32bitOS is taken as an example.+ − The following operations are done on ubuntu PC.+ − − − <code>sudo apt install git bc bison flex libssl-dev make libc6-dev libncurses5-dev</code> − − * 32-bit piOS target version − − <code>sudo apt install crossbuild-essential-armhf</code> − + − <code>sudo apt install crossbuild-essential-arm64</code>+ − ==== Download the standard version of the piOS source code ====+ − ===== Confirm the piOS version of your raspberry Pi =====+ − + − <code>$ uname -a</code>+ − <code>Linux raspberrypi 5.4.72-v7l+ #1356 SMP Thu Oct 22 13:57:51 BST 2020 armv7l GNU/Linux</code>+ − * <code>Code tag</code>+ − <code>$ cp /usr/share/doc/raspberrypi-bootloader/changelog.Debian.gz ./</code>+ − <code>gunzip changelog.Debian.gz</code>+ − Check the top lines and you will know the tag.+ − ===== There are two ways to get the source code of the version you want: ===== − # using git to clone the corresponding branch and checkout the corresponding tag.+ − − <code>git clone --branch rpi-5.4.y <nowiki>https://github.com/raspberrypi/linux</nowiki></code> − − <code>git checkout raspberrypi-kernel_1.20201022-1</code> − − ''PS: Please replace the above two commands with your own corresponding versions.'' − − 2. Manually download the version code of the corresponding tag directly from the link below. − − https://github.com/raspberrypi/linux/tags − − ==== Patch our code to kernel ==== − − *Driver source code − − camera driver path is ： linux/drivers/media/i2c，copy our camera module drivers to this path. − − *Modify the the Makefile and Kconfig files − − Modify the Config and Makefile in the same path，add the corresponding camera driver. − − *dts file − − dts file path is ：linux/arch/arm/boot/dts/overlays，copy our [camera]-overlay.dts files to this path. − − *Modify dts Makefile − − Modify the Makefile in the same path，add the corresponding dts compilation option. − − ==== Building ==== − − ==== Preparing ==== − − *For Raspberry Pi 1, Zero and Zero W, and Raspberry Pi Compute Module 1 default (32-bit only) build configuration − − <code>KERNEL=kernel</code> − − <code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcmrpi_defconfig</code> − − *For Raspberry Pi 2, 3, 3+ and Zero 2 W, and Raspberry Pi Compute Modules 3 and 3+ default 32-bit build configuration − − <code>KERNEL=kernel7</code> − − <code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig</code> − − *For Raspberry Pi 4 and 400, and Raspberry Pi Compute Module 4 default 32-bit build configuration − − <code>KERNEL=kernel7l</code> − − <code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2711_defconfig</code> − − * For Raspberry Pi 3, 3+, 4, 400 and Zero 2 W, and Raspberry Pi Compute Modules 3, 3+ and 4 default 64-bit build configuration − − <code>KERNEL=kernel8</code> − − <code>make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig</code> − ==== Add compilation options ==== − − * 32-bit version − − <code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig</code> − − * 64-bit version − − <code>make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig</code> − − Add the compilation options by the corresponding camera module, for 5.4 version kernel the path is driver-- > multimedia-- > i2C. − − For 5.10 version kernel,the path is Device Drivers --> Multimedia Support --> Media ancillary drivers --> Camera sensor devices. − − ===== Build ===== − − * <code>32-bit version</code> − − <code>make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs -j4</code> − − * <code>64-bit version</code> − − <code>make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image modules dtbs -j4</code> − − === Frequently Questions === − − * disagrees about version of symbol module_layout − − After the system has been booted, <code>dmesg</code> can see this error reported when the driver of camera is loaded. It is caused by the mismatch between the original piOS version on the board and the compiled camera driver version. − − It is recommended to refer to this [https://www.raspberrypi.com/documentation/computers/linux_kernel.html#cross-compiling-the-kernel link] to recompile from the code and install Image, dtb, modules as a whole. − − === References === − <nowiki>https://www.raspberrypi.org/documentation/linux/kernel/building.md</nowiki> − − === Document History ===
→Raspberry Pi 5 Setting
This article describes how to call the camera modules of VEYE series and CS series through V4L2 on the raspberry Pi platform.
This article describes how to call the camera modules of VEYE series and CS series through V4L2 on the raspberry Pi platform.
==== Camera module list====
{| class="wikitable"
|+Camera module dirver status list
!Series
! Model
! Status
|-
|VEYE series
|VEYE-MIPI-IMX327S
|Done
|-
|VEYE series
|VEYE-MIPI-IMX385
|Done
|-
|VEYE series
|VEYE-MIPI-IMX462
|Done
|-
|CS series
|CS-MIPI-IMX307
|Done
|-
|CS series
|CS-MIPI-SC132
|Done
|}
In addition, FPD-Link3 mode connection is now supported.
=== Hardware Setup ===
=== Hardware Setup ===
=== Driver Installation ===
=== Driver Installation ===
We have saved the code for v4l2 driver mode in this [https://github.com/veyeimaging/raspberrypi_v4l2 github repository].
==== Download the driver ====
==== Download the driver ====
<code>git clone <nowiki>https://github.com/veyeimaging/raspberrypi_v4l2.git</nowiki></code>
<code>wget <nowiki>https://github.com/veyeimaging/raspberrypi_v4l2/releases/latest/download/raspberrypi_v4l2.tgz</nowiki></code>
==== Install the driver ====
==== Install the driver ====
<code>tar -xzvf raspberrypi_v4l2.tgz</code>
<code>cd raspberrypi_v4l2/release/</code>
<code>cd raspberrypi_v4l2/release/</code>
<code>chmod +x *</code>
<code>chmod +x *</code>
<code>sudo ./install_driver.sh [camera module]</code>
* For Raspberry Pi 5
<code>sudo ./install_driver_rpi5.sh camera_module_name</code>
It will install both CAM1 and CAM0 overlays in /boot/config.txt.
* For other Raspberry Pi
<code>sudo ./install_driver.sh camera_module_name</code>
camera_module_name：could be veye327,csimx307,cssc132,veyecam2m.
'''Note: veyecam2m is a new version driver that can replace veye327 and supports all modules of the VEYE series with 200W resolution.'''
'''Note: veyecam2m is a new version driver that can replace veye327 and supports all modules of the VEYE series with 200W resolution.'''
If you need to change to using the camera by veye_raspicam software, or if you want to change to a driver of another camera module model, you must first uninstall the current driver.
If you need to change to using the camera by veye_raspicam software, or if you want to change to a driver of another camera module model, you must first uninstall the current driver.
<code>sudo ./uninstall_driver.sh [camera module]</code>
<code>sudo ./uninstall_driver.sh camera_module_name</code>
camera_module_name：could be veye327,csimx307,cssc132,veyecam2m.
==== Camera in FPD-Link3 Mode ====
For a camera operating in FPD-Link3 transmission mode, it is necessary to ensure that the ds90ub954 driver is loaded and initialized before loading the camera driver and performing probe.
Take the VEYE-MIPI-IMX327S camera in FPD-Link3 mode as an example:
Install the drivers correctly:
<code>sudo ./install_driver.sh ds90ub954</code>
<code>sudo ./install_driver.sh veyecam2m</code>
=== Check and Test the Camera ===
=== Check and Test the Camera ===
==== List the available video modes ====
==== List the available video modes ====
<code>v4l2-ctl --list-formats-ext</code>
<code>v4l2-ctl --list-formats-ext</code>
===Raspberry Pi 5 Setting===
On Raspberry Pi 5,the drivers now use the media controller API, and we must setup the media graph correctly first. This includes setting up the media pad formats correctly and correctly linking them together.
We provide a series of scripts to implement this functionality, saved in the rpi5_scripts directory.
*./find_entity.sh
<code>$ ./find_entity.sh</code>
<code>Found veyecam2m @ i2c-6 entity on /dev/media0</code>
<code>Plese get frame from /dev/video0 and use /dev/v4l-subdev2 for camera setting.</code>
After a reboot of Raspberry Pi 5, the media node and video node of the camera may change. Therefore, it is recommended to execute <code>./find_entity.sh</code> before performing subsequent operations to identify the device nodes.
=== Preview using VLC ===
The above prompt shows that the system has detected two cameras and their corresponding device nodes.
<code>i2c-4</code> corresponds to the CAM1 port on the board, and <code>i2c-6</code> corresponds to the CAM0 port on the board.
*media_setting_rpi5.sh
<code>./media_setting_rpi5.sh</code>
<code>Usage: ./media_setting_rpi5.sh veyecam2m/csimx307/cssc132/mvcam -fmt [UYVY/RAW8/RAW10/RAW12] -w [width] -h [height]</code>
<code>This shell script is designed to detect the connection of a camera on Raspberry Pi 5.</code>
<code> It utilizes media-ctl and v4l2-ctl commands to configure the linking relationships and data formats of the media pad.</code>
<code> Once completed, you can directly use /dev/video0 or /dev/video8 to obtain image data</code>
'''Note: This script must be executed after each startup to complete the configuration of the parameters before proceeding with further operations.'''
For instance:
<code>./media_setting_rpi5.sh veyecam2m</code>
===Preview===
On a raspberry pi 5, VLC for preview is now having problems and not working.
==== Preview using VLC ====
1. Open VLC with command line <code>vlc</code> , or click the icon to launch.
1. Open VLC with command line <code>vlc</code> , or click the icon to launch.
[[File:Play camera using VLC on RPI.png|alt=Play camera using VLC on RPI|center|thumb|1000x1000px|Play camera using VLC on RPI]]
[[File:Play camera using VLC on RPI.png|alt=Play camera using VLC on RPI|center|thumb|1000x1000px|Play camera using VLC on RPI]]
<br />
<br />
======Preview using qv4l2======
Install qv4l2，
<code>sudo apt install qv4l2</code>
1. Open VLC with command line <code>vlc</code> to launch.
2. Hit the <code>▶</code>(Play) button to call the open media window.
=== Gstreamer usage samples ===
=== Gstreamer usage samples ===
<code>export DISPLAY=:0</code>
<code>export DISPLAY=:0</code>
<code>sudo apt-get install libx264-dev libjpeg-dev</code>
<code>sudo apt-get install libx264-dev libjpeg-dev</code>
<code>sudo apt-get install libgstreamer1.0-devlibgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-ugly gstreamer1.0-tools gstreamer1.0-gl gstreamer1.0-gtk3</code>
<code>sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-ugly gstreamer1.0-tools gstreamer1.0-gl gstreamer1.0-gtk3</code>
==== Run gstreamer videotest command ====
==== Run gstreamer videotest command ====
==== Preview (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
==== Preview (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink sync=false -v</code>
<code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code>
Or：
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink sync=false -v</code>
==== Preview and Scale with timestamp on (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
==== Preview and Scale with timestamp on (VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ====
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! video/x-raw, width=640, height=360 ! autovideosink</code>
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! video/x-raw, width=640, height=360 ! autovideosink sync=false -v</code>
==== Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode) ====
==== Preview (CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode) ====
==== Streaming to file，save as mkv file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ====
==== Streaming to file，save as mkv file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ====
<code>gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! matroskamux ! filesink location=output.mkv</code>
<code>gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! matroskamux ! filesink location=output.mkv</code>
==== Streaming to file，save as mp4 file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ====
==== Streaming to file，save as mp4 file(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ====
<code>gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! mp4mux ! filesink location=video.mp4</code>
<code>gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! mp4mux ! filesink location=video.mp4</code>
==== TCP streaming====
==== TCP streaming====
'''''raspberrypi（sender）'''''
'''''raspberrypi（sender）'''''
It is recommended to use powershell if client is a windows system. '''''gstreamer''''' windows version download [https://gstreamer.freedesktop.org/documentation/installing/on-windows.html?gi-language=c here].
It is recommended to use powershell if client is a windows system. '''''gstreamer''''' windows version download [https://gstreamer.freedesktop.org/documentation/installing/on-windows.html?gi-language=c here].
=== Stream to OpenCV ===
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoscale ! "video/x-raw,width=640,height=480" ! videoconvert ! "video/x-raw, format=(string)BGR" ! appsink</code>
<code>pip uninstall opencv-python</code>
<code>sudo apt install python3-opencv</code>
To import camera data from v4l2 devices to opencv, we provide several [https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/samples/opencv samples].
===V4l2-ctl usage examples===
===V4l2-ctl usage examples===
====install v4l2-utils====
====install v4l2-utils====
Because of the high flexibility of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.
Because of the high flexibility of our camera parameters, we do not use V4L2 parameters to control, but use scripts to configure parameters.
https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/i2c_cmd
[[VEYE-MIPI-290/327 for Raspberry Pi#Video Control Toolkits Manual|VEYE Series Video Control Toolkits Manual]]
[[VEYE-MIPI-290/327 for Raspberry Pi#Video Control Toolkits Manual|VEYE Series Video Control Toolkits Manual]]
=== Notes for CM4 ===
=== Notes for CM4 ===
Cm4 supports the use of two cameras at the same time. Following above steps will only be able to use CAM1. To use two cameras, follow the following steps:
Cm4 supports the use of two cameras at the same time. Following above steps will only be able to use CAM1. To use two cameras, follow the following steps:
===== Hardware Connection =====
Refer to [https://www.raspberrypi.com/documentation/computers/compute-module.html this Raspberry Pi instruction] ， add jumper to J6.
==== Upgrade dt-blob.bin ====
==== Upgrade dt-blob.bin ====
<code>wget <nowiki>https://www.raspberrypi.org/documentation/hardware/computemodule/dt-blob-dualcam.bin</nowiki></code>
<code>sudo wget <nowiki>https://datasheets.raspberrypi.com/cmio/dt-blob-dualcam.bin</nowiki> -O /boot/dt-blob.bin</code>
==== For Buster OS（kernel5.10） ====
Upgrade dual camera version dtbo file, Use csimx307 as an example.
<code>sudo cp raspberrypi_v4l2/release/driver_bin/$(uname -r)/csimx307-dual-cm4.dtbo /boot/overlays/csimx307.dtbo</code>
<code>sudo reboot</code>
==== For Bullseye OS（kernel5.15） ====
Edit file<code>/boot/config.txt</code>，Add one line<code>dtoverlay=[camera],cam0</code>.
Take veyecam2m as an example.
<code>sudo cp dt-blob-dualcam.bin /boot/dt-blob.bin</code>
<code>[all]</code>
==== Upgrade dual camera version dtbo file ====
<code>dtparam=i2c_vc=on</code>
<code>sudo cp raspberrypi_v4l2/release/driver_bin/5.10.17-v7l+/csimx307-dual-cm4.dtbo /boot/overlays/csimx307.dtbo</code>
<code>dtoverlay=veyecam2m</code>
<code>sudo reboot</code>
<code>dtoverlay=veyecam2m,cam0</code>
==== Device node description ====
==== Device node description ====
The CM4 module uses two I2C channels to communicate with the two cameras respectively.
The CM4 module uses two I2C channels to communicate with the two cameras, respectively.
{| class="wikitable"
{| class="wikitable"
! description
! description
|CAM1
|CAM1
|10
|10
|video2
|video2（Buster），video1（Bullseye）
|}
|}
Note: If there is only one camera, video node is always video0.
Note: If there is only one camera, video node is always video0.
*CAM0 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
*CAM0 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink videosink=autovideosink sync=false text-overlay=false -v</code>
<code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code>
*CAM1 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
*CAM1 preview(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode)
<code>gst-launch-1.0 v4l2src device=/dev/video2 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink videosink=autovideosink sync=false text-overlay=false -v</code>
<code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code>
=== Source code cross compilation ===
=== Source code compilation ===
Please refer to: [[Build drivers from source for rpi]].
=== Document History ===
==== Build cross-compilation environment on Ubuntu 64-bitoperating system) ====
* 64-bit piOS target version
* 2024-01-01
Add support for Raspberrypi 5.
* 20230607
Add FPD-Link3 Driver and Its Description.
* <code>Release version</code>
* 20230304
Add the instructions related to CM4 in Bullseye system.
* 20220706
To optimize gstreamer's pipeline, it is recommended to use v4l2convert instead of videoconvert and try to use the capture-io-mode output-io-mode io-mode option.
* 20220505
Refine the description of CM4.
*20220411
The source code compilation method is separate to a separate article.
* 20220302
* 20220302