Line 11: |
Line 11: |
| | | |
| 本文描述怎样在树莓派平台,通过V4L2方式调用VEYE系列和CS系列的摄像头模组。 | | 本文描述怎样在树莓派平台,通过V4L2方式调用VEYE系列和CS系列的摄像头模组。 |
| + | ====支持的模组:==== |
| + | {| class="wikitable" |
| + | |+模组型号支持表 |
| + | !系列 |
| + | !型号 |
| + | ! 状态 |
| + | |- |
| + | |VEYE系列 |
| + | |VEYE-MIPI-IMX327S |
| + | |完成 |
| + | |- |
| + | |VEYE系列 |
| + | |VEYE-MIPI-IMX385 |
| + | |完成 |
| + | |- |
| + | |VEYE系列 |
| + | |VEYE-MIPI-IMX462 |
| + | |完成 |
| + | |- |
| + | |CS系列 |
| + | |CS-MIPI-IMX307 |
| + | |完成 |
| + | |- |
| + | |CS系列 |
| + | |CS-MIPI-SC132 |
| + | |完成 |
| + | |}VEYE系列和CS系列MIPI摄像头模组是内置ISP的,输出为MIPI-CSI2接口。 |
| | | |
| + | 此外,现在也支持FPD-Link3模式的连接。 |
| === 硬件准备及安装 === | | === 硬件准备及安装 === |
| [[VEYE MIPI 290/327 for Raspberry Pi/zh#.E7.A1.AC.E4.BB.B6.E5.87.86.E5.A4.87.E5.8F.8A.E5.AE.89.E8.A3.85|VEYE系列的硬件准备及安装]] | | [[VEYE MIPI 290/327 for Raspberry Pi/zh#.E7.A1.AC.E4.BB.B6.E5.87.86.E5.A4.87.E5.8F.8A.E5.AE.89.E8.A3.85|VEYE系列的硬件准备及安装]] |
Line 30: |
Line 58: |
| <code>chmod +x *</code> | | <code>chmod +x *</code> |
| | | |
− | <code>sudo ./install_driver.sh [camera module]</code> | + | * 对于树莓派5 |
| + | |
| + | <code>sudo ./install_driver_rpi5.sh [camera module]</code> |
| + | |
| + | 会在/boot/config.txt中安装CAM1和CAM0两个dtvoerlay。 |
| + | |
| + | * 其他树莓派 |
| + | |
| + | <code>sudo ./install_driver.sh camera_module_name</code> |
| | | |
− | camera module:可以是veye327,csimx307,cssc132,veyecam2m等。
| + | 然后重启树莓派。 |
| + | |
| + | camera_module_name:可以是csimx307,cssc132,veyecam2m等。 |
| | | |
| 备注:如果提示无法找到对应版本的驱动,说明我们没有提供对应您的piOS版本编译后的驱动程序。请尝试从源码编译。 | | 备注:如果提示无法找到对应版本的驱动,说明我们没有提供对应您的piOS版本编译后的驱动程序。请尝试从源码编译。 |
Line 47: |
Line 85: |
| 如需更换为raspicam模式,或者想要更换成其他摄像头模组型号的驱动,必须要先卸载现在的驱动。 | | 如需更换为raspicam模式,或者想要更换成其他摄像头模组型号的驱动,必须要先卸载现在的驱动。 |
| | | |
− | <code>sudo ./uninstall_driver.sh [camera module]</code> | + | <code>sudo ./uninstall_driver.sh camera_module_name</code> |
| + | |
| + | camera_module_name:可以是veyecam2m,csimx307,cssc132等。 |
| + | |
| + | ==== FPD-Link3模式的相机 ==== |
| + | 对于FPD-Link3传输模式的相机,需要首先保证ds90ub954驱动的加载并完成初始化,然后才能加载摄像头驱动并进行探测。下面以FPD-Link3模式的VEYE-MIPI-IMX327S为例: |
| + | |
| + | 正常安装驱动: |
| | | |
− | camera module:可以是veye327,csimx307,cssc132等。
| + | <code>sudo ./install_driver.sh ds90ub954</code> |
| | | |
| + | <code>sudo ./install_driver.sh veyecam2m</code> |
| === 摄像头状态验证 === | | === 摄像头状态验证 === |
| 以veye327为例: | | 以veye327为例: |
Line 71: |
Line 117: |
| ==== 列出支持的模式 ==== | | ==== 列出支持的模式 ==== |
| <code>v4l2-ctl --list-formats-ext</code> | | <code>v4l2-ctl --list-formats-ext</code> |
− | ===使用VLC预览图像=== | + | ===树莓派5的配置=== |
| + | 在树莓派5上,驱动程序现在使用media control API,我们必须先正确设置media graph。这包括正确设置media pad的格式并正确链接它们。 |
| + | |
| + | 我们提供了一系列脚本实现这个功能,保存在rpi5_scripts目录。 |
| + | |
| + | *./find_entity.sh |
| + | |
| + | <code>$ ./find_entity.sh</code> |
| + | |
| + | <code>Found veyecam2m @ i2c-6 entity on /dev/media0</code> |
| + | |
| + | <code>Plese get frame from /dev/video0 and use /dev/v4l-subdev2 for camera setting.</code> |
| + | |
| + | 树莓派5重启之后,相机的media node,video node可能发生变化,因此,建议执行后续操作前执行<code>./find_entity.sh</code>以明确设备节点。 |
| + | |
| + | 上述提示信息中,可以看到系统插入了两个相机,以及每个相机的对应的设备节点。 |
| + | |
| + | <code>i2c-4</code>对应板子上的CAM1插口,<code>i2c-6</code>对应板子上的CAM0插口。 |
| + | |
| + | *media_setting_rpi5.sh |
| + | |
| + | <code>./media_setting_rpi5.sh</code> |
| + | |
| + | <code>Usage: ./media_setting_rpi5.sh veyecam2m/csimx307/cssc132/mvcam -fmt [UYVY/RAW8/RAW10/RAW12] -w [width] -h [height]</code> |
| + | |
| + | <code>This shell script is designed to detect the connection of a camera on Raspberry Pi 5.</code> |
| + | |
| + | <code> It utilizes media-ctl and v4l2-ctl commands to configure the linking relationships and data formats of the media pad.</code> |
| + | |
| + | <code> Once completed, you can directly use /dev/video0 or /dev/video8 to obtain image data</code> |
| + | |
| + | '''注意:每次启动后,都必须执行此脚本以完成参数的配置,然后再进行后续操作。''' |
| + | |
| + | 举例: |
| + | |
| + | <code>./media_setting_rpi5.sh veyecam2m</code> |
| + | |
| + | === 实时预览 === |
| + | 在树莓派5上,vlc进行播放现在有问题,不能使用。 |
| + | |
| + | ====使用VLC预览图像==== |
| 1. Open VLC with command line <code>vlc</code> , or click the icon to launch. | | 1. Open VLC with command line <code>vlc</code> , or click the icon to launch. |
| | | |
Line 84: |
Line 170: |
| 6. Type in the <code>width</code> and <code>height</code>, for example, 1920 and 1080. | | 6. Type in the <code>width</code> and <code>height</code>, for example, 1920 and 1080. |
| | | |
− | 7. Hit <code>OK</code> to save the settings and see the video feed.[[File:Play camera using VLC on RPI.png|alt=Play camera using VLC on RPI|center|thumb|1000x1000px|Play camera using VLC on RPI|link=http://wiki.veye.cc/index.php/File:Play_camera_using_VLC_on_RPI.png]] | + | 7. Hit <code>OK</code> to save the settings and see the video feed.[[File:Play camera using VLC on RPI.png|alt=Play camera using VLC on RPI|center|thumb|1000x1000px|Play camera using VLC on RPI|link=http://wiki.veye.cc/index.php/File:Play_camera_using_VLC_on_RPI.png]]<br /> |
| + | ====使用qv4l2播放图像==== |
| + | 首先安装qv4l2 |
| + | |
| + | <code>sudo apt install qv4l2</code> |
| + | |
| + | 1. Open VLC with command line <code>vlc</code> to launch. |
| + | |
| + | 2. Hit the <code>▶</code>(Play) button to call the open media window. |
| === Gstreamer应用范例 === | | === Gstreamer应用范例 === |
| <code>export DISPLAY=:0</code> | | <code>export DISPLAY=:0</code> |
Line 105: |
Line 199: |
| | | |
| ==== 视频预览(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ==== | | ==== 视频预览(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ==== |
− | <code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink sync=false -v</code> | + | <code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code> |
| + | |
| + | 或者: |
| + | |
| + | <code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! autovideosink sync=false -v</code> |
| | | |
| ==== 视频预览并加时间戳显示(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ==== | | ==== 视频预览并加时间戳显示(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) ==== |
− | <code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! video/x-raw, width=640, height=360 ! autovideosink</code> | + | <code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! video/x-raw, width=640, height=360 ! autovideosink sync=false -v</code> |
| | | |
| ==== 视频预览(CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode) ==== | | ==== 视频预览(CS-MIPI-IMX307,CS-MIPI-SC132 @720p mode) ==== |
Line 120: |
Line 218: |
| | | |
| ==== h.264编码并保存为mkv文件(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ==== | | ==== h.264编码并保存为mkv文件(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ==== |
− | <code>gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! matroskamux ! filesink location=output.mkv</code> | + | <code>gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! matroskamux ! filesink location=output.mkv</code> |
| | | |
| ==== h.264编码并保存为mp4文件(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ==== | | ==== h.264编码并保存为mp4文件(VEYE-MIPI-X,CS-MIPI-IMX307 @1080p mode) ==== |
− | <code>gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! mp4mux ! filesink location=video.mp4</code> | + | <code>gst-launch-1.0 -e v4l2src io-mode=dmabuf device=/dev/video0 num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2h264enc capture-io-mode=dmabuf output-io-mode=dmabuf extra-controls="controls, h264_profile=4, video_bitrate=6200000" ! 'video/x-h264, profile=high, level=(string)4' ! h264parse ! mp4mux ! filesink location=video.mp4</code> |
| ==== 网络传输 TCP streaming==== | | ==== 网络传输 TCP streaming==== |
| '''''树莓派(Server)一侧执行''''' | | '''''树莓派(Server)一侧执行''''' |
Line 139: |
Line 237: |
| Client一侧,如为windows系统,建议使用powershell。[https://gstreamer.freedesktop.org/ gstreamer]的windows版本[https://gstreamer.freedesktop.org/documentation/installing/on-windows.html?gi-language=c 下载地址]。只安装runtime files即可。 | | Client一侧,如为windows系统,建议使用powershell。[https://gstreamer.freedesktop.org/ gstreamer]的windows版本[https://gstreamer.freedesktop.org/documentation/installing/on-windows.html?gi-language=c 下载地址]。只安装runtime files即可。 |
| | | |
− | ==== 使用opencv获取视频 ==== | + | === 使用opencv获取图片 === |
− | <code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoscale ! "video/x-raw,width=640,height=480" ! videoconvert ! "video/x-raw, format=(string)BGR" ! appsink</code> | + | <code>pip uninstall opencv-python</code> |
| + | |
| + | <code>sudo apt install python3-opencv</code> |
| + | |
| + | 为了将摄像头数据从v4l2设备导入到opencv,我们提供了几个[https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/samples/opencv sample]。 |
| ===V4l2-ctl 应用范例=== | | ===V4l2-ctl 应用范例=== |
| ====安装v4l2-utils==== | | ====安装v4l2-utils==== |
Line 168: |
Line 270: |
| 由于我们的摄像头参数自由度比较高,并没有采用V4L2参数进行控制,而是使用脚本进行参数配置。 | | 由于我们的摄像头参数自由度比较高,并没有采用V4L2参数进行控制,而是使用脚本进行参数配置。 |
| | | |
− | https://github.com/veyeimaging/raspberrypi/tree/master/i2c_cmd | + | https://github.com/veyeimaging/raspberrypi_v4l2/tree/main/i2c_cmd |
| | | |
| [[VEYE MIPI 290/327 for Raspberry Pi/zh#.E8.A7.86.E9.A2.91.E6.8E.A7.E5.88.B6.E8.BD.AF.E4.BB.B6.E5.8C.85.E4.BD.BF.E7.94.A8|VEYE系列参数控制软件包使用说明]]<br />[[CS-MIPI-X for Raspberry Pi/zh#.E8.A7.86.E9.A2.91.E6.8E.A7.E5.88.B6.E8.BD.AF.E4.BB.B6.E5.8C.85.E4.BD.BF.E7.94.A8|CS系列参数控制软件包使用说明]] | | [[VEYE MIPI 290/327 for Raspberry Pi/zh#.E8.A7.86.E9.A2.91.E6.8E.A7.E5.88.B6.E8.BD.AF.E4.BB.B6.E5.8C.85.E4.BD.BF.E7.94.A8|VEYE系列参数控制软件包使用说明]]<br />[[CS-MIPI-X for Raspberry Pi/zh#.E8.A7.86.E9.A2.91.E6.8E.A7.E5.88.B6.E8.BD.AF.E4.BB.B6.E5.8C.85.E4.BD.BF.E7.94.A8|CS系列参数控制软件包使用说明]] |
Line 175: |
Line 277: |
| cm4支持两路摄像头同时使用,采用上面的步骤安装的dtbo将只能使用CAM1。如需使用两路摄像头,请按如下步骤: | | cm4支持两路摄像头同时使用,采用上面的步骤安装的dtbo将只能使用CAM1。如需使用两路摄像头,请按如下步骤: |
| | | |
| + | ===== 硬件连接 ===== |
| + | 参考树莓派这个[https://www.raspberrypi.com/documentation/computers/compute-module.html 说明],将J6进行短接。 |
| ==== 更新dt-blob.bin ==== | | ==== 更新dt-blob.bin ==== |
| <code>sudo wget <nowiki>https://datasheets.raspberrypi.com/cmio/dt-blob-dualcam.bin</nowiki> -O /boot/dt-blob.bin</code> | | <code>sudo wget <nowiki>https://datasheets.raspberrypi.com/cmio/dt-blob-dualcam.bin</nowiki> -O /boot/dt-blob.bin</code> |
| | | |
− | ==== 更新双摄像头的dtbo ==== | + | ==== 对于Buster系统(kernel5.10) ==== |
− | 以csimx307,5.10.17-v7l+版本kernel为例:
| + | 更新双摄像头的dtbo,以csimx307为例: |
| | | |
| <code>sudo cp raspberrypi_v4l2/release/driver_bin/$(uname -r)/csimx307-dual-cm4.dtbo /boot/overlays/csimx307.dtbo</code> | | <code>sudo cp raspberrypi_v4l2/release/driver_bin/$(uname -r)/csimx307-dual-cm4.dtbo /boot/overlays/csimx307.dtbo</code> |
| | | |
| <code>sudo reboot</code> | | <code>sudo reboot</code> |
| + | |
| + | ==== 对于Bullseye系统(kernel5.15+) ==== |
| + | 手动编辑<code>/boot/config.txt</code>文件,增加一行<code>dtoverlay=[camera],cam0</code>,如下所示: |
| + | |
| + | <code>[all]</code> |
| + | |
| + | <code>dtparam=i2c_vc=on</code> |
| + | |
| + | <code>dtoverlay=veyecam2m</code> |
| + | |
| + | <code>dtoverlay=veyecam2m,cam0</code> |
| | | |
| ==== 设备文件说明 ==== | | ==== 设备文件说明 ==== |
Line 198: |
Line 313: |
| |CAM1 | | |CAM1 |
| |10 | | |10 |
− | |video2 | + | |video2(Buster),video1(Bullseye) |
| |} | | |} |
| 注:只连一个摄像头,无论CAM0还是CAM1,都是video0。 | | 注:只连一个摄像头,无论CAM0还是CAM1,都是video0。 |
Line 207: |
Line 322: |
| * CAM0视频预览(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) | | * CAM0视频预览(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) |
| | | |
− | <code>gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink videosink=autovideosink sync=false text-overlay=false -v</code> | + | <code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video0 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code> |
| | | |
| * CAM1视频预览(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) | | * CAM1视频预览(VEYE-MIPI-327,CS-MIPI-IMX307 @1080p mode) |
| | | |
− | <code>gst-launch-1.0 v4l2src device=/dev/video2 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! videoconvert ! fpsdisplaysink videosink=autovideosink sync=false text-overlay=false -v</code> | + | <code>gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080,framerate=(fraction)30/1" ! v4l2convert capture-io-mode=dmabuf output-io-mode=dmabuf ! autovideosink sync=false -v</code> |
| | | |
| === 从源代码编译驱动方法 === | | === 从源代码编译驱动方法 === |
Line 218: |
Line 333: |
| === 本文修改记录 === | | === 本文修改记录 === |
| | | |
− | * 20220505 | + | * 2024-01-01 |
| + | |
| + | 增加对树莓派5的相关说明。 |
| + | |
| + | * 20230607 |
| + | |
| + | 增加FPD-Link3驱动及其描述。 |
| + | |
| + | * 20230304 |
| + | |
| + | 增加Bullseye系统中CM4的相关说明。 |
| + | |
| + | * 20220706 |
| + | |
| + | 优化gstreamer的pipeline,建议使用v4l2convert替代videoconvert,并且尽量使用capture-io-mode output-io-mode io-mode选项。 |
| + | |
| + | *20220505 |
| | | |
| 完善关于CM4的说明。 | | 完善关于CM4的说明。 |