VEYE-MIPI-290/327 for Raspberry Pi
How to use VEYE-MIPI-X series camera module on Raspberry Pi（legacy mode）
1 Introduction
VEYE-MIPI-X is a is a special camera module which is compatible with RASPBERRY PI. It use SONY STARVIS sensor——IMX290/IMX327 and has excellent ISP functions build-in.The output format is stanard UYVY stream. It is very easy to use it with RASPBERRY PI.Of course, she also has some subtle differences with the official version. She only support 1080p resolution,and i2c cmd is different.
Theoretically,VEYE Camera Modules support all Raspberry Pi. For performance reasons, I recommend you use 3,3+,4 version. This article uses Raspberry Pi 3B as an example to explain how the VEYE-MIPI-290/327 camera module working with Raspberry Pi. It also explain the specials of RPI Computer Module and Zero using VEYE Camera Module.
2 About piOS---Bullseye
The latest system of Raspberry Pi - bullseye has removed the support of raspicam by default. But we still have 2 ways to use it.
- 1, Use the legacy version of the OS, that is, do not upgrade to Bullseye.
https://www.raspberrypi.com/software/operating-systems/
- 2, Open legacy camera support in raspi-config on Bullseye. We have upgraded the program to accommodate this usage.
If you want to add the legacy camera interfaces to Bullseye, please click your update icon in the taskbar to update. Then open a terminal (Ctrl-Alt-T) and type ‘sudo raspi-config’, go to ‘Interface Options’ and then ‘Legacy Camera’, and reboot. These camera interfaces are deprecated, and we are not supporting them going forwards.
https://www.raspberrypi.com/news/new-old-functionality-with-raspberry-pi-os-legacy/
By the way, Use this cmd to check your system version:
lsb_release -a
3 Hardware Setup
Essentials You Need：
- Raspberry PI Items : RPI, Network, MicroSD Card, Power, Monitor
- Camera Module Items : VEYE-MIPI-290/327，FFC，Dupont Wire，Lens，Adaptor Board. （We developed 2 types of adaptor boards, Mini Adaptor Board and Adaptor Board. Mini board only support Standard RPI，Adaptor Board support both Standard RPI and CM & Zero.)
Setup guide
- Connect the VEYE Camera Module the CSI-2 port using FFC Cable（type B).
- Connect Dupont Wire as shown below. VEYE-MIPI-290/327 need an addtional power.
- Startup RaspberryPi.
| VEYE-MIPI-IMX327S and RPI Connection
VEYE-MIPI-IMX462 and RPI Connection VEYE-MIPI-IMX385 and RPI Connection |
VEYE-MIPI-IMX327S FPC WIRE Connection
VEYE-MIPI-IMX462 FPC WIRE Connection VEYE-MIPI-IMX385 FPC WIRE Connection |
|---|---|
| VEYE-MIPI-327E and RPI Connection | VEYE-MIPI-327E FPC and PWR WIRE Connection |
|---|---|
| VEYE-MIPI-327B and RPI Connection | VEYE-MIPI-327B FPC and PWR WIRE Connection |
|---|---|
4 RaspberryPi System Setup and Configuration
- Setup RasapberryPi System，reference this link.
- Go to the main menu and open the Raspberry Pi Configuration tool. Enable Camera and I2C interface.
sudo raspi-config
- Enable SSH and Samba services.
5 Sofware Toolkits Install
5.1 Toolkits downloading
There are two way to install software toolkits.
- 1. Clone from github
git clone https://github.com/veyeimaging/raspberrypi.git
- 2. Downloading from github (link here), upload to RPI using samba or USB disk.
5.2 Toolkits Overview
Software toolkits contains three parts. veye_raspcam is raspcam alike Video Stream Toolkits, D_mipi_rpi is D-SDK Video Stream Toolkits, i2c_cmd is Video Control Toolkits.
- Video Stream Tookkits provides real-time display, capture, video recording, etc.
- Video Control Toolkits is a shell script which provides ISP parameters configuration capbility.
- Both is Open Source.
6 Video Stream Toolkits Manual
6.1 veye_raspcam: raspcam alike toolkits
cd raspberrypi/veye_raspcam/bin/
chmod +x *
VS toolkits provides several ELF file.
- veye_raspipreview
./veye_raspipreview -t 20000 -p '0,0,1280,720'
Dispaly real-time video to HDMI output,preview window is 1280*720, last 20 seconds.
./veye_raspipreview -t -1
If t is -1, Show continuous Real-time display to HDMI output,full screen.
- veye_raspivid
./veye_raspivid -t 5000 -o ~/test.h264
Implement H.264 format encoding and record for 5 seconds，Record the stream data to file ~/test.h264.
At the same time,Dispaly real-time video to HDMI output.(use -n if you do not want preview)
- veye_raspivid (stream over TCP using gstreamer)
RPI side
./veye_raspivid -b 4000000 -t 0 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=x.x.x.x port=5000
H.264 encoding on bitrate 4Mbps,forever，listen on TCP port 5000.
PC side,using gstreamer(Powershell recommended)
gst-launch-1.0 -v tcpclientsrc host=x.x.x.x port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! autovideosink sync=false
x.x.x.x is RPI IP address
gstreamer windows version download.
- veye_raspivid (stream over UDP using gstreamer)
PC side,using gstreamer(Powershell recommended)
gst-launch-1.0 -v udpsrc port=5321 ! gdpdepay ! rtph264depay ! avdec_h264 ! autovideosink sync=false
RPI side
./veye_raspivid -b 4000000 -t 0 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! udpsink host=x.x.x.x port=5000 sync=false
H.264 encoding on bitrate 4Mbps.
x.x.x.x is HOST PC IP address.You need to run the pc-side program first and then run the Raspberry Pi-side program.
- veye_raspivid (direct tcp stream with netcat，gstreamer etc..)
RPI side
./veye_raspivid -b 4000000 -t 0 -o - | nc -l -p 5000
H.264 encoding on bitrate 4Mbps,forever，listen on TCP port 5000.
PC side,using gstreamer(Powershell recommended)
./gst-launch-1.0 -v tcpclientsrc host=x.x.x.x port=5000 ! decodebin ! autovideosink
x.x.x.x is RPI IP address
gstreamer windows version download.
PC side, using mplayer
./mplayer -x 1280 -y 720 -geometry 0:0 -fps 200 -demuxer h264es -noborder ffmpeg://tcp://x.x.x.x:5000
x.x.x.x is RPI IP address
mplayer for windows download
- veye_raspividyuv
./veye_raspividyuv -t 200 -o /dev/shm/test.yuv
Record UYVY data to file /dev/shm/ directory(which is in DDR), time is 200ms.
- veye_raspiraw
./veye_raspiraw -md 2 -t 200 -o /dev/shm/out.%04d.yuv
Record one UYVY data to file /dev/shm/ directory(which is in DDR).
- veye_raspstillyuv
./veye_raspistillyuv -o /dev/shm/test.yuv
Capturing one image and save to /dev/shm directory(which is in DDR).1920*1080 YUV420 I420 format.
./veye_raspistillyuv -tl 50 -t 1000 -o /dev/shm/test%d.yuv
Capturing one image every 50 milliseconds and save to /dev/shm directory(which is in DDR).1920*1080 YUV420 I420 format.
- veye_raspistill
./veye_raspistill -o ~/test.jpg
Capturing one image and save to home directory.1920*1080 JPG format.
./veye_raspistill -e bmp -tl 50 -t 1000 -o /dev/shm/test%d.bmp
Capturing one image every 50 milliseconds and save to /dev/shm directory(which is in DDR).1920*1080 BMP format.
./veye_raspistill -k -o ~/test%d.jpg -t 0
Display video to HDMI,capture jpg image using keyboard，Enter to capture，X Enter to quit.
veye_raspistill support preview，-n will not preview。
6.2 D-SDK Toolkits Manual
It contains three parts: C language SDK(libdmipicam.so),C language sample,Python language sample.
6.2.1 Install support package
sudo apt-get update && sudo apt-get install libopencv-dev
sudo apt-get install python-opencv
sudo apt-get install python3-opencv
sudo apt-get install libzbar-dev
6.2.2 D-SDK
- interface：
opensource，D_mipicam.h
- compile：
./buildme
- install：
sudo install -m 644 ./libdmipicam.so /usr/lib/
6.2.3 C sample
- preview
Dispaly real-time video to HDMI output
- preview-dualcam
Dispaly real-time video to HDMI output for dual cameras（RPI CM）
- video
H.264 format encoding and record to file.
- capture
Capture one jpeg image.
- video2stdout
H.264 format encoding and send to stdout,which could be used by pipe. It's like veye_raspivid -o -.
./video2stdout | nc -l -p 5000
- capture_yuv
Capture one yuv image.
- capture-dualcam
Capture jpeg images for dual cameras.
- yuv_stream
Shows how to get yuv stream.
- capture2opencv
Shows how to get yuv stream,transfer to opencv format and display it.
- qrcode_detection
Shows how to get yuv stream,transfer to opencv format and display it and detect QR code.
6.2.4 Python sample
- preview.py
Dispaly real-time video to HDMI output
- capture.py
Capture one jpeg image.
- capture_yuv.py
capture_yuv
- video.py
H.264 format encoding and record to file.
- capture2opencv.py
Shows how to get yuv stream,transfer to opencv format and display it.
7 Video Control Toolkits Manual
On Raspberry Pi, We use I2C-0 as control bus for VEYE Camera Module. We provide a Shell Script — camera_i2c_config— to config pin usage.
Video Control Toolkits Manual : VEYE-MIPI-290/327 i2c
The new version of PiOS use i2c-10. If i2c-10 doesn't work, try -b 0.
8 RPI Computer Module and RPI Zero Additional Info
8.1 Overview
RPI CM and Zero have a different MIPI CSI-2 port with standard RPI. And CM has can support dual cameras.
For CM3, I use a Computer Module IO Board Plus designed by WAVESHARE ELECTRONICS. The official CMIO Board is similar.
For CM4，I use official CMIO board.
8.2 Hardware Setup
8.2.1 Connecting Video Stream Port
- Connect the VEYE Camera Module the CSI-2 port using FFC Cable（type A).
- Connect Dupont Line as shown below. VEYE-MIPI-290/327 need an addtional power.
- CM can support dual cameras.
8.2.2 Connecting I2C to CM3
- CAM1：
Attach CD1_SDA to GPIO2 .
Attach CD1_SCL to GPIO3 .
- CAM0：
Attach CD0_SDA to GPIO44 .
Attach CD0_SCL to GPIO45 .
For RPI official CMIO board, please refer to Link.
For WAVESHARE CMIO board, as shown below.
8.2.3 Connecting camera to CM4
8.3 System Setup and Configuration
For RPI official CMIO board, please refer to Link.
For WAVESHARE CMIO board, please refer to Link，Page 7.
8.4 Video Stream Toolkits
All command support [-cs] option.
option -cs 0 to control CAM0, -cs 1 to control CAM1. For example:
- veye_raspipreview
./veye_raspipreview -cs 0 -t 20000
Dispaly real-time video to HDMI output,from CAM0, lasts 20 seconds.
- veye_raspivid
./veye_raspivid -cs 1 -t 5000 -o ~/test.h264
Implement H.264 format encoding and record for 5 seconds from camera 1，Record the stream data to file ~/test.h264.
8.5 Video Control Toolkits
Downloading dt-blob.bin file from link,rename to dt-blob.bin and upload it to /boot/,reboot RPI.
8.5.1 CM3
CM3 use I2C-1 to control 2 cameras，It is time-sharing multiplexing.
./camera_i2c_config [cameranum]
| cameranum | description |
|---|---|
| 0 | CAM0 |
| 1 | CAM1 |
./veye_mipi_i2c.sh -b 1
8.5.2 CM4
The CM4 module uses two I2C to communicate with two cameras respectively.
./camera_i2c_config
| i2c bus num | description |
|---|---|
| 0 | CAM0 |
| 10 | CAM1 |
./veye_mipi_i2c.sh -b [i2c bus num]
Other options see : VEYE-MIPI-290/327 i2c
9 FFC Cable Remarks
- For Standard RPI，use FFC cable TypeB.
- For RPI CM3 ，use FFC cable TypeA.
10 FAQ
- could not find component 'vc.ril.rawcam'
mmal: mmal_component_create_core: could not find component 'vc.ril.rawcam'
mmal: Failed to create camera component
mmal: main: Failed to create camera component
Due to the variety of PiOS versions for the Raspberry Pi, some precompiled programs may encounter this issue during runtime in certain versions. In this case, please recompile the veye_raspcam program. The specific method is to go to the source directory and run the buildme script.