Difference between revisions of "Mv mipi camera manual"

From wiki_veye
Jump to navigation Jump to search
(41 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Mv mipi camera manual/zh|查看中文]]
 
[[Mv mipi camera manual/zh|查看中文]]
  
−
'''<big>MV series MIPI image module manual</big>'''
+
'''<big>MV series and RAW series MIPI camea module manual</big>'''
 
===Overview===
 
===Overview===
 
MV series is a series of camera modules specially designed for machine vision. It is excellent in performance, compact and affordable in price. In addition to the video streaming mode, a trigger mode and a series of functions related to triggering are also provided. Provides a better option for embedded systems for machine vision applications.  
 
MV series is a series of camera modules specially designed for machine vision. It is excellent in performance, compact and affordable in price. In addition to the video streaming mode, a trigger mode and a series of functions related to triggering are also provided. Provides a better option for embedded systems for machine vision applications.  
 +
 +
The RAW series cameras have only one mcu as an auxiliary control chip and do not contain an ISP module. Therefore, its functions are a subset of the MV series. Whether or not the feature is applicable to RAW series cameras will be listed in the description of each feature later on.
 +
 +
In addition, RAW series cameras support SDA (Sensor direct access) mode, which will be detailed in the last section of this article.
  
 
To facilitate rapid development, we provide hardware datasheet , register manuals and drivers for embedded platforms, demos.
 
To facilitate rapid development, we provide hardware datasheet , register manuals and drivers for embedded platforms, demos.
Line 10: Line 14:
  
 
In this article, the ''[Related Commands]'' column after each function lists the commands in the [[Mv mipi i2c.sh user guide|mv_mipi_i2c.sh]] script that are related to this function.
 
In this article, the ''[Related Commands]'' column after each function lists the commands in the [[Mv mipi i2c.sh user guide|mv_mipi_i2c.sh]] script that are related to this function.
 +
====Camera Module List====
 +
{| class="wikitable"
 +
!Series
 +
!Model
 +
! Max Resolution
 +
!Shutter Mode
 +
|-
 +
| rowspan="6" |MV series
 +
|MV-MIPI-IMX178M
 +
|3088×2064@22.3 fps
 +
|Rolling
 +
|-
 +
|MV-MIPI-IMX287M
 +
|720×544 @ 530fps RAW8
 +
720×544 @ 437fps RAW10
 +
 +
720×544 @ 320fps RAW12
 +
|Global
 +
|-
 +
|MV-MIPI-SC130M
 +
|1280×1024@214fps RAW8
 +
1280×1024@177fps RAW10
 +
|Global
 +
|-
 +
|MV-MIPI-IMX296M
 +
|1456×1088@60.3fps
 +
|Global
 +
|-
 +
|MV-MIPI-IMX265M
 +
|2064×1544@55fps
 +
|Global
 +
|-
 +
|MV-MIPI-IMX264M
 +
|2464×2056@35fps
 +
|Global
 +
|-
 +
| rowspan="3" |RAW series
 +
|RAW-MIPI-SC132M
 +
|1080×1280@120fps
 +
|Global
 +
|-
 +
|RAW-MIPI-IMX462M
 +
|4lane mode@RAW10：1944 x 1096 @120fps
 +
2lane mode@RAW10/RAW12：1944 x 1096 @60fps
 +
|Rolling
 +
|-
 +
|RAW-MIPI-AR0234M
 +
|4lane mode：1920 x 1200 @120fps
 +
2lane mode：1920 x 1200  @60fps
 +
|Global
 +
|}
 
===Image Acquisition===
 
===Image Acquisition===
 
====Rolling shutter and Global shutter====
 
====Rolling shutter and Global shutter====
Line 37: Line 92:
  
 
===== Rolling shutter =====
 
===== Rolling shutter =====
 +
This section applies to model: MV-MIPI-IMX178M.
 +
 
For a rolling shutter sensor, such as IMX178, a complete image frame requires two image cycles to complete the process from exposure to output. So the maximum frame rate is half of the video streaming mode.
 
For a rolling shutter sensor, such as IMX178, a complete image frame requires two image cycles to complete the process from exposure to output. So the maximum frame rate is half of the video streaming mode.
  
Line 45: Line 102:
 
The following figure shows an example of a trigger period  with Trigger Number set to 2：<br />
 
The following figure shows an example of a trigger period  with Trigger Number set to 2：<br />
 
[[File:Rolling shutter Normal trigger mode new2.png|alt=|center|thumb|799x799px|Normal trigger mode( Rolling shutter)]]
 
[[File:Rolling shutter Normal trigger mode new2.png|alt=|center|thumb|799x799px|Normal trigger mode( Rolling shutter)]]
−
===== Global shutter =====
+
===== Smartsens's Global shutter =====
−
TODO
+
This section applies to model: MV-MIPI-SC130M,RAW-MIPI-SC132M.
 +
 
 +
For Smartsens's global shutter sensor, such as the SC130GS, a complete image frame requires an exposure time and a read out cycle to complete. The second frame can be exposed only after the  read out cycle is completed.
 +
 
 +
In normal trigger mode, for the global shutter sensor, we call one exp time and one read out cycle a '''''trigger cycle'''''.
 +
 
 +
Therefore the maximum frame rate in trigger mode is influenced by the length of the exposure time,and cannot achieve the same frame rate as the video streaming mode.
 +
 
 +
The following figure shows an example of a trigger period  with Trigger Number set to 2：
 +
[[File:Normal trigger mode (Global shutter).png|center|thumb|800x800px|Smartsens's  Normal trigger mode (Global shutter)]]
 +
<br />For RAW series modules, Exposure Delay and Trigger Dealy are not supported and Trigger number only support 1.
 +
 
 +
===== Sony's Global shutter =====
 +
This section applies to model: MV-MIPI-IMX296M,MV-MIPI-IMX265M,MV-MIPI-IMX264M,MV-MIPI-IMX287M.
 +
 
 +
For Sony's global shutter sensor, the readout of the previous frame and the exposure of the next frame can be synchronized in the trigger mode.
  
 +
Therefore, it is possible to achieve almost the same external trigger frame rate as in video streaming mode.
 +
[[File:Sony's Normal trigger mode (Global shutter).png.png|link=http://wiki.veye.cc/index.php/File:Sony's%20Normal%20trigger%20mode%20(Global%20shutter).png.png|center|thumb|800x800px|Sony's Normal trigger mode (Global shutter)]]
 +
<br />
 
==== High-speed continuous trigger mode ====
 
==== High-speed continuous trigger mode ====
  
 
===== Rolling shutter =====
 
===== Rolling shutter =====
 +
This section applies to model: MV-MIPI-IMX178M.
 +
 
In high-speed trigger mode, assume '''Trigger Number''' is N, frames 2 to N are output immediately after the previous frame and there is no more trigger interval. This allows the same maximum frame rate to be achieved as in video streaming mode.
 
In high-speed trigger mode, assume '''Trigger Number''' is N, frames 2 to N are output immediately after the previous frame and there is no more trigger interval. This allows the same maximum frame rate to be achieved as in video streaming mode.
  
Line 86: Line 163:
  
 
===Image Feature===
 
===Image Feature===
 +
 +
==== Pixel Format ====
 +
The output format of MV series cameras is compliant with the MIPI CSI-2 standard, and the supported pixel formats may vary from model to model. Refer to the following table.
 +
{| class="wikitable"
 +
! Model
 +
!Pixel Format
 +
!Note
 +
|-
 +
|MV-MIPI-IMX178M
 +
|Mono8，Mono10，Mono12
 +
|sensor's AD is 12bit-depth
 +
|-
 +
|MV-MIPI-SC130M
 +
|Mono8，Mono10
 +
|sensor's AD is 10bit-depth
 +
|-
 +
|MV-MIPI-IMX296M
 +
|Mono8，Mono10
 +
|sensor's AD is 10bit-depth
 +
|-
 +
|MV-MIPI-IMX287M
 +
|Mono8，Mono10，Mono12
 +
|sensor's AD is 8,10,12bit-depth
 +
|-
 +
|MV-MIPI-IMX265M
 +
|Mono8，Mono10，Mono12
 +
|sensor's AD is 12bit-depth
 +
|-
 +
|MV-MIPI-IMX264M
 +
|Mono8，Mono10，Mono12
 +
|sensor's AD is 12bit-depth
 +
|-
 +
|RAW-MIPI-SC132M
 +
|Mono8，Mono10
 +
|sensor's AD is 10bit-depth
 +
|-
 +
|RAW-MIPI-IMX462M
 +
|Mono10，Mono12
 +
|sensor的AD is 10,12bit-depth
 +
|-
 +
|RAW-MIPI-AR0234M
 +
|Mono8，Mono10
 +
|sensor的AD is 10bit-depth
 +
|}
 +
Related commands ：fmtcap, pixelformat.
 +
 +
==== Image size and Max framerate ====
 +
Image size depends on the sensor. It varies from model to model.
 +
 +
The maximum image frame rate depends on the sensor and the camera processing power, and the maximum frame rate varies for different image sizes.
 +
 +
After setting the completed image size (ROI area), the camera will automatically calculate the maximum frame rate and provide it to the user for reference.
 +
 +
The maximum frame rates for typical sizes are shown in the following table.
 +
{| class="wikitable"
 +
!model
 +
!size
 +
!max framerate
 +
!Note
 +
|-
 +
| rowspan="4" | MV-MIPI-IMX178M
 +
|3088 × 2064
 +
|22.3
 +
| Max size
 +
|-
 +
|2592 x 1944
 +
|28.15
 +
|
 +
|-
 +
|2560 x 2048
 +
|27
 +
|
 +
|-
 +
|1920 x 1080
 +
|67
 +
|
 +
|-
 +
| rowspan="3" |MV-MIPI-SC130M
 +
|1280 × 1024
 +
| 214
 +
|Max size，RAW8
 +
|-
 +
|1280 x 720
 +
|249
 +
|RAW8
 +
|-
 +
|640 x 480
 +
|367
 +
|RAW8
 +
|-
 +
| rowspan="2" |MV-MIPI-IMX296M
 +
|1456×1088
 +
|60.3
 +
| Max size
 +
|-
 +
|1280 x 720
 +
| 90
 +
|
 +
|-
 +
| rowspan="3" |MV-MIPI-IMX287M
 +
| rowspan="3" |720x544
 +
|530
 +
|RAW8
 +
|-
 +
|437
 +
| RAW10
 +
|-
 +
|320
 +
|RAW12
 +
|-
 +
|MV-MIPI-IMX265M
 +
|2064×1544
 +
|55
 +
| Max size
 +
|-
 +
| rowspan="2" |MV-MIPI-IMX264M
 +
|2464×2056
 +
|35
 +
|Max size
 +
|-
 +
|1920 x 1080
 +
| 67
 +
|
 +
|-
 +
|RAW-MIPI-SC132M
 +
|1080×1280
 +
|120
 +
|Max size
 +
|-
 +
| rowspan="2" |RAW-MIPI-IMX462M
 +
|1920x1088@4lane raw10
 +
|120
 +
|
 +
|-
 +
|1920x1088@2lane raw10/12
 +
|60
 +
|
 +
|-
 +
| rowspan="2" |RAW-MIPI-AR0234M
 +
|1920x1200@4lane
 +
|120
 +
|
 +
|-
 +
|1920x1200@2lane
 +
|60
 +
|
 +
|}The actual frame rate can be less than the maximum frame rate and can be set to a decimal number.
 +
 +
To guarantee the camera frame rate, we specify that the maximum exposure time cannot be greater than [1/fps] seconds.
 +
 +
Related commands：maxwh，maxfps，fps。
 +
 +
==== ROI area ====
 +
Only specific areas of the image can be transmitted by setting the camera's region of interest (ROI).
 +
 +
The parameters of the output region include the horizontal offset (x), vertical offset (y), width (w), and height (h) of the output region.
 +
 +
By default, the ROI area of the camera is the full resolution area of the sensor.
 +
 +
===== ROI parameter requirements =====
 +
[x] ，[y]， [height] must be 4-aligned and [width] must be 8-aligned.
 +
{| class="wikitable"
 +
! model
 +
!minimum  ROI area
 +
!Note
 +
|-
 +
|MV-MIPI-IMX178M
 +
|376 x 320
 +
|
 +
|-
 +
|MV-MIPI-SC130M
 +
|64 x 64
 +
|
 +
|-
 +
|MV-MIPI-IMX296M
 +
|80x4
 +
|
 +
|-
 +
|MV-MIPI-IMX287M
 +
|264x4
 +
|
 +
|-
 +
|MV-MIPI-IMX265M
 +
|264x4
 +
|
 +
|-
 +
|MV-MIPI-IMX264M
 +
|264x4
 +
|
 +
|-
 +
|RAW-MIPI-SC132M
 +
|64x64
 +
|
 +
|-
 +
|RAW-MIPI-IMX462M
 +
|368x304
 +
|
 +
|-
 +
|RAW-MIPI-AR0234M
 +
|64x64
 +
|
 +
|}
 +
The camera will make necessary adjustments to the user parameters to meet the parameter requirements.
 +
 +
So it is highly recommended that you read out the actual ROI parameters after writing them.
 +
 +
Related commands：roi.
 +
 +
==== Mirror and Flip ====
 +
This function is relatively easy to understand. A special note is that with both ROI and Mirror/Flip on, the image is first  ROI cropped, and then mirrored/flipped.
 +
[[File:ROI Mirror Flip.jpg|alt=|center|thumb|799x799px|ROI and Mirror/Flip]]
 +
<br />Related commands：imgdir.
 +
 +
==== Blacklevel ====
 +
Currently the application range of the black level value can only be selected for all pixels, and pixel selection is not supported. The lower the black level value, the darker the image is; the higher the black level value, the brighter the image is.
 +
 +
Related command: blacklevel.
 +
 +
==== Test Pattern ====
 +
 +
===== MV-MIPI-IMX178M =====
 +
The following is an example of the test chart pattern in 12bit data format.
 +
 +
====== IMX178 Mode 1: Grayscale gradient test pattern ======
 +
The value of the first column is 0, and the grayscale value increases by 4 for each column to the right.<br />
 +
[[File:Test-pattern1.png|link=http://wiki.veye.cc/index.php/File:Test-pattern1.png|center|thumb|800x800px|Test pattern 1]]
 +
 +
====== IMX178 Mode 2: Vertical stripe test chart ======
 +
The pixels in each column are FFEh/555h/AAAh/001h in order.
 +
[[File:Test pattern2 enlarged.png|link=http://wiki.veye.cc/index.php/File:Test%20pattern2%20enlarged.png|center|thumb|800x800px|Test pattern2 （enlarged）]]
 +
 +
===== MV-MIPI-SC130M RAW-MIPI-SC132M =====
 +
 +
====== Mode 1: Grayscale gradient test pattern ======
 +
[[File:Test-pattern1 sc130m.png|center|thumb|800x800px]]
 
<br />
 
<br />
 +
===== RAW-MIPI-AR0234M=====
 +
 +
====== Mode 1：Solid Color Test Pattern ======
 +
<br />[[File:Test pattern1 of 0234m.jpg|center|thumb|800x800px|Test pattern1 of 0234m|link=http://wiki.veye.cc/index.php/File:Test_pattern1_of_0234m.jpg]]<br />
 +
 +
====== Mode 2：color bar ======
 +
[[File:Test pattern2 of ar0234m.jpg|center|thumb|800x800px|Test pattern2 of ar0234m|link=http://wiki.veye.cc/index.php/File:Test_pattern2_of_ar0234m.jpg]]Related command: testimg.
 +
 
===ISP===
 
===ISP===
−
<br />
+
Generally, only manual exposure and manual gain are supported by RAW series cameras in this section. However, RAW-MIPI-AR0234M does support AEC control.
 +
 
 +
Functions supported by RAW-MIPI-AR0234M include:
 +
 
 +
AAROI,Manual Exposure,Auto Exposure,Manual Gain,Auto Gain.
 +
==== Exposure and Gain Control ====
 +
 
 +
===== AAROI =====
 +
The camera counts the current image brightness within the AAROI region and uses it for auto exposure and auto gain calculations. By default, AAROI is not enabled, and the real statistical area in effect is the entire area of the output image, i.e., the ROI area.
 +
 
 +
The AAROI coordinates are relative coordinates within the ROI area. So the area must be smaller than the image ROI.
 +
 
 +
Related command: aaroi，aaroienable.
 +
 
 +
===== Exposure =====
 +
The camera exposure time range is (0-1000000/fps)us. All parameters of exposure time are in microseconds. However, due to the properties of the sensor, the actual exposure time unit of the sensor is 1 line, can not be accurate to 1us.
 +
 
 +
In any mode, the actual current exposure time can be read with the '''''exptime''''' command.
 +
 
 +
Exposure control modes: ME, AE, AE once.
 +
 
 +
Related command: expmode，exptime.
 +
 
 +
====== Manual Exposure(ME) ======
 +
Use a fixed exposure time.
 +
 
 +
Related command：expmode,metime。
 +
 
 +
====== Auto Exposure(AE) ======
 +
Automatically adjusts the exposure time of the camera so that the average brightness in the AAROI reaches the target value.
 +
 
 +
This process continues all the time and therefore allows for rapid adaptation to changes in the environment light.
 +
 
 +
The target brightness value ranges from 0 to 255, independent of pixelformat.
 +
 
 +
Related command：expmode,aatarget, aemaxtime.
 +
 
 +
====== Auto Exposure once(AE once) ======
 +
When the camera is set to ''AE once'' mode by the '''''expmode''''' command, the camera will make one auto exposure adjustment and turn off the auto adjustment after finishing. If the adjustment cannot be made to the target brightness due to a mismatch between the environment brightness and the maximum exposure time, the adjustment will also stop after timeout.
 +
 
 +
Related command：expmode,aatarget, aemaxtime.
 +
 
 +
===== Gain =====
 +
The gain range of the camera, depending on the sensor used. An increase in gain will lead to an increase in noise, it is recommended not to use too much gain.
 +
{| class="wikitable"
 +
! model
 +
!Range
 +
!Step
 +
|-
 +
|MV-MIPI-IMX178M
 +
|0-48dB
 +
|0.1dB
 +
|-
 +
|MV-MIPI-SC130M
 +
|0-40dB
 +
|0.1dB
 +
|-
 +
|MV-MIPI-IMX296M
 +
|0-48dB
 +
|0.1dB
 +
|-
 +
|MV-MIPI-IMX287M
 +
|0-48dB
 +
|0.1dB
 +
|-
 +
|MV-MIPI-IMX265M
 +
|0-48dB
 +
|0.1dB
 +
|-
 +
|MV-MIPI-IMX264M
 +
|0-48dB
 +
|0.1dB
 +
|-
 +
|RAW-MIPI-SC132M
 +
|0-40dB
 +
|0.1dB
 +
|}
 +
 
 +
====== Manual Gain(MG) ======
 +
Use a fixed gain.
 +
 
 +
Related command：gainmode,mgain.
 +
 
 +
====== Auto Gain(AG) ======
 +
Automatically adjusts the gain of the camera so that the average brightness in the AAROI reaches the target value.
 +
 
 +
This process continues all the time and therefore allows for rapid adaptation to changes in the environment light.
 +
 
 +
The target brightness value ranges from 0 to 255, independent of pixelformat.
 +
 
 +
Related command：gainmode,aatarget, agmaxgain.
 +
 
 +
====== Auto Gain once(AG once) ======
 +
When the camera is set to ''AG once'' mode by the '''''gainmode''''' command, the camera will make one auto gain adjustment and turn off the auto adjustment after finishing. If the adjustment cannot be made to the target brightness due to a mismatch between the environment brightness and the maximum gain， the adjustment will also stop after timeout.
 +
 
 +
Related command：gainmode,aatarget, agmaxgain.
 +
 
 +
===== Other notes =====
 +
1. When enabling AE and AG at the same time, the camera gives priority to the use of exposure time and minimizes the use of gain. Try to avoid the noise caused by gain.
 +
 
 +
2. aeag_run_once_save command execute AE&AG once operation and save the results to the camera as manual values.This is a function that is useful when the camera is installed.
 +
 
 +
==== gamma ====
 +
Users can use the Gamma function to optimize the brightness of the captured image to display the user's desired image brightness on the monitor.
 +
 
 +
The camera applies a Gamma (γ) to each pixel to change the luminance value of each pixel according to the following formula。
 +
[[File:Gamma formula.jpg|none|thumb|400x400px|gamma_formula]]
 +
For 8-bit pixel format, the maximum pixel value (Ymax) is 255.
 +
 
 +
For 10-bit pixel format, the maximum pixel value (Ymax) is 1023.
 +
 
 +
For 12-bit pixel format, the maximum pixel value (Ymax) is 4095.
 +
 
 +
The gamma value ranges from 0 to 4.00.
 +
 
 +
a) When Gamma = 1.0, the overall brightness remains the same.
 +
 
 +
b) When Gamma < 1.0, the overall brightness increases.
 +
 
 +
c) When Gamma > 1.0, the overall brightness decreases.
 +
 
 +
In all cases, black pixels (brightness value = 0) and white pixels (brightness value = maximum) will not have their brightness adjusted.
 +
 
 +
For now, gamma only works on 8bit depth image.
 +
 
 +
Related command：gammaenable,gamma.
 +
 
 +
==== Lookup Table（LUT） ====
 +
The camera can replace the camera output values with the target values one by one according to a user-defined lookup table.
 +
 
 +
Related command：lutenable,lut.
 +
 
 
===IO Control===
 
===IO Control===
 +
For RAW series cameras, the input IO supports trigger edge and trigger delay, and does not support trigger filtering function. The output IO is the strobe out pin of the sensor that leads directly.
  
 
==== Input IO Configuration ====
 
==== Input IO Configuration ====
Line 115: Line 567:
 
Related commands ：trgdelay.
 
Related commands ：trgdelay.
  
−
===== Output IO Configuration =====
+
==== Output IO Configuration ====
 
MV series cameras now have 2 output signals, OUT_IO1 and OUT_IO2.
 
MV series cameras now have 2 output signals, OUT_IO1 and OUT_IO2.
  
Line 133: Line 585:
  
 
===== TriggerWait =====
 
===== TriggerWait =====
 +
The trigger wait signal indicates that the camera is now ready to respond to the trigger signal.
 +
 +
The camera's trigger wait signal can be used to optimize trigger image acquisition and can effectively avoid the problem of over-triggering.
 +
 +
====== Smartsens's Normal trigger mode(Global shutter) ======
 +
The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 2 as an example.
 +
[[File:Normal trigger mode details (Global shutter).png|center|thumb|800x800px|Smartsens's Normal trigger mode details (Global shutter)]]
 +
<br />
 +
====== Sony's Normal trigger mode(Global shutter) ======
 +
The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 2 as an example.
 +
[[File:Sony Normal trigger mode detail(Global shutter).png.png|link=http://wiki.veye.cc/index.php/File:Sony%20Normal%20trigger%20mode%20detail(Global%20shutter).png.png|center|thumb|800x800px|Sony's Normal trigger mode, Strobe and TriggerWait]]
 +
 +
====== Normal trigger mode(Rolling shutter) ======
 +
The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 2 as an example.
 +
[[File:Rolling shutter Normal trigger mode full-new.png|alt=|center|thumb|800x800px|Normal trigger mode ; Strobe and TriggerWait]]
 +
 +
====== High-speed trigger mode(Rolling shutter) ======
 +
The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 3 as an example.
 +
[[File:Rolling shutter High-speed trigger mode full-2.png|alt=|center|thumb|800x800px|High-speed trigger mode; Strobe and TriggerWait]]
 
<br />
 
<br />
 
===== XVS =====
 
===== XVS =====
 +
Output the XVS signal of the sensor.
  
 
===== XHS =====
 
===== XHS =====
 +
Output the XVS signal of the sensor.
  
 
===== Pin Reverse =====
 
===== Pin Reverse =====
 +
To facilitate camera IO configuration and connection, the MV Series cameras have an output pin level reversal feature.
 +
 +
Related commands ：outio1_rvs，outio2_rvs.
  
 
==== Read pin status ====
 
==== Read pin status ====
 +
All input and output IO pins can be read for their current status via the i2c bus.
 +
 +
Related commands ：gpios_status.
 +
 +
=== Special note for RAW series camera ===
 +
RAW series cameras have only one MCU as the auxiliary chip. Normal mode and SDA mode can be switched by [[Mv mipi i2c.sh user guide#mcubypass|mcubypass]] command.
 +
 +
==== Normal mode ====
 +
In normal mode, the motherboard cannot access the sensor directly. the MCU provides the initial configuration of the sensor and the configuration of various functions. For different embedded platforms, we have provided corresponding drivers, scripts, and demos.
 +
 +
The functions are described in the above description of this article.
 +
 +
==== SDA mode ====
 +
SDA mode is short for Sensor Direct Access, when the MCU no longer has any control over the i2c of the sensor. The user controls the sensor directly.
 +
 +
Meanwhile, in order to provide necessary system information, mode switching, and configuration of trigger signals, the MCU itself still provides i2c interface to the outside.
 +
 +
For example, the RAW series cameras has two i2c addresses in SDA mode: 0x30 is the i2c address of the sensor and 0x3b is the i2c address of the MCU.
 +
 +
In SDA mode, the drivers and demos we provided are no longer applicable, only some [[Mv mipi i2c.sh user guide|mv_mipi_i2c.sh]] instructions and [[MV Series MIPI Camera Register Map|registers]] are available.
 +
 +
==== Trigger Delay ====
 +
For RAW series cameras, the trigger signal needs to be forwarded to the sensor by the MCU, and this forwarding has a fixed delay of about 850ns.
 +
<br />

Revision as of 09:47, 22 October 2023

查看中文

MV series and RAW series MIPI camea module manual

1 Overview

MV series is a series of camera modules specially designed for machine vision. It is excellent in performance, compact and affordable in price. In addition to the video streaming mode, a trigger mode and a series of functions related to triggering are also provided. Provides a better option for embedded systems for machine vision applications.

The RAW series cameras have only one mcu as an auxiliary control chip and do not contain an ISP module. Therefore, its functions are a subset of the MV series. Whether or not the feature is applicable to RAW series cameras will be listed in the description of each feature later on.

In addition, RAW series cameras support SDA (Sensor direct access) mode, which will be detailed in the last section of this article.

To facilitate rapid development, we provide hardware datasheet , register manuals and drivers for embedded platforms, demos.

This article focuses on the functional principles of the MV camera in detail. For hardware manuals, registers, configuration scripts, drivers for different embedded platforms and usage, please refer to the corresponding documents.

In this article, the [Related Commands] column after each function lists the commands in the mv_mipi_i2c.sh script that are related to this function.

1.1 Camera Module List

Series Model Max Resolution Shutter Mode
MV series MV-MIPI-IMX178M 3088×2064@22.3 fps Rolling
MV-MIPI-IMX287M 720×544 @ 530fps RAW8

720×544 @ 437fps RAW10

720×544 @ 320fps RAW12

Global
MV-MIPI-SC130M 1280×1024@214fps RAW8

1280×1024@177fps RAW10

Global
MV-MIPI-IMX296M 1456×1088@60.3fps Global
MV-MIPI-IMX265M 2064×1544@55fps Global
MV-MIPI-IMX264M 2464×2056@35fps Global
RAW series RAW-MIPI-SC132M 1080×1280@120fps Global
RAW-MIPI-IMX462M 4lane mode@RAW10：1944 x 1096 @120fps

2lane mode@RAW10/RAW12：1944 x 1096 @60fps

Rolling
RAW-MIPI-AR0234M 4lane mode：1920 x 1200 @120fps

2lane mode：1920 x 1200 @60fps

Global

2 Image Acquisition

2.1 Rolling shutter and Global shutter

2.1.1 Rolling shutter

The rolling shutter sensor is implemented as shown in the figure below.。The exposure starts from the first line and the second line is exposed only after one line cycle.And so on, after N-1 rows the Nth row is exposed.It takes one cycle time to read out a row of data after the first row is exposed.After the first line is completely read out, the second line just starts to be read out. In this order, the whole image is completely read out. The rolling shutter sensors are less technically difficult than global exposure sensors, cheaper, and have greater resolution, making them a good choice for some static or slow object photography.

Rolling shutter mode
Rolling shutter mode
2.1.2 Global shutter

The global shutter sensor implementation is shown in the figure below. All rows of the sensor start exposure at the same time and end exposure at the same time. At the end of the exposure, the sensor transfers all electrons from the sensing area to the storage area and then reads out the pixel data row by row.The advantage of such exposure is that the image will not be shifted and skewed when shooting moving objects.

Global shutter mode
Global shutter mode


2.2 Start/Stop Acquisition

You can send the start acquisition and stop acquisition commands to the camera at any time.

Once the camera receives the start acquisition command, if working in video streaming mode, it will immediately start exposure and output images; if working in trigger mode, it will enter the state of waiting for the trigger signal. At the same time, the camera enters the running state.

Once the camera receives the stop acquisition command, it will complete the currently transmitting frames to ensure frame integrity, and then stop outputting images and enter the standby state. Note that the triggering process will be interrupted if triggering multiple frames is set and not all image frames have been output yet. That is, stopping the acquisition operation only guarantees the integrity of the current frame, not the entire trigger cycle.

Related commands ：imgacq.

2.3 Video streaming mode

In video streaming mode, the camera continuously exposes and outputs images according to the configured ROI region and frame rate. It is recommended to get the maximum frame rate supported by the maxfps function after changing the ROI.

Related commands ：trgmode,roi,fps,maxfps.

2.4 Normal trigger mode

2.4.1 Rolling shutter

This section applies to model: MV-MIPI-IMX178M.

For a rolling shutter sensor, such as IMX178, a complete image frame requires two image cycles to complete the process from exposure to output. So the maximum frame rate is half of the video streaming mode.

In normal trigger mode, for rolling shutter sensor, we refer to two consecutive frame cycles as one trigger cycle.

In normal trigger mode, if multiple frames are triggered at one time, the trigger delay acts after this trigger signal, and the trigger interval and exposure delay are valid before each trigger cycle.

The following figure shows an example of a trigger period with Trigger Number set to 2：

Normal trigger mode( Rolling shutter)
2.4.2 Smartsens's Global shutter

This section applies to model: MV-MIPI-SC130M,RAW-MIPI-SC132M.

For Smartsens's global shutter sensor, such as the SC130GS, a complete image frame requires an exposure time and a read out cycle to complete. The second frame can be exposed only after the read out cycle is completed.

In normal trigger mode, for the global shutter sensor, we call one exp time and one read out cycle a trigger cycle.

Therefore the maximum frame rate in trigger mode is influenced by the length of the exposure time,and cannot achieve the same frame rate as the video streaming mode.

The following figure shows an example of a trigger period with Trigger Number set to 2：

Smartsens's Normal trigger mode (Global shutter)


For RAW series modules, Exposure Delay and Trigger Dealy are not supported and Trigger number only support 1.

2.4.3 Sony's Global shutter

This section applies to model: MV-MIPI-IMX296M,MV-MIPI-IMX265M,MV-MIPI-IMX264M,MV-MIPI-IMX287M.

For Sony's global shutter sensor, the readout of the previous frame and the exposure of the next frame can be synchronized in the trigger mode.

Therefore, it is possible to achieve almost the same external trigger frame rate as in video streaming mode.

Sony's Normal trigger mode (Global shutter)


2.5 High-speed continuous trigger mode

2.5.1 Rolling shutter

This section applies to model: MV-MIPI-IMX178M.

In high-speed trigger mode, assume Trigger Number is N, frames 2 to N are output immediately after the previous frame and there is no more trigger interval. This allows the same maximum frame rate to be achieved as in video streaming mode.

The following figure shows an example of a trigger period with Trigger Number set to 3：

High-speed trigger mode（Rolling shutter）


2.6 Switching of modes

The working mode cannot be switched during operation; you must stop the acquisition before switch to a new working mode.

2.7 Trigger source

The soft trigger and hard trigger are only different in the source of trigger signal. The trigger delay, exposure delay, trigger frame number, trigger interval configurations and functions are exactly the same.

Related commands ：trgsrc.

2.7.1 Software trigger mode

By writing 1 to the corresponding register of the camera via the i2c bus, a soft trigger is performed.

Since there is a certain delay in software processing and i2c transmission, the timeliness of soft trigger is worse than hard trigger , and hard trigger is recommended for the occasions with high timeliness requirements.

Related commands ：trgone.

2.7.2 Hardware trigger mode

In hard trigger mode, the camera obtains the trigger signal by capturing the level change of the TrigIN IO. See the IO Control chapter for details.

2.8 Trigger statistics

The trigger statistics function counts the total number of triggers as well as the number of lost triggers.

Definition of total trigger number: in hard trigger mode, it is the number of triggers after trigger filtering; in soft trigger mode, it is the number of all triggers.

When the camera receives a trigger signal, it will not be able to respond to the new trigger signal if it is in the trigger cycle, which will cause the loss of the trigger signal.

Related commands ：trgcount, trgclr.

3 Image Feature

3.1 Pixel Format

The output format of MV series cameras is compliant with the MIPI CSI-2 standard, and the supported pixel formats may vary from model to model. Refer to the following table.

Model Pixel Format Note
MV-MIPI-IMX178M Mono8，Mono10，Mono12 sensor's AD is 12bit-depth
MV-MIPI-SC130M Mono8，Mono10 sensor's AD is 10bit-depth
MV-MIPI-IMX296M Mono8，Mono10 sensor's AD is 10bit-depth
MV-MIPI-IMX287M Mono8，Mono10，Mono12 sensor's AD is 8,10,12bit-depth
MV-MIPI-IMX265M Mono8，Mono10，Mono12 sensor's AD is 12bit-depth
MV-MIPI-IMX264M Mono8，Mono10，Mono12 sensor's AD is 12bit-depth
RAW-MIPI-SC132M Mono8，Mono10 sensor's AD is 10bit-depth
RAW-MIPI-IMX462M Mono10，Mono12 sensor的AD is 10,12bit-depth
RAW-MIPI-AR0234M Mono8，Mono10 sensor的AD is 10bit-depth

Related commands ：fmtcap, pixelformat.

3.2 Image size and Max framerate

Image size depends on the sensor. It varies from model to model.

The maximum image frame rate depends on the sensor and the camera processing power, and the maximum frame rate varies for different image sizes.

After setting the completed image size (ROI area), the camera will automatically calculate the maximum frame rate and provide it to the user for reference.

The maximum frame rates for typical sizes are shown in the following table.

model size max framerate Note
MV-MIPI-IMX178M 3088 × 2064 22.3 Max size
2592 x 1944 28.15
2560 x 2048 27
1920 x 1080 67
MV-MIPI-SC130M 1280 × 1024 214 Max size，RAW8
1280 x 720 249 RAW8
640 x 480 367 RAW8
MV-MIPI-IMX296M 1456×1088 60.3 Max size
1280 x 720 90
MV-MIPI-IMX287M 720x544 530 RAW8
437 RAW10
320 RAW12
MV-MIPI-IMX265M 2064×1544 55 Max size
MV-MIPI-IMX264M 2464×2056 35 Max size
1920 x 1080 67
RAW-MIPI-SC132M 1080×1280 120 Max size
RAW-MIPI-IMX462M 1920x1088@4lane raw10 120
1920x1088@2lane raw10/12 60
RAW-MIPI-AR0234M 1920x1200@4lane 120
1920x1200@2lane 60

The actual frame rate can be less than the maximum frame rate and can be set to a decimal number.

To guarantee the camera frame rate, we specify that the maximum exposure time cannot be greater than [1/fps] seconds.

Related commands：maxwh，maxfps，fps。

3.3 ROI area

Only specific areas of the image can be transmitted by setting the camera's region of interest (ROI).

The parameters of the output region include the horizontal offset (x), vertical offset (y), width (w), and height (h) of the output region.

By default, the ROI area of the camera is the full resolution area of the sensor.

3.3.1 ROI parameter requirements

[x] ，[y]， [height] must be 4-aligned and [width] must be 8-aligned.

model minimum ROI area Note
MV-MIPI-IMX178M 376 x 320
MV-MIPI-SC130M 64 x 64
MV-MIPI-IMX296M 80x4
MV-MIPI-IMX287M 264x4
MV-MIPI-IMX265M 264x4
MV-MIPI-IMX264M 264x4
RAW-MIPI-SC132M 64x64
RAW-MIPI-IMX462M 368x304
RAW-MIPI-AR0234M 64x64

The camera will make necessary adjustments to the user parameters to meet the parameter requirements.

So it is highly recommended that you read out the actual ROI parameters after writing them.

Related commands：roi.

3.4 Mirror and Flip

This function is relatively easy to understand. A special note is that with both ROI and Mirror/Flip on, the image is first ROI cropped, and then mirrored/flipped.

ROI and Mirror/Flip


Related commands：imgdir.

3.5 Blacklevel

Currently the application range of the black level value can only be selected for all pixels, and pixel selection is not supported. The lower the black level value, the darker the image is; the higher the black level value, the brighter the image is.

Related command: blacklevel.

3.6 Test Pattern

3.6.1 MV-MIPI-IMX178M

The following is an example of the test chart pattern in 12bit data format.

3.6.1.1 IMX178 Mode 1: Grayscale gradient test pattern

The value of the first column is 0, and the grayscale value increases by 4 for each column to the right.

Test pattern 1
3.6.1.2 IMX178 Mode 2: Vertical stripe test chart

The pixels in each column are FFEh/555h/AAAh/001h in order.

Test pattern2 （enlarged）
3.6.2 MV-MIPI-SC130M RAW-MIPI-SC132M
3.6.2.1 Mode 1: Grayscale gradient test pattern
Test-pattern1 sc130m.png


3.6.3 RAW-MIPI-AR0234M
3.6.3.1 Mode 1：Solid Color Test Pattern


Test pattern1 of 0234m


3.6.3.2 Mode 2：color bar
Test pattern2 of ar0234m

Related command: testimg.

4 ISP

Generally, only manual exposure and manual gain are supported by RAW series cameras in this section. However, RAW-MIPI-AR0234M does support AEC control.

Functions supported by RAW-MIPI-AR0234M include:

AAROI,Manual Exposure,Auto Exposure,Manual Gain,Auto Gain.

4.1 Exposure and Gain Control

4.1.1 AAROI

The camera counts the current image brightness within the AAROI region and uses it for auto exposure and auto gain calculations. By default, AAROI is not enabled, and the real statistical area in effect is the entire area of the output image, i.e., the ROI area.

The AAROI coordinates are relative coordinates within the ROI area. So the area must be smaller than the image ROI.

Related command: aaroi，aaroienable.

4.1.2 Exposure

The camera exposure time range is (0-1000000/fps)us. All parameters of exposure time are in microseconds. However, due to the properties of the sensor, the actual exposure time unit of the sensor is 1 line, can not be accurate to 1us.

In any mode, the actual current exposure time can be read with the exptime command.

Exposure control modes: ME, AE, AE once.

Related command: expmode，exptime.

4.1.2.1 Manual Exposure(ME)

Use a fixed exposure time.

Related command：expmode,metime。

4.1.2.2 Auto Exposure(AE)

Automatically adjusts the exposure time of the camera so that the average brightness in the AAROI reaches the target value.

This process continues all the time and therefore allows for rapid adaptation to changes in the environment light.

The target brightness value ranges from 0 to 255, independent of pixelformat.

Related command：expmode,aatarget, aemaxtime.

4.1.2.3 Auto Exposure once(AE once)

When the camera is set to AE once mode by the expmode command, the camera will make one auto exposure adjustment and turn off the auto adjustment after finishing. If the adjustment cannot be made to the target brightness due to a mismatch between the environment brightness and the maximum exposure time, the adjustment will also stop after timeout.

Related command：expmode,aatarget, aemaxtime.

4.1.3 Gain

The gain range of the camera, depending on the sensor used. An increase in gain will lead to an increase in noise, it is recommended not to use too much gain.

model Range Step
MV-MIPI-IMX178M 0-48dB 0.1dB
MV-MIPI-SC130M 0-40dB 0.1dB
MV-MIPI-IMX296M 0-48dB 0.1dB
MV-MIPI-IMX287M 0-48dB 0.1dB
MV-MIPI-IMX265M 0-48dB 0.1dB
MV-MIPI-IMX264M 0-48dB 0.1dB
RAW-MIPI-SC132M 0-40dB 0.1dB
4.1.3.1 Manual Gain(MG)

Use a fixed gain.

Related command：gainmode,mgain.

4.1.3.2 Auto Gain(AG)

Automatically adjusts the gain of the camera so that the average brightness in the AAROI reaches the target value.

This process continues all the time and therefore allows for rapid adaptation to changes in the environment light.

The target brightness value ranges from 0 to 255, independent of pixelformat.

Related command：gainmode,aatarget, agmaxgain.

4.1.3.3 Auto Gain once(AG once)

When the camera is set to AG once mode by the gainmode command, the camera will make one auto gain adjustment and turn off the auto adjustment after finishing. If the adjustment cannot be made to the target brightness due to a mismatch between the environment brightness and the maximum gain， the adjustment will also stop after timeout.

Related command：gainmode,aatarget, agmaxgain.

4.1.4 Other notes

1. When enabling AE and AG at the same time, the camera gives priority to the use of exposure time and minimizes the use of gain. Try to avoid the noise caused by gain.

2. aeag_run_once_save command execute AE&AG once operation and save the results to the camera as manual values.This is a function that is useful when the camera is installed.

4.2 gamma

Users can use the Gamma function to optimize the brightness of the captured image to display the user's desired image brightness on the monitor.

The camera applies a Gamma (γ) to each pixel to change the luminance value of each pixel according to the following formula。

gamma_formula

For 8-bit pixel format, the maximum pixel value (Ymax) is 255.

For 10-bit pixel format, the maximum pixel value (Ymax) is 1023.

For 12-bit pixel format, the maximum pixel value (Ymax) is 4095.

The gamma value ranges from 0 to 4.00.

a) When Gamma = 1.0, the overall brightness remains the same.

b) When Gamma < 1.0, the overall brightness increases.

c) When Gamma > 1.0, the overall brightness decreases.

In all cases, black pixels (brightness value = 0) and white pixels (brightness value = maximum) will not have their brightness adjusted.

For now, gamma only works on 8bit depth image.

Related command：gammaenable,gamma.

4.3 Lookup Table（LUT）

The camera can replace the camera output values with the target values one by one according to a user-defined lookup table.

Related command：lutenable,lut.

5 IO Control

For RAW series cameras, the input IO supports trigger edge and trigger delay, and does not support trigger filtering function. The output IO is the strobe out pin of the sensor that leads directly.

5.1 Input IO Configuration

Now there is only one input pin: TriggerIN_IO. The function is: input signal for hard trigger.

The active edge of the trigger signal can be configured via the Trigger Edge function.

Related commands ：trgedge.

5.1.1 Trigger Filter

In order to suppress the interference in the external trigger signal, MV series cameras have external trigger filtering function, including low level filtering and high level filtering.

The function of low level filtering is to filter out the interference signal on the low level, and the function of high level filtering is to filter out the interference signal on the high level.

The range of the filtering window can be set to [1,1000000]us.

The following figure shows an example of enabling both low level filtering and high level filtering with a filtering time of 1ms：

Trigger Filter

Related commands ：trgfilter_enable，trgfilter_time.

5.1.2 Trigger Delay

MV series cameras have a trigger delay function. The trigger delay time can be set to [1,1000000]us. It is in effect for both hard and soft triggers.

Related commands ：trgdelay.

5.2 Output IO Configuration

MV series cameras now have 2 output signals, OUT_IO1 and OUT_IO2.

OUT_IO1 can be configured for Strobe and Userout functions.

OUT_IO2 can be configured for five functions, Strobe, Userout, TriggerWait, XVS and XHS.

Related commands: outio1_mode, outio2_mode.

5.2.1 Strobe

In this mode, the camera sends a trigger signal to activate the flash. Strobe signal is active high, and when the trigger signal is received, the strobe signal level is pulled up, and the pull-up time is the sum of the exposure delay and trigger cycle.

5.2.2 UserOut

In this mode users can set their own constant output level of the camera to do special processing, such as controlling the constant light source or an alarm device.

The output level can be set individually for different OUT_IO.

5.2.3 TriggerWait

The trigger wait signal indicates that the camera is now ready to respond to the trigger signal.

The camera's trigger wait signal can be used to optimize trigger image acquisition and can effectively avoid the problem of over-triggering.

5.2.3.1 Smartsens's Normal trigger mode(Global shutter)

The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 2 as an example.

Smartsens's Normal trigger mode details (Global shutter)


5.2.3.2 Sony's Normal trigger mode(Global shutter)

The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 2 as an example.

Sony's Normal trigger mode, Strobe and TriggerWait
5.2.3.3 Normal trigger mode(Rolling shutter)

The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 2 as an example.

Normal trigger mode ; Strobe and TriggerWait
5.2.3.4 High-speed trigger mode(Rolling shutter)

The following figure lists the Strobe signal and the TriggerWait signal with the number of trigger frames set to 3 as an example.

High-speed trigger mode; Strobe and TriggerWait


5.2.4 XVS

Output the XVS signal of the sensor.

5.2.5 XHS

Output the XVS signal of the sensor.

5.2.6 Pin Reverse

To facilitate camera IO configuration and connection, the MV Series cameras have an output pin level reversal feature.

Related commands ：outio1_rvs，outio2_rvs.

5.3 Read pin status

All input and output IO pins can be read for their current status via the i2c bus.

Related commands ：gpios_status.

6 Special note for RAW series camera

RAW series cameras have only one MCU as the auxiliary chip. Normal mode and SDA mode can be switched by mcubypass command.

6.1 Normal mode

In normal mode, the motherboard cannot access the sensor directly. the MCU provides the initial configuration of the sensor and the configuration of various functions. For different embedded platforms, we have provided corresponding drivers, scripts, and demos.

The functions are described in the above description of this article.

6.2 SDA mode

SDA mode is short for Sensor Direct Access, when the MCU no longer has any control over the i2c of the sensor. The user controls the sensor directly.

Meanwhile, in order to provide necessary system information, mode switching, and configuration of trigger signals, the MCU itself still provides i2c interface to the outside.

For example, the RAW series cameras has two i2c addresses in SDA mode: 0x30 is the i2c address of the sensor and 0x3b is the i2c address of the MCU.

In SDA mode, the drivers and demos we provided are no longer applicable, only some mv_mipi_i2c.sh instructions and registers are available.

6.3 Trigger Delay

For RAW series cameras, the trigger signal needs to be forwarded to the sensor by the MCU, and this forwarding has a fixed delay of about 850ns.

Retrieved from "http://wiki.veye.cc/index.php?title=Mv_mipi_camera_manual&oldid=7139"