嵌入式 视频处理前端

VPFE

1.OVERVIEW

1.1 CCDC

1）产生SD，HD时序信号

2）可编程镜头阴影校正（ Lens Shading Correction）.

3）支持BT656，YCBCR422（8BIT，16BIT，WITH HS，VS），及14BIT RAW DATA FROMCCD/CMOS

4) 可编程14BIT 到8BIT输出

5）可通过外部写能动信号EN控制向DDR写数据

6）SENSOR CLK可达75MHZ，若用到H3A，则只能达67.5MHZ

7）DEFECT CORRECT

1.2.H3A

1）auto white balance and auto eexposure by collectingmetrics（统计）

2）读写DDR

3）只接收RAW DATA

1.3.IPIPE

1）只接收RAW DATA ，且转换14BIT RAW DATA为YCBCR422或YCRCR422 RESIZE

2）支持RGB Bayer pattern，经过彩色空间转换也支持CMYG

3）每个分量增益控制

4）可编程的RGB TO YCBCR的转换系数

5）可配置成仅RESIZE模式，即直接YCBCR422 RESIZE，不经过其它模块处理

6）RGB (32bit/16bit) output to SDRAM

7）要求至少BLANK 8PIXEL/行，4行 BLANK，在ONE SHOT 下，至少10行 BLANK，在处理之后

8）最大支持1344PIXEL 输入输出宽度

   9）DEFECT CORRECT

1.4.IPIPEIF

1）读CCDC，SDRAM，写IPIPE

2）重新调整 HD, VD, and PCLK timing to the IPIPE input.

3）dark-frame subtract黑色帧消除法降噪（用于长时间瀑光引起的噪声点）

2.引脚IO 接口

1. YIN[7..0]/CCDIN[7..0]

2. CIN[5..0]/CCDIN[13..0]

3.PCLK

4.CAM_HD,CAM_VD

5.CAM_WEN_FIELD:/设 CCD Write Enable/Field ID signal

     1）作FIELD标识信号，FLDMODE定类型提供，由CCD/CMOSFIDMD/FLDPOL

     2）作WEN用，EXWEN打开，WENLOG设定AND 或 OR，与在SPH, NPH, SLV,NLV内的有效像素相AND或OR

 

RAM模式引脚:

以上全需用到

另少于14BIT时,一般用CCD[13..0]高位,舍去底位,当用时SPI时除外

----------------------------------------------------------------

  引脚配置REG: PINMUX0(VPFE.PDFPAGE38)

3。VPFE/ISPIntegration

VPSS Events:9个作为中断送ARM，4个作为EVENT送EDMA

Number   AcronymModule     Description
0     CCDC VDINT0 CCDC   Triggeredafter a programmable number of input lines for each frame.
1 CCDC VDINT1 CCDC Triggered after a programmable number of inputlines for each frame.
2 CCDC VDINT2 CCDC Triggered at the rising edge of WEN signal
3 H3AINT H3A Triggered at the end of AF or AEW writes to DDR foreach frame
4 VENCINT VENC Triggered at the rising edge of VSYNC
5 OSDINT OSD Triggered at the end of each frame read from DDR
6 IPIPEIFINT IPIPEIF Triggered at the rising edge of VD ifenabled
7 IPIPE_INT0_HST IPIPE Triggered when Histogram processing isfinished for each frame
8 IPIPE_INT1_SDR IPIPE Triggered when writes to DDR are finishedfor each frame
9 IPIPE_INT2_RZA IPIPE Triggered when the number of linesprogrammed has been output of RZA
10 IPIPE_INT3_RZB IPIPE Triggered when the number of linesprogrammed has been output of RZB
12 IPIPE_INT5_MMR IPIPE Triggered when MMR modifications for thenext frame can be made

 

中断：VPSSBL.INTSEL选择

INT Number Acronym
0 VPSSINT0
1 VPSSINT1
2 VPSSINT2
3 VPSSINT3
4 VPSSINT4
5 VPSSINT5
6 VPSSINT6
7 VPSSINT7
8 VPSSINT8

事件：VPSSBL.EVTSEL选择

Event Number Binary Event Name
4 0000100 VPSSEVT1
5 0000101 VPSSEVT2
6 0000110 VPSSEVT3
7 0000111 VPSSEVT4

 

VPSS REG：

VPFE Module Register Map
VPSSRegisters                  Address Range                   Size
VPSSCLK                        0x01C700000x01C7007F            128B
H3A                            0x01C700800x01C700FF            128B
IPIPEIF                        0x01C701000x01C701FF            256B
OSD                            0x01C70200 0x01C702FF            256B
VENC                           0x01C704000x01C705FF            512B
CCDC                           0x01C706000x01C707FF            512B
VPSSBL                         0x01C708000x01C708FF            256B
IPIPE                           0x01C710000x01C73FFF            12K-

-------------------------------------------------------------------------

4 。VPFE/ISP FunctionalDescription

4.1 CCDC

参数：MODESET.INPMOD输出到DDR的数据选择，YCBCR或RGB

    RAW模式时序参数：HDW，VDW，PPLN，HLPFR （从AFE来）

   RAW模式时CCDCFG.YCINSWP=0

4.1.2.1 DIGITAL CLAMP

CLAMP的黑电平可用常量也可从像素中计算得到,也可从CCD SENSOR的上黑边，左黑边提取基量

以求平均值计算。

参数：CLAMP.OBST 黑像素开始点；CLAMP.OBSLEN每行的用来求平均的像素数；DCSUB.OBSLN求平均的行数

4.1.2.2 Color Space Conversion

CSCCTL[0]开启。由乘加器，LINE MEMORY组成，CMYG--》RGBG

FMTSPH, FMTLNH, FMTSLV, and FMTLNV指定数据区域

转换后数据会有一行LATENCY（延迟）。

最后一行至少要有一无效像素，一FRAME中至少要有一无效行

 

4.1.2.3 Defect Correction：坏点校正

   DFCCTL[0]开启，可校正1024个点

4.1.2.4 Lens Shading Correction：镜头阴影校正

   LSCCFG[0]开启，由于镜头中心较亮，向边缘渐暗。由此而校正

4.1.2.5 Black Level Compensation：太亮，太暗补偿

BLKCMP0.R_YE, BLKCMP0.GR_CY, BLKCMP1.GB_G, BLKCMP1.B_MG)

4.1.2.6 MedianFilter：中央滤波。一路送SDRAM，另一路送IPIPE，可单独能动

    1. Average Filter Configuration

2. Median Filter Configuration

GAMMAWD.MFILT1 and GAMMAWD.MFILT2

4.1.2.7 SDRAM RAW Output Formatting

 

4.1.4 Data Output Control

• SDOFST.FIINV – invert interpretation of the Field ID signal
• SDOFST.FIINV – invert interpretation of the Field ID signal
• SDOFST.LOFTS0 – offset, in lines, between even lines on evenfields (field 0)
• SDOFST.LOFTS1 – offset, in lines, between odd lines on evenfields (field 0)
• SDOFST.LOFTS2 – offset, in lines, between even lines on oddfields (field 1)
• SDOFST.LOFTS3 – offset, in lines, between odd lines on odd fields(field 1)

 

 

4.2 IPIPEIF

参数：HDW=8PIXEL，VDW=4LINE（CONTINUE）OR 10LINE（ONE SHOT），PPLN=HD间隔，LPFR=VD 间隔>1LINE，HNUM=水平有效像素（从HD上升沿开始？？？），VNUM=垂直有效像素（从VD下降沿开始，即隔4或10LINE）

4.2.1.1 CCDC RAW Input Mode (CFG.INPSRC = 0)

14BIT

4.2.1.2 SDRAM RAW Input Mode (CFG.INPSRC = 1h)

从SDRAM中读取8BIT或16BIT，8BIT为经过A-LAW压缩的，可设CFG.IALAW解压，也可填0补齐

4.2.1.3 CCDC RAW Input With Dark Frame Subtract From SDRAMMode (CFG.INPSRC = 2h)

IPIPEIF INPUT = CCDC RAW INPUT - SDRAM INPUT

此时PPLN，LPFR的意义与其它输入模式不同，表示从CCDC DATA中减去时的水平，垂直方向的起始位置

4.2.1.4 SDRAM YCbCr 4:2:2 Input Mode (CFG.INPSRC =3h)

16BIT YCbCr 4:2:2 data

4.2.2  时序产生

当输入为0或2时，HD/VD/WEN ，PCLK由CCDC提供， 为1或3时，IPIEIF产生HD/VD（基于LPFR andPPLN），PCLK可继续用CCDC的PCLK（CFG.CLKSEL=0），也可自已产生（CFG.CLKSEL=1）（用VPSSCLK分频CFG.CLKDIV），产生PCLK时要考虑下面IPIPERESIZE时最大时钟要求及SDRAM实时操作要求

当CFG.INPSRC非0时，IPIPE I/F SDRAM data读及时序产生即可在ONE-SHOT下，也可在CONTINUE模式下（(ENABLE.ENABLE）

当输入大于1344PIXEL时开启CFG.DECM设定比例系数以缩小到1344以下

4.3 IPIPE

参数：IPIPE_HST,IPIPE_VST定义开始点（与HD，VD上升沿为原点），IPIPE_HSZ,IPIPE_VSZ定义要处理的数据区域大小

IPIPE_COLPAT定义处理数据的Color pattern

Input Modes (IPIPE_DPATHS.FMT)

0: RAW to YCbCr

1: RAW to RAW

2: RAW to Boxcar

3: YCbCr to YCbCr

In RAW pass through mode,可以支持最大4096PIXEL/LINE，直接写到SDRAM

4.3.3 Defect Correction

4.3.4 Noise Filter

4.3.5 Pre Filter

为了减少LINE-CRAWLING，只针对green pixels.有两种方式（PRE_TYP选择）：1.常量 2.算法

4.3.6 White Balance

白平衡就是调整增益，有两种：

1.Digital gain调整影像总亮度，x 0 – x 3.996 (step = 1/256)

2. white balancegains 调整CFA中每个分量的增益，x 0 – x 7.992 (step =1/128)

4.3.7 CFA Interpolation

由 Bayer RGB转换成RGB 4:4:4 data

4.3.8 RGB2RGB Blending（混合）

RGB混合，即增加一OFFSET，调成人类的色彩光谱。(RGB_MUL_[R,G,B][R,G,B], &RGB_OFT_[R,G,B])

4.3.9 Gamma Correction Module

摄像时亮度与电压成非线性关系，称此为GAMMA曲线，GAMMA校正即是校正此非线性

有三个通路：1.BYPASS  2. 从RAM TABLE 3. 从ROM TABLE

4.3.10 RGB2YCbCr Conversion Module

YCC_MUL_[R,G,B][Y,CB,CR], and YCC_OFT_[Y,CB,CR]

4.3.11 4:2:2 Conversion Module

转换成YCBCR422格式

4.3.12 2-D Edge Enhancer

针对亮度信号，提高IMAGE质量。YEE_EN.EN开启。用一高通FILER来获取图像边缘，FILER系数可编程(YEE_MUL_[0,1,2][0,1,2])，输出可下SHIFT（YEE_SHF.SHF）。增加RAM表值可加强sharpness

 4.3.13 Median NR

用来减少边缘强度的pseudo peak（假峰值），YEE_EMF.EN开启

4.3.14 Chrominance Suppression

FCS_EN.EN开启。在图像的亮度很强区域，可能会有一两个色彩通道饱和，但其它色彩没有，这样就会造成假色彩，如最常出现的就是原本是白色的却出现的是粉红色， 色度抑制可减少假色彩。

4.3.15 Horizontal and Vertical Resize Module

范围： x1/16 scale-down to x8

参数：RSZ[n].RSZ_H_TYP类型选择，HRSZ (RSZ[n].RSZ_H_DIF) and VRSZ(RSZ[n].RSZ_V_DIF)比例，上下限为32-4096，对应X8-X1/16，实际以256/HRSZ，256/VRSZ计算，影像输出尺寸最大为1344pixels/line for RSZ[0] and 640 pixels/line for RSZ[1].

Vertical resize有两种模式：输入行号模式（RSZ_SEQ.TMM =1)，连续模式。RSZ[n].RSZ_V_PHS_O定义最后输出位置

4.3.16 Output Interface

YCC_Y_MAX, YCC_Y_MIN, YCC_C_MAX, and YCC_C_MIN 定义Y，C大小范围，YCBCR422TO YCBCR444，TO RGB，输出32BIT

4.3.17 RGB Converter

在H/VRESIZER中，YCBCR422通过线性插值，转成YCBCR444，由于此关系，第一行的一前一后两个像素会丢失，即RESIZE后会比想要的少两个像素。

在SDRAMBURST（256BIT）时以数据打包方式访问，有两种打包方式，32BIT，16BIT，(RSZ[n].RGB_TYP.TYP选择。32BIT时输出RGB各8位，及8BITalpha value(在RSZ[n].RGB_BLD设定）；In 16-bit mode, R (5-bit), G(6-bit), and B (5-bit)输入到SDRAM。

 

4.4 H3A

5 Programming Model

                      VPFE Memory Master Selection
SharedMemory                 RegisterField                    Settings
Shared Read Buffer(SRB)      PCR.RBLCTRL                     0: IPIPEIF
                                                              1: Reserved
                                                              2: H3A
Shared Write Buffer(SWB)     PCR.WBLCTRL                     0: IPIPEIF
                                                              1: Reserved
DFCTable                      MEMCTRL.DFCCTRL                 0: IPIPE
                                                              1: CCDC
RSZ LineMemory               MEMCTRL.RSZ_CTRL                0: IPIPEIF
                                                              1: Reserved

 

5.4 编程CCDC

模块能动：SYNCEN.VDHDEN能动CCDC，在能动之前需先设置一些REG。

         SYNCEN.WEN  写SDRAM能动

 

若CCDC为MASTER MODE，则当SYNCEN.VDHDEN 后立即进行数据处理

若CCDC为SLAVE  MODE,则 SYNCEN.VDHDEN 应先于外部设备打开,以免数据丢失

设置顺序:

  1. Set Data output address (STADRH &STADRL).
 2. Enable HD/VD and WEN at the same time(MODESET.WEN & SYNCEN.VDHDEN)

中断:

VDINT0, VDINT1, and VDINT2

VDINT0,VDINT1达到VDINT.VDINT0 and VDINT.VDINT1中的行数后中断

VDINT2 以 CAM_WEN_FIELD为参考,在下降沿中断

 

MODESET.FLDSTAT场状态指示

 

REG整体上可分两种：

Shadowed Registers (event latchedregisters)—可随时读写，但只有当一定的事件发生时才起作用

Busy-Writeable Registers—可随时读写，即时发生作用

 

5.5 Programming the Image Pipe Interface(IPIPEIF)

   模块能动：ENABLE.ENABLE，当输入为CCDC RAW时，不需能动，CCDC 送什么就处理什么

当输入为SDRAM时CFG.ONESHOT定义ONESHOT模式还是CONTINOUS模式。在ONESHOT模式，只有一FRAME被处理，且ENABLE.ENABLE自动清0

会产生一 IPIPEIF event to the VPSSBL

 5.6 Programming the Image Pipe(IPIPE)

    模块能动：IPIPE_EN.EN

    IPIPEEVENT

IRQ0 Final pixel from each frame is flushed from the image pipe(not resizer)   and    histogram   completion event
IRQ1 SDRAM Write Completion Event
IRQ2 RZA Interval Completion Event
IRQ3 RZB Interval Completion Event
IRQ5 Register Update Ready Notification Event

5.6.1.2.1 Resizer Bypass Mode
Since the YCbCr data still passes through the RZA block in resizerbypass mode, the following registers
must be set accordingly:
GCK_SDR =0           RSZ[0].RSZ_V_DIF = 256
RSZ_SEQ.SEQ =0       RSZ[0].RSZ_H_PHS = 0
RSZ_SEQ.TMM =0       RSZ[0].RSZ_H_DIF = 256
RSZ_AAL =0           RSZ[0].RSZ_H_LSE = 0
RSZ[0].RSZ_O_HPS = 0  RSZ[1].RSZ_EN = 0
RSZ[0].RSZ_V_PHS = 0

5.6.1.2.2 RAW input, RAW output Mode (IPIPE_DPATHS.FMT =1)
In this mode, the RAW data bypasses the RAW to YCbCr processes (seeFigure 37), but since it still
passes through the YCbCr processing blocks, the following registersmust be set accordingly:
IPIPE_DPATHS.FMT = 1 YEE_EN = 0 RSZ[0].RSZ_V_DIF = 256
BOX_EN = 0 FCS_EN = 0 RSZ[0].RSZ_H_PHS = 0
YCC_ADJ.CTR = 16 RSZ_SEQ.TMM = 0 RSZ[0].RSZ_H_DIF = 256
YCC_ADJ.BRT = 0 RSZ[0].RSZ_EN = 1 RSZ[0].RSZ_H_TYP = 0
YCC_Y_MIN = 0 RSZ[0].RSZ_I_VPS = 0 RSZ[0].RSZ_H_LSE = 0
YCC_Y_MAX = 255 RSZ[0].RSZ_I_HPS = 0 RSZ[0].RSZ_H_LPF = 0
YCC_C_MIN = 0 RSZ[0].RSZ_O_HPS = 0 RSZ[0].RSZ_RGB_EN = 0
YCC_C_MAX = 255 RSZ[0].RSZ_V_PHS = 0 RSZ[1].RSZ_EN = 0

5.6.1.3.1 Read/Write Procedures
The IPIPE internal memory values can be written, read, and writtenafter read. If multiple values need to
be set in incremental addresses, then the RAM_MODE.ADR bit can beset so that only the first address
needs to be set and subsequent reads/writes will auto-increment theinternal address. The following
procedures should be followed to access the internalmemories:
For memory write:
1. Write to RAM_MODE: set memory selection and set EXT = 0 andWDT =1.
2. Write to RAM_ADR: set starting offset into memory.
3. Write to RAM_WDT: Write data.
For memory read:
1. Write to RAM_MODE: set memory selection and set EXT = 0 andWDT =0.
2. Write to RAM_ADR: set starting offset into memory.
3. Write to RAM_WDT: Write dummy data.
4. Read the data of the internal memory from RAM_RDT.
For memory write after read:
1. Write to RAM_MODE: set memory selection and set EXT = 0 andWDT =1.
2. Write to RAM_ADR: set starting offset into memory.
3. Write to RAM_WDT: Write data.
4. Read the data of the internal memory from RAM_RDT.

 

5.7 Programming the Hardware 3A (H3A)

   模块能动： PCR.AF_EN ，PCR.AEW_EN

   当输入为SDRAM时，ONE-SHOT模式处理.若要处理每FRAME,则需设定PCR.SDR_FETCH_EN

    当输入为CCDC时,CONTINUOUS模式处理,此时依据CCDC的时序来处理.应先CCDC能动 

   此模块处理完一FRAME后,可产生一中断和一EVENT

 

5.8 Programming the Buffer Logic (VPSSBL and VPSSCLKRegisters)

VPSSBL and VPSSCLK Required Configuration Parameters
Function                     Configuration Required
CCDC                         VPSSBL.MEMCTRL.DFCCTRL
                           VPSSCLK.CLKCTRL.CCDCCLK
IPIPEIF                      VPSSBL.PCR.RBLCTRL
                            VPSSCLK.CLKCTRL.IPIPECLK
IPIPE                        VPSSBL.PCR.WBLCTRL
                            VPSSBL.MEMCTRL.IPIPE_WD_EN
                            VPSSBL.MEMCTRL.RESZ_CTRL
                            VPSSBL.MEMCTRL.DFCCTRL
                             VPSSCLK.CLKCTRL.IPIPECLK
H3A                          VPSSBL.PCR.RBLCTRL
                            VPSSCLK.CLKCTRL.H3ACLK

中断及事件选择VPSSBL.INTSEL and VPSSBL.EVTSEL