【数字图像处理】水平翻转、垂直翻转

图像翻转是常见的数字图像处理方式,分为水平翻转和垂直翻转。本文主要介绍 FPGA 实现图像翻转的基本思路,以及使用紫光同创 PGL22G 开发板实现数字图像水平翻转、垂直翻转的过程。

【数字图像处理】水平翻转、垂直翻转_第1张图片

目录

1 水平翻转与垂直翻转

2 FPGA 布署与实现

2.1 功能与指标定义

2.2 逻辑设计

2.3 上板调试


1 水平翻转与垂直翻转

        在数字图像处理中,图像翻转是指将图像进行水平或者垂直方向的翻转,使其呈现不同的效果,分为水平翻转和垂直翻转。

        水平翻转是将图像沿着水平轴线进行翻转,将左半部分和右半部分进行交换。垂直翻转则是将图像沿着垂直轴线进行翻转,将上部分和下部分进行交换。

        使用 FPGA 实现水平翻转时,由于每次向 DDR3 写入一行数据,因此可以借助写 DDR3 之前的 Dual-port RAM,将一行数据进行翻转,实现水平翻转的功能。FPGA 逻辑设计的大致思路是:写 Dual-port RAM 时,写地址从 0 增加;读 Dual-port RAM 写 DDR3 时,读地址从最大值逐步减小到 0 结束。 

【数字图像处理】水平翻转、垂直翻转_第2张图片

        对于垂直翻转功能,则需要借助 DDR3 来实现。像素数据第 1 行写到第 N 行的位置,第 2 行写到第 N-1 行的位置,以此类推。FPGA 逻辑设计的大致思路是:每一行数据写入 DDR3 时,用图像高度减去原来的行地址,作为新的行地址。

【数字图像处理】水平翻转、垂直翻转_第3张图片

2 FPGA 布署与实现

2.1 功能与指标定义

        使用紫光同创 FPGA 平台实现图像翻转功能,FPGA 需要实现的功能与指标如下:

(1)与电脑的串口通信,用于接收上位机下发的图像数据,波特率为 256000 Bd/s;

(2)水平翻转与图像翻转:借助 Dual-port RAM 与 DDR3,分别实现水平翻转与垂直翻转功能;

(3)DDR3 读写控制,将处理前后的图像数据分别写入 DDR3 的不同区域,实现图像的拼接;

(4)HDMI 输出,输出一路 HDMI 信号源,用于将拼接后的图像显示在外接显示器上,分辨率为 1024×768。

2.2 逻辑设计

        图像翻转工程主要的设计模块层次与功能说明如下:

模块名称 功能说明
top_uart uart_rx_slice 串口接收驱动模块
uart_rx_parse 串口数据解析模块,从上位机接收 8bit 原始图像
top_vidin vidin_pipeline pipeline 单元模块,缓存两行图像数据,并将数据提交到 ddr3 数据调度模块
merge_out dvi_timing_gen HDMI 视频时序产生模块
dvi_ddr_rd 根据 HDMI 控制信号,提交读指令到 ddr3 数据调度模块
dvi_encoder HDMI 输出编码(8b10b 编码)与输出驱动模块

其中,vidin_pipeline 模块实现图像翻转功能,代码如下:

`timescale 1 ns/ 1 ps

`include "../ddr_scheduler/ddr_parameter.vh"

module vidin_pipeline (
   // System level
   sys_rst,
   sys_clk,
   ddr_init_done,
   flip_lr,
   flip_ud,

   // pipeline input ports
   pipeline_in_info,
   pipeline_in_data,
   pipeline_in_wren,
   pipeline_in_end,

   // pipeline output ports
   pipeline_out_info,
   pipeline_out_data,
   pipeline_out_vld,
   pipeline_out_end,

   // User defined ports for ddr_scheduler
   ddr_wr_baseaddr,
   ddr_wr_addr,
   ddr_wr_priority,
   ddr_wr_burstsize,
   ddr_wr_req,
   ddr_wr_ack,
   ddr_wr_end,
   ddr_wr_rden,
   ddr_wr_q,
   ddr_wr_mask
);

// IO direction/register definitions
input                     sys_rst;
input                     sys_clk;
input                     ddr_init_done;
input                     flip_lr;
input                     flip_ud;

input  [31:0]             pipeline_in_info;
input  [31:0]             pipeline_in_data;
input                     pipeline_in_wren;
input                     pipeline_in_end;

output [31:0]             pipeline_out_info;
output [31:0]             pipeline_out_data;
output                    pipeline_out_vld;
output                    pipeline_out_end;

input  [`DDR_A_W-1:0]     ddr_wr_baseaddr;
output [`DDR_A_W-1:0]     ddr_wr_addr;
output                    ddr_wr_priority;
output [`DDR_BURST_W-1:0] ddr_wr_burstsize;
output                    ddr_wr_req;
input                     ddr_wr_ack;
input                     ddr_wr_end;
input                     ddr_wr_rden;
output [`DDR_D_W-1:0]     ddr_wr_q;
output [`DDR_D_W/8-1:0]   ddr_wr_mask;

reg    [31:0]             pipeline_out_info;
reg    [31:0]             pipeline_out_data;
reg                       pipeline_out_vld;
reg                       pipeline_out_end;

// internal signal declarations
reg    [`DDR_CMD_W-1:0]   ddr_cmd_data;
reg                       ddr_cmd_vld;

reg    [9:0]              blk_mem_waddr;
reg    [31:0]             blk_mem_wdata;
reg                       blk_mem_wren;
reg    [9:0]              blk_mem_raddr;
wire   [31:0]             blk_mem_rdata;
reg                       blk_mem_rden;
reg                       blk_mem_rd_busy;
reg                       blk_mem_rd_end;
reg                       blk_mem_rd_vld;

// line_buffer_inst: Dual-port ram for line pixel data buffer
blk_mem_1024x32b line_buffer_inst (
   .wr_data         (blk_mem_wdata   ), // input 32-bit
   .wr_addr         (blk_mem_waddr   ), // input 10-bit
   .wr_en           (blk_mem_wren    ), // input 1-bit
   .wr_clk          (sys_clk         ), // input 1-bit
   .wr_rst          (sys_rst         ), // input 1-bit
   .rd_addr         (blk_mem_raddr   ), // input 10-bit
   .rd_data         (blk_mem_rdata   ), // output 32-bit
   .rd_clk          (sys_clk         ), // input 1-bit
   .rd_rst          (sys_rst         )  // input 1-bit
);
// End of line_buffer_inst instantiation

always @(posedge sys_rst or posedge sys_clk) begin
   if (sys_rst) begin
      blk_mem_waddr <= {10{1'b0}};
      blk_mem_wdata <= {32{1'b0}};
      blk_mem_wren  <= 1'b0;
   end
   else begin
      blk_mem_wdata <= pipeline_in_data;
      blk_mem_wren  <= pipeline_in_wren;

      // Use ping-pong storage here
      if (pipeline_in_end) 
         blk_mem_waddr <= {~blk_mem_waddr[9], {9{1'b0}}};
      else if (pipeline_in_wren) 
         blk_mem_waddr <= {blk_mem_waddr[9], blk_mem_waddr[0+:9]+1'b1};
   end
end

always @(posedge sys_rst or posedge sys_clk) begin
   if (sys_rst) begin
      blk_mem_raddr   <= {10{1'b0}};
      blk_mem_rden    <= 1'b0;
      blk_mem_rd_busy <= 1'b0;
      blk_mem_rd_end  <= 1'b0;
      blk_mem_rd_vld  <= 1'b0;
   end
   else begin
      if (~blk_mem_rd_busy && pipeline_in_end) begin
         blk_mem_rd_busy <= 1'b1;
         if (flip_lr == 1'b0) 
            blk_mem_raddr <= {blk_mem_raddr[9], {9{1'b0}}};
         else
            blk_mem_raddr <= {blk_mem_raddr[9], {9{1'b1}}};
      end
      else if (blk_mem_rd_busy) begin
         // Use ping-pong storage here
         if (flip_lr == 1'b0) begin
            if (& blk_mem_raddr[0+:9]) 
               blk_mem_raddr <= {~blk_mem_raddr[9], {9{1'b0}}};
            else 
               blk_mem_raddr <= {blk_mem_raddr[9], blk_mem_raddr[0+:9]+1'b1};
         end
         else begin
            if (blk_mem_raddr[0+:9] == 0)
               blk_mem_raddr <= {~blk_mem_raddr[9], {9{1'b1}}};
            else
               blk_mem_raddr <= {blk_mem_raddr[9], blk_mem_raddr[0+:9]-1'b1};
         end

         // Pull down read busy flag
         if (flip_lr == 1'b0) begin
            if (& blk_mem_raddr[0+:9]) 
               blk_mem_rd_busy <= 1'b0;
         end
         else begin
            if (blk_mem_raddr[0+:9] == 0)
               blk_mem_rd_busy <= 1'b0;
         end
      end

      blk_mem_rden <= blk_mem_rd_busy;
      blk_mem_rd_vld <= blk_mem_rden;

      if (blk_mem_rd_busy) begin
         if (flip_lr == 1'b0) begin
            if (& blk_mem_raddr[0+:9]) 
               blk_mem_rd_end <= 1'b1;
            else 
               blk_mem_rd_end <= 1'b0;
         end
         else begin
            if (blk_mem_raddr[0+:9] == 0)
               blk_mem_rd_end <= 1'b1;
            else 
               blk_mem_rd_end <= 1'b0;
         end
      end
      else
         blk_mem_rd_end <= 1'b0;
   end
end

always @(posedge sys_rst or posedge sys_clk) begin
   if (sys_rst) begin
      pipeline_out_info <= {32{1'b0}};
      pipeline_out_data <= {32{1'b0}};
      pipeline_out_vld  <= 1'b0;
      pipeline_out_end  <= 1'b0;
   end
   else begin
      if (pipeline_in_end) 
         pipeline_out_info <= pipeline_in_info;

      pipeline_out_data <= blk_mem_rdata;
      pipeline_out_vld  <= blk_mem_rd_vld;
      pipeline_out_end  <= blk_mem_rd_end;
   end
end

/
always @(posedge sys_rst or posedge sys_clk) begin
   if (sys_rst) begin
      ddr_cmd_data <= {`DDR_CMD_W{1'b0}};
      ddr_cmd_vld <= 1'b0;
   end
   else begin
      if (pipeline_in_end) begin
         ddr_cmd_data[32+:`DDR_BURST_W] <= 8'h7F; // used fixed size here, 512 /4 -1 = 127
         if (flip_ud == 1'b0)
            ddr_cmd_data[0+:28] <= {pipeline_in_info[0+:16], 12'd0};
         else
            ddr_cmd_data[0+:28] <= {16'd383-pipeline_in_info[0+:16], 12'd0};
      end

      if (blk_mem_rd_end) 
         ddr_cmd_vld <= 1'b1;
      else 
         ddr_cmd_vld <= 1'b0;
   end
end

// vid_ddr_wr_inst: ddr write control module
vid_ddr_wr vid_ddr_wr_inst (
   .sys_rst          (sys_rst           ), // input 1-bit
   .sys_clk          (sys_clk           ), // input 1-bit
   .ddr_init_done    (ddr_init_done     ), // input 1-bit
   .vid_cmd_data     (ddr_cmd_data      ), // input 40-bit
   .vid_cmd_vld      (ddr_cmd_vld       ), // input 1-bit
   .vid_img_data     (blk_mem_rdata     ), // input 32-bit
   .vid_img_data_vld (blk_mem_rd_vld    ), // input 1-bit
   .ddr_wr_baseaddr  (ddr_wr_baseaddr   ), // input 27-bit
   .ddr_wr_addr      (ddr_wr_addr       ), // output 27-bit
   .ddr_wr_priority  (ddr_wr_priority   ), // output 1-bit
   .ddr_wr_burstsize (ddr_wr_burstsize  ), // output 8-bit
   .ddr_wr_req       (ddr_wr_req        ), // output 1-bit
   .ddr_wr_ack       (ddr_wr_ack        ), // input 1-bit
   .ddr_wr_end       (ddr_wr_end        ), // input 1-bit
   .ddr_wr_rden      (ddr_wr_rden       ), // input 1-bit
   .ddr_wr_q         (ddr_wr_q          ), // output 128-bit
   .ddr_wr_mask      (ddr_wr_mask       )  // output 16-bit
);
// End of vid_ddr_wr_inst instantiation

endmodule

2.3 上板调试

        使用 PyQt5 和 OpenCV 库编写上位机程序,通过串口发送原始图像数据,以及水平翻转、垂直翻转参数。

# -*- Coding: UTF-8 -*-
import cv2
import sys
import struct
import numpy as np
from PyQt5 import Qt, QtGui, QtCore, QtWidgets, QtSerialPort


class mainWindow(Qt.QWidget):
   def __init__(self, com_port, parent=None):
      super(mainWindow, self).__init__(parent)
      self.setFixedSize(530, 384)
      self.setWindowTitle("PGL OpenCV Tool")

      self.flip_horizontal = False
      self.flip_vertical = False

      # 创建标签与按钮
      self.img_widget = QtWidgets.QLabel()
      self.btn1 = QtWidgets.QPushButton("打开")
      self.btn1.clicked.connect(self.getfile)
      self.btn2 = QtWidgets.QPushButton("关闭")
      self.btn2.clicked.connect(self.close)
      self.btn3 = QtWidgets.QPushButton("水平翻转")
      self.btn3.clicked.connect(self.flip_lr)
      self.btn4 = QtWidgets.QPushButton("垂直翻转")
      self.btn4.clicked.connect(self.flip_ud)

      # 创建布局
      centralLayout = QtWidgets.QVBoxLayout()
      centralLayout.addWidget(self.img_widget)
      bottomLayout = QtWidgets.QHBoxLayout()
      bottomLayout.addWidget(self.btn1)
      bottomLayout.addWidget(self.btn2)
      bottomLayout.addWidget(self.btn3)
      bottomLayout.addWidget(self.btn4)
      centralLayout.addLayout(bottomLayout)
      self.setLayout(centralLayout)

      # 串口对象
      self.COM = QtSerialPort.QSerialPort()
      self.COM.setPortName(com_port)
      self.COM.setBaudRate(256000)
      self.open_status = False
      self.row_cnt = 0
      self.img = None
      self.timer = QtCore.QTimer()
      self.timer.timeout.connect(self.sendImage)
      self.startup()

   def startup(self):
      """Write code here to run once"""
      for com_port in QtSerialPort.QSerialPortInfo.availablePorts():
         print(com_port.portName())

      # Try open serial port
      if not self.COM.open(QtSerialPort.QSerialPort.ReadWrite):
         self.open_status = False
         print("Open Serial Port failed.")
      else:
         self.open_status = True

   def flip_lr(self):
      """水平翻转回调函数"""
      if self.flip_horizontal == False:
         self.flip_horizontal = True
         self.btn3.setStyleSheet("QPushButton{color:rgb(128,128,255)}")
      else:
         self.flip_horizontal = False
         self.btn3.setStyleSheet("QPushButton{color:rgb(0,0,0)}")

   def flip_ud(self):
      """垂直翻转回调函数"""
      if self.flip_vertical == False:
         self.flip_vertical = True
         self.btn4.setStyleSheet("QPushButton{color:rgb(128,128,255)}")
      else:
         self.flip_vertical = False
         self.btn4.setStyleSheet("QPushButton{color:rgb(0,0,0)}")

   def getfile(self):
      """获取图像路径"""
      fname = QtWidgets.QFileDialog.getOpenFileName(self, 'Open file',
         'C:\\Users\\Administrator\\Pictures', "Image files(*.jpg *.png)")
      self.clipImage(fname[0])
      self.updateImage()
      self.sendImage()

   def clipImage(self, fname):
      """读取并裁剪图片至512x384大小"""
      if fname:
         img = cv2.imread(fname, cv2.IMREAD_COLOR)
         img_roi = img[:384,:512,:]
         print(img_roi.shape)
         cv2.imwrite('./img_roi.png', img_roi)

   def updateImage(self):
      """显示裁剪后的图像"""
      self.img_widget.setPixmap(QtGui.QPixmap('./img_roi.png'))
      self.img = cv2.imread('./img_roi.png')

      if self.open_status:
         self.timer.start(100)

   def sendImage(self):
      """通过串口发送图片"""
      pattern = ">2BH{:d}B".format(512*3)

      # 获取图像翻转信息
      flip_flag = 0x00
      if self.flip_horizontal:
         flip_flag = flip_flag + 0x10
      
      if self.flip_vertical:
         flip_flag = flip_flag + 0x01

      # 发送图像数据
      if self.open_status:
         if self.row_cnt == 384+3:
            self.row_cnt = 0
            self.timer.stop()
         else:
            args1 = [0x55, flip_flag, self.row_cnt]
            args2 = [rgb for rgb in self.img[(self.row_cnt % 384),:].reshape(-1)]
            send_data = struct.pack(pattern, *(args1+args2))
            self.row_cnt += 1
            self.COM.write(send_data)

   def closeEvent(self, event):
      super().closeEvent(event)
      #self.slider_window.close()

      # 定时器停止
      self.timer.stop()
      if self.open_status:
         self.COM.close() # 关闭串口

def main():
   app = QtWidgets.QApplication(sys.argv)
   window = mainWindow('COM21')
   window.show()
   #for win in (window, window.slider_window):
   #   win.show()
   sys.exit(app.exec_())

if __name__ == "__main__":
   main()

【数字图像处理】水平翻转、垂直翻转_第4张图片

        连接 HDMI 线和串口线,选择与发送图像,就可以看到 FPGA 的处理效果了。以下是水平翻转效果。

【数字图像处理】水平翻转、垂直翻转_第5张图片

以下是垂直翻转效果。

【数字图像处理】水平翻转、垂直翻转_第6张图片

你可能感兴趣的:(【FPGA,数字图像处理】,fpga开发,数字图像处理,紫光同创)