ME & block matching

1 Motion Estimation

图片.png

monocular vision 单眼视觉

图片.png

2 cases:

• optical flow is zero although there is motion in the scene
• optical flow is non-zero but there is no motion in the scene

图片.png

In removing noise from a sequence, we are interested in using both the spatial neighbourhood of a pixel in a frame, as well as the temporal neighbourhood. In this case, we want to filter along the motion trajectory

图片.png

motion based segmentation of the objects

图片.png

直接重复前一帧(zero-order hold)的object会产生 jagged motion
duplicate frame t1 into frame t
this is straightforward way to generate missing frame, however, it can generate jagged motion, because during going from t1 to t2, the person has considerably moved

图片.png

更平滑的approach是用motion compensation, find the motion between this block at frame t1 and t2

图片.png

temporal filtering denoise就是利用加性噪声时间平均减弱的原理, 加上motion estimation, 对于object block进行平均减弱加性噪声
the general idea is that i want to be able to find the motion and then perform a type of filtering, spacial temporal filtering along the motion trajectories

2 Block Matching

图片.png

block matching的四个前提假设
① no change int the ambient light, 环境光没有明显改变
② objects are rigid, 刚性运动
③ objects are translated in the 3D world on a plane parallel to the image place, 物体在画面上运动没有明显的大小变化, 运动是在一个平面(和成像平面平行)完成的
④ no objects appear or left the scene, 没有物体出画入画

图片.png

图片.png

  exausting search, 就是穷尽所有可能

图片.png

  将search region限定在一定区域内

图片.png

  进一步减少比较的次数, 就是空间二分

图片.png

图片.png

  用下采样的方式减少计算量, 每四个pixel只用其中的一个pixel计算error

图片.png

  另外一种减少计算量的方法, 不计算单个block的error, 而是计算16个方向上(x方向8个, y方向8个)的error, 在这个slide中, 原本要计算64个error, 现在只用计算16个, 具体的算法和应用场景具体用到的时候再去看一下

图片.png

  大尺度的motion estimation结果怎么与小尺度的motion estimation结果联合起来用, 就是motion estimation x2之后要怎么和当前这个尺度计算出来的motion estimation结合起来得到一个motion estimation的输出. serves as initial condition for the estimation of the highest levle??? 这个是什么意思, initial condition具体怎么用?
  这种尺度金字塔的方式经常被使用, 能够明显 improve ME

图片.png

  half-pel interpolation, 这是让motion vector方向更精细的意思吗(half-pel accurate motion vector)?

图片.png

  不同的searching方法效果的比较, 这里motion vector是只有大小没有方向吗?

图片.png

  BM的demo软件