opencv(python)计算LK稀疏光流
光流法详解之一(LK光流）
cv.CalcOpticalFlowPyrLK(官方文档)
文章目录
- 参数说明
- 代码
参数说明
Python: cv.CalcOpticalFlowPyrLK(prev, curr, prevPyr, currPyr, prevFeatures, winSize, level, criteria, flags, guesses=None) -> (currFeatures, status, track_error)
Parameters:
prevImg – first 8-bit input image or pyramid constructed by buildOpticalFlowPyramid().
nextImg – second input image or pyramid of the same size and the same type as prevImg.
prevPts – vector of 2D points for which the flow needs to be found; point coordinates must be single-precision floating-point numbers.
nextPts – output vector of 2D points (with single-precision floating-point coordinates) containing the calculated new positions of input features in the second image; when OPTFLOW_USE_INITIAL_FLOW flag is passed, the vector must have the same size as in the input.
status – output status vector (of unsigned chars); each element of the vector is set to 1 if the flow for the corresponding features has been found, otherwise, it is set to 0.
err – output vector of errors; each element of the vector is set to an error for the corresponding feature, type of the error measure can be set in flags parameter; if the flow wasn’t found then the error is not defined (use the status parameter to find such cases).
winSize – size of the search window at each pyramid level.
maxLevel – 0-based maximal pyramid level number; if set to 0, pyramids are not used (single level), if set to 1, two levels are used, and so on; if pyramids are passed to input then algorithm will use as many levels as pyramids have but no more than maxLevel.
criteria – parameter, specifying the termination criteria of the iterative search algorithm (after the specified maximum number of iterations criteria.maxCount or when the search window moves by less than criteria.epsilon.
flags –
operation flags:
OPTFLOW_USE_INITIAL_FLOW uses initial estimations, stored in nextPts; if the flag is not set, then prevPts is copied to nextPts and is considered the initial estimate.
OPTFLOW_LK_GET_MIN_EIGENVALS use minimum eigen values as an error measure (see minEigThreshold description); if the flag is not set, then L1 distance between patches around the original and a moved point, divided by number of pixels in a window, is used as a error measure.
minEigThreshold – the algorithm calculates the minimum eigen value of a 2x2 normal matrix of optical flow equations (this matrix is called a spatial gradient matrix in [Bouguet00]), divided by number of pixels in a window; if this value is less than minEigThreshold, then a corresponding feature is filtered out and its flow is not processed, so it allows to remove bad points and get a performance boost.
代码
参考：Opencv Python版学习笔记（五）光流跟踪 Lucas-Kanade（LK）算法
修改：
- 支持视频(视频路径) / 图片列表(list[ndarray]) 两种输入方式
- 修改了
__init__() - 限制了跟踪的角点数
self.max_points
import numpy as np
import cv2
lk_params = dict(winSize=(15, 15),
maxLevel=2,
criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))
feature_params = dict(maxCorners=500,
qualityLevel=0.3,
minDistance=7,
blockSize=7)
class App:
def __init__(self, video, track_len=10, detect_interval=1, max_points=5000, img_size=(1200, 800)): # 构造方法，初始化一些参数和视频路径
self.video = cv2.VideoCapture(video)
self.isImgList = type(video) == list
self.track_len = track_len
self.detect_interval = detect_interval
self.max_points = max_points
self.img_size = img_size
self.frame_idx = 0
self.tracks = []
def run(self): # 光流运行方法
while True:
if self.isImgList:
ret, frame = self.frame_idx < len(self.video), self.video[self.frame_idx]
else:
ret, frame = self.video.read() # 读取视频帧
if ret:
frame = cv2.resize(frame, self.img_size)
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # 转化为灰度虚图像
vis = frame.copy()
if len(self.tracks) > 0: # 检测到角点后进行光流跟踪
img0, img1 = self.prev_gray, frame_gray
p0 = np.float32([tr[-1] for tr in self.tracks]).reshape(-1, 1, 2)
p1, st, err = cv2.calcOpticalFlowPyrLK(img0, img1, p0, None,
**lk_params) # 前一帧的角点和当前帧的图像作为输入来得到角点在当前帧的位置
p0r, st, err = cv2.calcOpticalFlowPyrLK(img1, img0, p1, None,
**lk_params) # 当前帧跟踪到的角点及图像和前一帧的图像作为输入来找到前一帧的角点位置
d = abs(p0 - p0r).reshape(-1, 2).max(-1) # 得到角点回溯与前一帧实际角点的位置变化关系
good = d < 1 # 判断d内的值是否小于1，大于1跟踪被认为是错误的跟踪点
self.tracks, p0, p1 = [self.tracks[i] for i in range(len(good)) if good[i]], p0[good].reshape(-1, 2), p1[good].reshape(-1, 2)
for tr, (x, y) in zip(self.tracks, p1): # 将跟踪正确的点列入成功跟踪点
tr.append((x, y))
if len(tr) > self.track_len:
del tr[0]
cv2.circle(vis, (x, y), 2, (0, 255, 0), -1)
cv2.polylines(vis, [np.int32(tr) for tr in self.tracks], False,
(0, 255, 0)) # 以上一振角点为初始点，当前帧跟踪到的点为终点划线
if self.frame_idx % self.detect_interval == 0: # 每隔detect_interval帧检测一次特征点
mask = np.zeros_like(frame_gray) # 初始化和视频大小相同的图像
mask[:] = 255 # 将mask赋值255也就是算全部图像的角点
for x, y in [np.int32(tr[-1]) for tr in self.tracks]: # 跟踪的角点画圆
cv2.circle(mask, (x, y), 5, 0, -1)
p = cv2.goodFeaturesToTrack(frame_gray, mask=mask, **feature_params) # 像素级别角点检测
if p is not None:
for x, y in np.float32(p).reshape(-1, 2):
if len(self.tracks) > self.max_points:
del self.tracks[0]
self.tracks.append([(x, y)]) # 将检测到的角点放在待跟踪序列中
self.frame_idx += 1
self.prev_gray = frame_gray
# vis = cv2.resize(vis, (1200,800))
cv2.putText(vis, str(self.frame_idx), (50, 100), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255, 255, 255), 2)
cv2.imshow('lk_track', vis)
print(self.frame_idx)
cv2.waitKey(1)
else:
break
def main():
import sys
try:
video_src = sys.argv[1]
except:
video_src = "video.mp4"
App(video_src).run()
cv2.destroyAllWindows()
if __name__ == '__main__':
main()