全景图（Panorama Stitching）

参考资料：
斯坦福课程CS131-fall1718作业3
harris、RANSAC都可以在本人本系列博客中找到

• 利用harris寻找角点（大概就是变化比较大的点）

• Descriptor两幅图片对应角点

我们上面已经找出变化比较大的点了，现在的问题是，怎么把两幅图片的点对应起来。
作业首先实现一个简单的Descriptor（下面代码和注释很清楚），在每个角点附近选定一个patch，然后用Descriptor转化为一个向量。A中有转化好的向量a0，B有转化好地多个向量bs，a0对应bs哪个的算法：a0分别计算跟bs中每个向量的距离欧拉距离，如果最短的距离b1远远小于第二短的距离b2，那么a0跟b1就对应起来了。
简单的Descriptor，其实其原理就是自己之前博客读到的算法NCC，归一化后，后面用欧拉距离进行比较。

def simple_descriptor(patch):
    """
    Describe the patch by normalizing the image values into a standard 
    normal distribution (having mean of 0 and standard deviation of 1) 
    and then flattening into a 1D array. 
    
    The normalization will make the descriptor more robust to change 
    in lighting condition.
    
    Hint:
        If a denominator is zero, divide by 1 instead.
    
    Args:
        patch: grayscale image patch of shape (h, w)
    
    Returns:
        feature: 1D array of shape (h * w)
    """
    feature = []
    ### YOUR CODE HERE
    feature = ((patch - np.mean(patch))/np.std(patch)).flatten()
    ### END YOUR CODE
    return feature

• affine变换矩阵求解-最小二乘法
  主要是利用np.linalg.lstsq

def fit_affine_matrix(p1, p2):
    """ Fit affine matrix such that p2 * H = p1 
    
    Hint:
        You can use np.linalg.lstsq function to solve the problem. 
        
    Args:
        p1: an array of shape (M, P)
        p2: an array of shape (M, P)
        
    Return:
        H: a matrix of shape (P * P) that transform p2 to p1.
    """

    assert (p1.shape[0] == p2.shape[0]),\
        'Different number of points in p1 and p2'
    p1 = pad(p1)
    p2 = pad(p2)

    ### YOUR CODE HERE
    H, _, _, _ = np.linalg.lstsq(p2, p1)
    ### END YOUR CODE

    # Sometimes numerical issues cause least-squares to produce the last
    # column which is not exactly [0, 0, 1]
    H[:,2] = np.array([0, 0, 1])
    return H

注意作业还将根据image1和image2，转换矩阵H计算出能容纳两个图片的空间

最后将两个图片merge，重叠部分取平均

merged = img1_warped + img2_warped

# Track the overlap by adding the masks together
overlap = (img1_mask * 1.0 +  # Multiply by 1.0 for bool -> float conversion
           img2_mask)

# Normalize through division by `overlap` - but ensure the minimum is 1
normalized = merged / np.maximum(overlap, 1)
plt.imshow(normalized)
plt.axis('off')
plt.show()

• RANSAC

从上面的结果可以看到效果并不是很好，于是用RANSAC避免一些错误的点的干扰，算法具体内容看我RANSAC对应的文章。

• HOG Descriptor

HOG详细的东西看本人之前的文章
跟上面的结果好像差不多