python:随机采样一致性（RANSAC）直线模型拟合原理及代码实现

过程原理

简介

随机抽样一致算法（Random Sample Consensus，RANSAC）。它采用迭代的方式从一组包含离群的被观测数据中估算出数学模型的参数。 RANSAC是一个非确定性算法，在某种意义上说，它会产生一个在一定概率下合理的结果，而更多次的迭代会使这一概率增加。此RANSAC算法在1981年由Fischler和Bolles首次提出。

RANSAC的基本假设：

1. “内群”(inlier, 似乎译为内点群更加妥当，即正常数据，正确数据)数据可以通过几组模型的参数来叙述其分布，而“离群”(outlier,似乎译为外点群更加妥当，异常数​​据，噪声点)数据则是不适合模型化的数据。
2. 数据会受噪声影响，噪声指的是离群，例如从极端的噪声或错误解释有关数据的测量或不正确的假设。
3. RANSAC假定，给定一组（通常很小）的内点群，存在一个模型，可以估算最佳解释或最适用于这一数据模型的参数。

算法原理

1. 在数据中随机选择若干个点设定为内点群
2. 计算拟合内点群的模型
3. 把其它刚才没选到的点带入刚才建立的模型中，计算是否属于内点群
4. 记下内点群数量
5. 重复以上步骤
6. 比较哪次计算中内点群数量最多，内点群最多的那次所建的模型就是我们所要求的解

这里有几个问题

• 一开始的时候我们要随机选择多少点（n）
• 以及要重复做多少次（k）

参数求解

假设每个点是真正内点群的几率是 $w$，则：

$w$ = 真正内点群的数目 / 数据总数

通常我们不知道$w$ 是多少，$w^n$是所选择的$n$个点都是内点群的几率，$1-w^n$是所选择的 $n$个点至少有一个不是内点群的几率，$(1-w^n{)}^k$是表示重复$k$ 次都没有全部的 $n$ 个点都是内点群的几率，假设算法跑$k$ 次以后成功的几率是$p$，那么：
$$1-p=(1-w^n{)}^k$$$$k=log(1-p)/log(1-w^n{)}^k$$

所以如果希望成功几率高，$p=0.99$，
当 $n$不变时，$k$越大，$p$ 越大，
当 $w$不变时，$n$ 越大，所需的 $k$ 就越大，
通常 $w$ 未知，所以 $n$ 选小一点比较好。

数学知识

参考1
参考2

在拟合过程中会用到一些数学的基础知识，长时间不用，很容易忘记，这里简单介绍下，有助于代码的理解。
以直线模型为例，即$y=kx+b$或者是多项式的写法$y=a0+a1x$，由此也可以泛化到n次多项式中，即曲线模型。
将拟合方程转换成矩阵形式：$$XA=Y$$矩阵$A=(k,b{)}^{-1}$即为所求的模型系数。
$$A=(X^{T}X{)}^{-1}{X}^{T}Y$$
若X为满秩的方阵，则：
$$A=(X{)}^{-1}Y$$

代码实现

伪代码

参考链接
参考链接

 伪码形式的算法如下所示：
输入：
data —— 一组观测数据
model —— 适应于数据的模型
n —— 适用于模型的最少数据个数
k —— 算法的迭代次数
t —— 用于决定数据是否适应于模型的阀值
d —— 判定模型是否适用于数据集的数据数目
输出：
best_model —— 跟数据最匹配的模型参数（如果没有找到好的模型，返回null）
best_consensus_set —— 估计出模型的数据点
best_error —— 跟数据相关的估计出的模型错误

iterations = 0
best_model = null
best_consensus_set = null
best_error = 无穷大
while ( iterations < k )
    maybe_inliers = 从数据集中随机选择n个点
    maybe_model = 适合于maybe_inliers的模型参数
    consensus_set = maybe_inliers

    for ( 每个数据集中不属于maybe_inliers的点 ）
        if ( 如果点适合于maybe_model，且错误小于t ）
            将点添加到consensus_set
    if （ consensus_set中的元素数目大于d ）
        已经找到了好的模型，现在测试该模型到底有多好
        better_model = 适合于consensus_set中所有点的模型参数
        this_error = better_model究竟如何适合这些点的度量
        if ( this_error < best_error )
            我们发现了比以前好的模型，保存该模型直到更好的模型出现
            best_model =  better_model
            best_consensus_set = consensus_set
            best_error =  this_error
    增加迭代次数
返回 best_model, best_consensus_set, best_error

    RANSAC算法的可能变化包括以下几种：
    （1）如果发现了一种足够好的模型（该模型有足够小的错误率），则跳出主循环。这样可能会节约计算额外参数的时间。
    （2）直接从maybe_model计算this_error，而不从consensus_set重新估计模型。这样可能会节约比较两种模型错误的时间，但可能会对噪声更敏感。

Given:
    data – A set of observations.
    model – A model to explain observed data points.
    n – Minimum number of data points required to estimate model parameters.
    k – Maximum number of iterations allowed in the algorithm.
    t – Threshold value to determine data points that are fit well by model.
    d – Number of close data points required to assert that a model fits well to data.

Return:
    bestFit – model parameters which best fit the data (or null if no good model is found)

iterations = 0
bestFit = null
bestErr = something really large

while iterations < k do
    maybeInliers := n randomly selected values from data
    maybeModel := model parameters fitted to maybeInliers
    alsoInliers := empty set
    for every point in data not in maybeInliers do
        if point fits maybeModel with an error smaller than t
             add point to alsoInliers
        end if
    end for
    if the number of elements in alsoInliers is > d then
        // This implies that we may have found a good model
        // now test how good it is.
        betterModel := model parameters fitted to all points in maybeInliers and alsoInliers
        thisErr := a measure of how well betterModel fits these points
        if thisErr < bestErr then
            bestFit := betterModel
            bestErr := thisErr
        end if
    end if
    increment iterations
end while

return bestFit

python代码实现

# 测试发现每次拟合结果都不一样，还是有很多次拟合不准，需要多调参
def ransac(points:np.ndarray, best_model=None, max_iters=2000, min_samples=2, num_inliers=10,
                residual_threshold=0.1, best_error=np.inf, probability=0.99):
    """Determine number trials such that at least one outlier-free subset is
    sampled for the given inlier/outlier ratio.
    Parameters
    ----------
    n_inliers : int
        Number of inliers in the data.
    n_samples : int
        Total number of samples in the data.
    min_samples : int
        Minimum number of samples chosen randomly from original data.
    probability : float
        Probability (confidence) that one outlier-free sample is generated.
    Returns
    -------
    """
    rng = np.random.default_rng()
    _EPSILON = np.spacing(1)
    n_samples = points.shape[0]
    best_inliers = 0
    iterations = 0
    best_iters = 0
    x_points = points[:, 0].reshape(-1,1)
    y_points = points[:, 1].reshape(-1,1)
    while iterations < max_iters:
        print(f"-----------------------------iterations {iterations}----------------------------")
        ridx = rng.permutation(points.shape[0])
        select_inliers = ridx[:min_samples] #随机排序后取前几个
        r = x_points[select_inliers, :].shape[0]
        X = x_points[select_inliers]
        X = np.hstack([np.ones((r, 1)), X])
        Y = y_points[select_inliers]
        try:
            np.linalg.inv(X.T @ X)
        except:
            print('存在不可逆')
            continue
        params = np.linalg.inv(X.T @ X) @ X.T @ Y # A = (x.T*x)^-1 * x.T * Y

        y_true = y_points[ridx][min_samples:]
        y_pred = np.hstack([np.ones((x_points[ridx][min_samples:].shape[0], 1)), x_points[ridx][min_samples:]]) @ params

        residual = y_true - y_pred
        inliers_ids = np.where(abs(residual) < residual_threshold)[0]
        # inliers = []
        # for k in range(n_samples):
        #     y_hat = slope * X[k][0] + intercept
        #     if abs(y[k] - y_hat) < residual_threshold:
        #         inliers.append(k)
        inliers = inliers_ids.size
        if inliers > best_inliers:
            best_inliers = inliers
            best_model = params

        iterations += 1

        # 求最大迭代次数
        # max_iters = np.ceil(np.log(max(_EPSILON, 1 - probability)) / 
        #     np.log(max(_EPSILON, 1 - (inliers / n_samples) ** min_samples)))

        if inliers > num_inliers:
            inliers_points = np.hstack([select_inliers, inliers_ids])
            x_inliers = x_points[inliers_points]
            x_matrix = np.hstack([np.ones((x_inliers.shape[0], 1)), x_inliers])
            y_inliers = y_points[inliers_points]
            better_model = np.linalg.inv(x_matrix.T @ x_matrix) @ x_matrix.T @ y_inliers
            y_pred_better = x_matrix @ better_model
            mean_square_error = np.sum((y_inliers - y_pred_better) ** 2) / y_inliers.shape[0]
            if mean_square_error < best_error:
                best_error = mean_square_error
                best_model = better_model
                best_iters = iterations
                # best_slope = best_model[1,0]
                # best_intercept = best_model[0,0]
        
        print('max_iters: ', max_iters, best_iters)
        print(f"#n_liers: {inliers}, #best_n_inliers: {best_inliers}, best_error:{best_error}")
        
    print(f"quit after {iterations} iterations")
    
    return best_model, best_inliers

from copy import copy
import numpy as np
from numpy.random import default_rng
rng = default_rng()


class RANSAC:
    def __init__(self, n=10, k=100, t=0.05, d=10, model=None, loss=None, metric=None):
        self.n = n              # `n`: Minimum number of data points to estimate parameters
        self.k = k              # `k`: Maximum iterations allowed
        self.t = t              # `t`: Threshold value to determine if points are fit well
        self.d = d              # `d`: Number of close data points required to assert model fits well
        self.model = model      # `model`: class implementing `fit` and `predict`
        self.loss = loss        # `loss`: function of `y_true` and `y_pred` that returns a vector
        self.metric = metric    # `metric`: function of `y_true` and `y_pred` and returns a float
        self.best_fit = None
        self.best_error = np.inf

    def fit(self, X, y):

        for _ in range(self.k):
            ids = rng.permutation(X.shape[0])

            maybe_inliers = ids[: self.n]
            maybe_model = copy(self.model).fit(X[maybe_inliers], y[maybe_inliers])

            thresholded = (
                self.loss(y[ids][self.n :], maybe_model.predict(X[ids][self.n :]))
                < self.t
            )

            inlier_ids = ids[self.n :][np.flatnonzero(thresholded).flatten()]

            if inlier_ids.size > self.d:
                inlier_points = np.hstack([maybe_inliers, inlier_ids])
                better_model = copy(self.model).fit(X[inlier_points], y[inlier_points])

                this_error = self.metric(
                    y[inlier_points], better_model.predict(X[inlier_points])
                )

                if this_error < self.best_error:
                    self.best_error = this_error
                    self.best_fit = maybe_model

        return self

    def predict(self, X):
        return self.best_fit.predict(X)


def square_error_loss(y_true, y_pred):
    return (y_true - y_pred) ** 2

def mean_square_error(y_true, y_pred):
    return np.sum(square_error_loss(y_true, y_pred)) / y_true.shape[0]

class LinearRegressor:
    def __init__(self):
        self.params = None

    def fit(self, X: np.ndarray, y: np.ndarray):
        r, _ = X.shape
        X = np.hstack([np.ones((r, 1)), X])
        self.params = np.linalg.inv(X.T @ X) @ X.T @ y
        return self

    def predict(self, X: np.ndarray):
        r, _ = X.shape
        X = np.hstack([np.ones((r, 1)), X])
        return X @ self.params

if __name__ == "__main__":

    regressor = RANSAC(model=LinearRegressor(), loss=square_error_loss, metric=mean_square_error)

    X = np.array([-0.848,-0.800,-0.704,-0.632,-0.488,-0.472,-0.368,-0.336,-0.280,-0.200,-0.00800,-0.0840,0.0240,0.100,0.124,0.148,0.232,0.236,0.324,0.356,0.368,0.440,0.512,0.548,0.660,0.640,0.712,0.752,0.776,0.880,0.920,0.944,-0.108,-0.168,-0.720,-0.784,-0.224,-0.604,-0.740,-0.0440,0.388,-0.0200,0.752,0.416,-0.0800,-0.348,0.988,0.776,0.680,0.880,-0.816,-0.424,-0.932,0.272,-0.556,-0.568,-0.600,-0.716,-0.796,-0.880,-0.972,-0.916,0.816,0.892,0.956,0.980,0.988,0.992,0.00400]).reshape(-1,1)
    y = np.array([-0.917,-0.833,-0.801,-0.665,-0.605,-0.545,-0.509,-0.433,-0.397,-0.281,-0.205,-0.169,-0.0531,-0.0651,0.0349,0.0829,0.0589,0.175,0.179,0.191,0.259,0.287,0.359,0.395,0.483,0.539,0.543,0.603,0.667,0.679,0.751,0.803,-0.265,-0.341,0.111,-0.113,0.547,0.791,0.551,0.347,0.975,0.943,-0.249,-0.769,-0.625,-0.861,-0.749,-0.945,-0.493,0.163,-0.469,0.0669,0.891,0.623,-0.609,-0.677,-0.721,-0.745,-0.885,-0.897,-0.969,-0.949,0.707,0.783,0.859,0.979,0.811,0.891,-0.137]).reshape(-1,1)

    regressor.fit(X, y)

    import matplotlib.pyplot as plt
    plt.style.use("seaborn-darkgrid")
    fig, ax = plt.subplots(1, 1)
    ax.set_box_aspect(1)

    plt.scatter(X, y)

    line = np.linspace(-1, 1, num=100).reshape(-1, 1)
    plt.plot(line, regressor.predict(line), c="peru")
    plt.show()