用Python的statsmodels包做前向逐步回归

Python的statsmodels包含了一些R风格的统计模型和工具。在内部实现上,statsmodels使用patsy包将数据转化为矩阵并建立线性模型,具体信息参见pasty主页http://patsy.readthedocs.io/en/latest/overview.html。但是,Python的statsmodels工具中没有向前逐步回归算法。逐步回归的基本思想是将变量逐个引入模型,每引入一个解释变量后都要进行F检验,并对已经选入的解释变量逐个进行t检验,当原来引入的解释变量由于后面解释变量的引入变得不再显著时,则将其删除。以确保每次引入新的变量之前回归方程中只包含显著性变量。这是一个反复的过程,直到既没有显著的解释变量选入回归方程,也没有不显著的解释变量从回归方程中剔除为止。以保证最后所得到的解释变量集是最优的(https://baike.baidu.com/item/%E9%80%90%E6%AD%A5%E5%9B%9E%E5%BD%92/585832?fr=aladdin)。

网上有人用statsmodels写了一个向前逐步回归的工具,具体网址见https://planspace.org/20150423-forward_selection_with_statsmodels/。 我试了一下,速度还不错,比我用sklearn写的要好。具体代码如下:

import statsmodels.formula.api as smf
import pandas as pd

def forward_selected(data, response):
    """前向逐步回归算法,源代码来自https://planspace.org/20150423-forward_selection_with_statsmodels/
    使用Adjusted R-squared来评判新加的参数是否提高回归中的统计显著性
    Linear model designed by forward selection.

    Parameters:
    -----------
    data : pandas DataFrame with all possible predictors and response

    response: string, name of response column in data

    Returns:
    --------
    model: an "optimal" fitted statsmodels linear model
           with an intercept
           selected by forward selection
           evaluated by adjusted R-squared
    """
    remaining = set(data.columns)
    remaining.remove(response)
    selected = []
    current_score, best_new_score = 0.0, 0.0
    while remaining and current_score == best_new_score:
        scores_with_candidates = []
        for candidate in remaining:
            formula = "{} ~ {} + 1".format(response,
                                           ' + '.join(selected + [candidate]))
            score = smf.ols(formula, data).fit().rsquared_adj
            scores_with_candidates.append((score, candidate))
        scores_with_candidates.sort()
        best_new_score, best_candidate = scores_with_candidates.pop()
        if current_score < best_new_score:
            remaining.remove(best_candidate)
            selected.append(best_candidate)
            current_score = best_new_score
    formula = "{} ~ {} + 1".format(response,
                                   ' + '.join(selected))
    model = smf.ols(formula, data).fit()

    return model


def main():

    '''
    首先从网上读取普林斯顿大学52位员工的工资信息,工资文件一共有6列,各列意义解释如下:
    sx = Sex, coded 1 for female and 0 for male
    rk = Rank, coded
        1 for assistant professor,
        2 for associate professor, and
        3 for full professor
    yr = Number of years in current rank
    dg = Highest degree, coded 1 if doctorate, 0 if masters
    yd = Number of years since highest degree was earned
    sl = Academic year salary, in dollars.

    '''
    url = "http://data.princeton.edu/wws509/datasets/salary.dat"
    data = pd.read_csv(url, sep='\\s+')

    #将sl(年收入)设为目标变量
    model = forward_selected(data, 'sl')

    #打印出最后的回归模型
    print(model.model.formula)
    # sl ~ rk + yr + 1
    print(model.params)
    # Intercept          16203.268154
    # rk[T.associate]     4262.284707
    # rk[T.full]          9454.523248
    # yr                   375.695643
    # dtype: float64
    # 0.835190760538

    print(model.rsquared_adj)
    # 0.835190760538

if __name__ == '__main__':
    main()

最好,该方法也有缺点。随着模型中参数增多,Adjusted R-squared的增加不太明显,所以后来加的很多参数对模型的影响不大。下面是在我自己的数据集(包含98个参数)上做的测试图,我们看到,当参数个数多于7个时,再增加参数,模型的R2并不能显著增加,所以最终模型取前7个参数即可。

用Python的statsmodels包做前向逐步回归_第1张图片

你可能感兴趣的:(数据挖掘,Python)