datawhale6

我们拥有的泰坦尼克号的数据集，那么我们这次的目的就是，完成泰坦尼克号存活预测这个任务。
In [1]:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from IPython.display import Image
In [2]:

%matplotlib inline
In [3]:

plt.rcParams[‘font.sans-serif’] = [‘SimHei’] # 用来正常显示中文标签
plt.rcParams[‘axes.unicode_minus’] = False # 用来正常显示负号
plt.rcParams[‘figure.figsize’] = (10, 6) # 设置输出图片大小
载入这些库，如果缺少某些库，请安装他们
【思考】这些库的作用是什么呢？你需要查一查
In [4]:

#思考题回答
​
​
In [5]:

%matplotlib inline
载入我们提供清洗之后的数据(clear_data.csv)，大家也将原始数据载入（train.csv），说说他们有什么不同
In [9]:

#写入代码
data = pd.read_csv(‘clear_data.csv’)
data.head()
​
​
Out[9]:
PassengerId Pclass Age SibSp Parch Fare Sex_female Sex_male Embarked_C Embarked_Q Embarked_S
0 0 3 22.0 1 0 7.2500 0 1 0 0 1
1 1 1 38.0 1 0 71.2833 1 0 1 0 0
2 2 3 26.0 0 0 7.9250 1 0 0 0 1
3 3 1 35.0 1 0 53.1000 1 0 0 0 1
4 4 3 35.0 0 0 8.0500 0 1 0 0 1
In [10]:

#写入代码
train = pd.read_csv(‘train.csv’)
train.head()
​
​
Out[10]:
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th… female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
In [8]:

#写入代码
​
​
​
模型搭建
处理完前面的数据我们就得到建模数据，下一步是选择合适模型
在进行模型选择之前我们需要先知道数据集最终是进行监督学习还是无监督学习
模型的选择一方面是通过我们的任务来决定的。
除了根据我们任务来选择模型外，还可以根据数据样本量以及特征的稀疏性来决定
刚开始我们总是先尝试使用一个基本的模型来作为其baseline，进而再训练其他模型做对比，最终选择泛化能力或性能比较好的模型
这里我的建模，并不是从零开始，自己一个人完成完成所有代码的编译。我们这里使用一个机器学习最常用的一个库（sklearn）来完成我们的模型的搭建
下面给出sklearn的算法选择路径，供大家参考
In [9]:

sklearn模型算法选择路径图

Image(‘sklearn.png’)
Out[9]:

【思考】数据集哪些差异会导致模型在拟合数据是发生变化
In [2]:

#思考回答
​
​
​
任务一：切割训练集和测试集
这里使用留出法划分数据集
将数据集分为自变量和因变量
按比例切割训练集和测试集(一般测试集的比例有30%、25%、20%、15%和10%)
使用分层抽样
设置随机种子以便结果能复现
【思考】
划分数据集的方法有哪些？
为什么使用分层抽样，这样的好处有什么？
任务提示1
切割数据集是为了后续能评估模型泛化能力
sklearn中切割数据集的方法为train_test_split
查看函数文档可以在jupyter noteboo里面使用train_test_split?后回车即可看到
分层和随机种子在参数里寻找
要从clear_data.csv和train.csv中提取train_test_split()所需的参数
In [11]:

#写入代码
from sklearn.model_selection import train_test_split
​
​
In [12]:

#写入代码
​
X = data
y = train[‘Survived’]
​
In [13]:

#写入代码
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=0)
​
​
In [14]:

#写入代码
X_train.shape, X_test.shape
​
​
Out[14]:
((668, 11), (223, 11))
【思考】
什么情况下切割数据集的时候不用进行随机选取
In [52]:

#思考回答
​
​
​
任务二：模型创建
创建基于线性模型的分类模型（逻辑回归）
创建基于树的分类模型（决策树、随机森林）
分别使用这些模型进行训练，分别的到训练集和测试集的得分
查看模型的参数，并更改参数值，观察模型变化
提示
逻辑回归不是回归模型而是分类模型，不要与LinearRegression混淆
随机森林其实是决策树集成为了降低决策树过拟合的情况
线性模型所在的模块为sklearn.linear_model
树模型所在的模块为sklearn.ensemble
In [15]:

#写入代码
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
​
​
In [16]:

#写入代码
lr = LogisticRegression()
lr.fit(X_train, y_train)
​
​
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
Out[16]:
LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, l1_ratio=None, max_iter=100,
multi_class=‘auto’, n_jobs=None, penalty=‘l2’,
random_state=None, solver=‘lbfgs’, tol=0.0001, verbose=0,
warm_start=False)
In [17]:

#写入代码
print(“Training set score: {:.2f}”.format(lr.score(X_train, y_train)))
print(“Testing set score: {:.2f}”.format(lr.score(X_test, y_test)))
​
​
Training set score: 0.80
Testing set score: 0.79
In [18]:

#写入代码
lr2 = LogisticRegression(C=100)
lr2.fit(X_train, y_train)
​
​
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
Out[18]:
LogisticRegression(C=100, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, l1_ratio=None, max_iter=100,
multi_class=‘auto’, n_jobs=None, penalty=‘l2’,
random_state=None, solver=‘lbfgs’, tol=0.0001, verbose=0,
warm_start=False)
In [19]:

print(“Training set score: {:.2f}”.format(lr2.score(X_train, y_train)))
print(“Testing set score: {:.2f}”.format(lr2.score(X_test, y_test)))
Training set score: 0.79
Testing set score: 0.78
In [20]:

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)
Out[20]:
RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,
criterion=‘gini’, max_depth=None, max_features=‘auto’,
max_leaf_nodes=None, max_samples=None,
min_impurity_decrease=0.0, min_impurity_split=None,
min_samples_leaf=1, min_samples_split=2,
min_weight_fraction_leaf=0.0, n_estimators=100,
n_jobs=None, oob_score=False, random_state=None,
verbose=0, warm_start=False)
In [21]:

print(“Training set score: {:.2f}”.format(rfc.score(X_train, y_train)))
print(“Testing set score: {:.2f}”.format(rfc.score(X_test, y_test)))
Training set score: 1.00
Testing set score: 0.82
In [22]:

rfc2 = RandomForestClassifier(n_estimators=100, max_depth=5)
rfc2.fit(X_train, y_train)
Out[22]:
RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,
criterion=‘gini’, max_depth=5, max_features=‘auto’,
max_leaf_nodes=None, max_samples=None,
min_impurity_decrease=0.0, min_impurity_split=None,
min_samples_leaf=1, min_samples_split=2,
min_weight_fraction_leaf=0.0, n_estimators=100,
n_jobs=None, oob_score=False, random_state=None,
verbose=0, warm_start=False)
In [23]:

print(“Training set score: {:.2f}”.format(rfc2.score(X_train, y_train)))
print(“Testing set score: {:.2f}”.format(rfc2.score(X_test, y_test)))
Training set score: 0.86
Testing set score: 0.82
【思考】
为什么线性模型可以进行分类任务，背后是怎么的数学关系
对于多分类问题，线性模型是怎么进行分类的
In [69]:

#思考回答
​
​
任务三：输出模型预测结果
输出模型预测分类标签
输出不同分类标签的预测概率
提示3
一般监督模型在sklearn里面有个predict能输出预测标签，predict_proba则可以输出标签概率
In [24]:

#写入代码
pred = lr.predict(X_train)
​
​
In [25]:

#写入代码
pred[:10]
​
​
Out[25]:
array([0, 1, 1, 1, 0, 0, 1, 0, 1, 1])
In [26]:

#写入代码
pred_proba = lr.predict_proba(X_train)
​
​
In [27]:

#写入代码
pred_proba[:10]
​
​
Out[27]:
array([[0.60836411, 0.39163589],
[0.17813963, 0.82186037],
[0.40994022, 0.59005978],
[0.19005944, 0.80994056],
[0.87986596, 0.12013404],
[0.91371848, 0.08628152],
[0.13316471, 0.86683529],
[0.90616578, 0.09383422],
[0.05311329, 0.94688671],
[0.10966201, 0.89033799]])
加载下面的库
In [2]:

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from IPython.display import Image
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
In [3]:

%matplotlib inline
In [4]:

plt.rcParams[‘font.sans-serif’] = [‘SimHei’] # 用来正常显示中文标签
plt.rcParams[‘axes.unicode_minus’] = False # 用来正常显示负号
plt.rcParams[‘figure.figsize’] = (10, 6) # 设置输出图片大小
任务：加载数据并分割测试集和训练集
In [5]:

#写入代码
​
from sklearn.model_selection import train_test_split
In [6]:

#写入代码
data = pd.read_csv(‘clear_data.csv’)
train = pd.read_csv(‘train.csv’)
X = data
y = train[‘Survived’]
​
In [7]:

#写入代码
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=0)
​
In [8]:

#写入代码
lr = LogisticRegression()
lr.fit(X_train, y_train)
​
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
Out[8]:
LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, l1_ratio=None, max_iter=100,
multi_class=‘auto’, n_jobs=None, penalty=‘l2’,
random_state=None, solver=‘lbfgs’, tol=0.0001, verbose=0,
warm_start=False)
模型评估
模型评估是为了知道模型的泛化能力。
交叉验证（cross-validation）是一种评估泛化性能的统计学方法，它比单次划分训练集和测试集的方法更加稳定、全面。
在交叉验证中，数据被多次划分，并且需要训练多个模型。
最常用的交叉验证是 k 折交叉验证（k-fold cross-validation），其中 k 是由用户指定的数字，通常取 5 或 10。
准确率（precision）度量的是被预测为正例的样本中有多少是真正的正例
召回率（recall）度量的是正类样本中有多少被预测为正类
f-分数是准确率与召回率的调和平均
【思考】：将上面的概念进一步的理解，大家可以做一下总结
In [34]:

#思考回答：
​
​
任务一：交叉验证
用10折交叉验证来评估之前的逻辑回归模型
计算交叉验证精度的平均值
In [6]:

#提示：交叉验证
Image(‘Snipaste_2020-01-05_16-37-56.png’)
Out[6]:

提示4
交叉验证在sklearn中的模块为sklearn.model_selection
In [9]:

#写入代码
from sklearn.model_selection import cross_val_score
​
In [10]:

#写入代码
lr = LogisticRegression(C=100)
scores = cross_val_score(lr, X_train, y_train, cv=10)
​
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
In [11]:

#写入代码
scores
​
Out[11]:
array([0.82089552, 0.74626866, 0.74626866, 0.80597015, 0.86567164,
0.8358209 , 0.76119403, 0.82089552, 0.74242424, 0.75757576])
In [12]:

#写入代码
print(“Average cross-validation score: {:.2f}”.format(scores.mean()))
​
Average cross-validation score: 0.79
思考4
k折越多的情况下会带来什么样的影响？
In [35]:

#思考回答
​
​
任务二：混淆矩阵
计算二分类问题的混淆矩阵
计算精确率、召回率以及f-分数
【思考】什么是二分类问题的混淆矩阵，理解这个概念，知道它主要是运算到什么任务中的
In [37]:

#思考回答
​
​
In [40]:

#提示：混淆矩阵
Image(‘Snipaste_2020-01-05_16-38-26.png’)
Out[40]:

In [42]:

#提示：准确率 (Accuracy),精确度（Precision）,Recall,f-分数计算方法
Image(‘Snipaste_2020-01-05_16-39-27.png’)
Out[42]:

提示5
混淆矩阵的方法在sklearn中的sklearn.metrics模块
混淆矩阵需要输入真实标签和预测标签
精确率、召回率以及f-分数可使用classification_report模块
In [13]:

#写入代码
from sklearn.metrics import confusion_matrix
​
In [14]:

#写入代码
lr = LogisticRegression(C=100)
lr.fit(X_train, y_train)
​
/Users/ctwoclover/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
Out[14]:
LogisticRegression(C=100, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, l1_ratio=None, max_iter=100,
multi_class=‘auto’, n_jobs=None, penalty=‘l2’,
random_state=None, solver=‘lbfgs’, tol=0.0001, verbose=0,
warm_start=False)
In [15]:

#写入代码
pred = lr.predict(X_train)
​
In [16]:

#写入代码
confusion_matrix(y_train, pred)
​
Out[16]:
array([[354, 58],
[ 83, 173]])
In [17]:

from sklearn.metrics import classification_report
In [18]:

print(classification_report(y_train, pred))
precision recall f1-score support

       0       0.81      0.86      0.83       412
       1       0.75      0.68      0.71       256

accuracy                           0.79       668

macro avg 0.78 0.77 0.77 668
weighted avg 0.79 0.79 0.79 668

【思考】
如果自己实现混淆矩阵的时候该注意什么问题
In [43]:

#思考回答
​
​
任务三：ROC曲线
绘制ROC曲线
【思考】什么是OCR曲线，OCR曲线的存在是为了解决什么问题？
In [44]:

#思考
​
​
​
提示6
ROC曲线在sklearn中的模块为sklearn.metrics
ROC曲线下面所包围的面积越大越好
In [19]:

#写入代码
from sklearn.metrics import roc_curve
​
In [20]:

#写入代码
fpr, tpr, thresholds = roc_curve(y_test, lr.decision_function(X_test))
plt.plot(fpr, tpr, label=“ROC Curve”)
plt.xlabel(“FPR”)
plt.ylabel(“TPR (recall)”)

找到最接近于0的阈值

close_zero = np.argmin(np.abs(thresholds))
plt.plot(fpr[close_zero], tpr[close_zero], ‘o’, markersize=10, label=“threshold zero”, fillstyle=“none”, c=‘k’, mew=2)
plt.legend(loc=4)
​
Out[20]:

findfont: Font family [‘sans-serif’] not found. Falling back to DejaVu Sans.