ISLR第三章线性回归应用练习题答案(上)

ISLR；R语言； 机器学习 ；线性回归

一些专业词汇只知道英语的，中文可能不标准，请轻喷

8.利用简单的线性回归处理Auto数据集

    library(MASS)
    library(ISLR)
    library(car)
    Auto=read.csv("Auto.csv",header=T,na.strings="?")
    Auto=na.omit(Auto)
    attach(Auto)
    summary(Auto)

输出结果：

        mpg          cylinders      displacement     horsepower   
   Min.   : 9.00   Min.   :3.000   Min.   : 68.0   Min.   : 46.0  
   1st Qu.:17.00   1st Qu.:4.000   1st Qu.:105.0   1st Qu.: 75.0  
   Median :22.75   Median :4.000   Median :151.0   Median : 93.5  
   Mean   :23.45   Mean   :5.472   Mean   :194.4   Mean   :104.5  
   3rd Qu.:29.00   3rd Qu.:8.000   3rd Qu.:275.8   3rd Qu.:126.0  
   Max.   :46.60   Max.   :8.000   Max.   :455.0   Max.   :230.0  

       weight      acceleration        year           origin     
   Min.   :1613   Min.   : 8.00   Min.   :70.00   Min.   :1.000  
   1st Qu.:2225   1st Qu.:13.78   1st Qu.:73.00   1st Qu.:1.000  
   Median :2804   Median :15.50   Median :76.00   Median :1.000  
   Mean   :2978   Mean   :15.54   Mean   :75.98   Mean   :1.577  
   3rd Qu.:3615   3rd Qu.:17.02   3rd Qu.:79.00   3rd Qu.:2.000  
   Max.   :5140   Max.   :24.80   Max.   :82.00   Max.   :3.000  

                   name    
   amc matador       :  5  
   ford pinto        :  5  
   toyota corolla    :  5  
   amc gremlin       :  4  
   amc hornet        :  4  
   chevrolet chevette:  4  
   (Other)           :365  

线性回归：

    lm.fit=lm(mpg~horsepower)
    summary(lm.fit)

输出结果：

 Call:
 lm(formula = mpg ~ horsepower)

 Residuals:
     Min       1Q   Median       3Q      Max 
 -13.5710  -3.2592  -0.3435   2.7630  16.9240 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
 (Intercept) 39.935861   0.717499   55.66   <2e-16 ***
 horsepower  -0.157845   0.006446  -24.49   <2e-16 ***
 ---
 Signif. codes:  0 ‘\*\*\*’ 0.001 ‘\*\*’ 0.01 ‘\*’ 0.05 ‘.’ 0.1 ‘ ’ 1

 Residual standard error: 4.906 on 390 degrees of freedom
 Multiple R-squared:  0.6059,    Adjusted R-squared:  0.6049 
 F-statistic: 599.7 on 1 and 390 DF,  p-value: < 2.2e-16

a)

• 零假设 H 0：βhorsepower=0，假设horsepower与mpg不相关。
  由于F-statistic值远大于1，p值接近于0，拒绝原假设，则horsepower和mpg具有统计显著关系。
• mpg的平均值为23.45，线性回归的RSE为4.906，有20.9248%的相对误差。R-squared为0.6059，说明60.5948%的mpg可以被horsepower解释。
• 线性回归系数小于零，说明mpg与horsepower之间的关系是消极的。

• 预测mpg

  predict(lm.fit,data.frame(mpg=c(98)),interval="prediction")
  Warning message:
  'newdata'必需有1行 但变量里有392行 
  

修改办法：

  predictor=mpg
  response=horsepower
  lm.fit2=lm(predictor~response)
  predict(lm.fit2,data.frame(response=c(98)),interval="confidence")
      fit   lwr   upr
  1 24.47 23.97 24.96
  predict(lm.fit2,data.frame(response=c(98)),interval="prediction")
       fit     lwr      upr
  1 24.46708 14.8094 34.12476

b)绘制mpg与horsepower散点图和最小二乘直线

    plot(response,predictor)
    abline(lm.fit2,lwd=3,col="red")

c)诊断最小二乘法

par(mfrow=c(2,2))
plot(lm.fit2)

有许多证据表明，mpg与horsepower非线性相关。

9.利用联合的线性回归处理Auto数据集
a)绘制散点图矩阵

pairs(Auto)

b)计算相关性矩阵

 cor(subset(Auto,select=-name))

                      mpg  cylinders displacement horsepower     weight
  mpg           1.0000000 -0.7776175   -0.8051269 -0.7784268 -0.8322442
  cylinders    -0.7776175  1.0000000    0.9508233  0.8429834  0.8975273
  displacement -0.8051269  0.9508233    1.0000000  0.8972570  0.9329944
  horsepower   -0.7784268  0.8429834    0.8972570  1.0000000  0.8645377
  weight       -0.8322442  0.8975273    0.9329944  0.8645377  1.0000000
  acceleration  0.4233285 -0.5046834   -0.5438005 -0.6891955 -0.4168392
  year          0.5805410 -0.3456474   -0.3698552 -0.4163615 -0.3091199
  origin        0.5652088 -0.5689316   -0.6145351 -0.4551715 -0.5850054
               acceleration       year     origin
  mpg             0.4233285  0.5805410  0.5652088
  cylinders      -0.5046834 -0.3456474 -0.5689316
  displacement   -0.5438005 -0.3698552 -0.6145351
  horsepower     -0.6891955 -0.4163615 -0.4551715
  weight         -0.4168392 -0.3091199 -0.5850054
  acceleration    1.0000000  0.2903161  0.2127458
  year            0.2903161  1.0000000  0.1815277
  origin          0.2127458  0.1815277  1.0000000

c)多元线性回归：

lm.fit3=lm(mpg~.-name,data=Auto)
summary(lm.fit3)

Call:
lm(formula = mpg ~ . - name, data = Auto)

Residuals:
    Min      1Q  Median      3Q     Max 
-9.5903 -2.1565 -0.1169  1.8690 13.0604 

 Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
  (Intercept)  -17.218435   4.644294  -3.707  0.00024 ***
  cylinders     -0.493376   0.323282  -1.526  0.12780    
  displacement   0.019896   0.007515   2.647  0.00844 ** 
  horsepower    -0.016951   0.013787  -1.230  0.21963    
  weight        -0.006474   0.000652  -9.929  < 2e-16 ***
  acceleration   0.080576   0.098845   0.815  0.41548    
  year           0.750773   0.050973  14.729  < 2e-16 ***
  origin         1.426141   0.278136   5.127 4.67e-07 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 3.328 on 384 degrees of freedom
  Multiple R-squared:  0.8215,    Adjusted R-squared:  0.8182 
  F-statistic: 252.4 on 7 and 384 DF,  p-value: < 2.2e-16

• 零假设 ：假设mpg与其他变量不相关。
  由于F-statistic值远大于1，p值接近于0，拒绝原假设，则mpg与其他变量具有统计显著关系。
• 参照每个变量的P值，displacement、weight 、year 、origin在统计显著关系。
• 汽车对于能源的利用率逐年增长

d)

  par(mfrow=c(2,2))
  plot(lm.fit3)

残差仍未明显的曲线，说明多元线性回归不正确。

  plot(predict(lm.fit3), rstudent(lm.fit3))

由权重图知，14号点没有较大的残差也有非常大的权重。
e)

lm.fit4=lm(mpg~displacement*weight+year*origin)
summary(lm.fit4)

运行结果：

  Call:
  lm(formula = mpg ~ displacement * weight + year * origin)

  Residuals:
      Min      1Q  Median      3Q     Max 
  -9.5758 -1.6211 -0.0537  1.3264 13.3266 

  Coefficients:
                        Estimate Std. Error t value Pr(>|t|)    
  (Intercept)          1.793e+01  8.044e+00   2.229 0.026394 *  
  displacement        -7.519e-02  9.091e-03  -8.271 2.19e-15 ***
  weight              -1.035e-02  6.450e-04 -16.053  < 2e-16 ***
  year                 4.864e-01  1.017e-01   4.782 2.47e-06 ***
  origin              -1.503e+01  4.232e+00  -3.551 0.000432 ***
  displacement:weight  2.098e-05  2.179e-06   9.625  < 2e-16 ***
  year:origin          1.980e-01  5.436e-02   3.642 0.000308 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 2.969 on 385 degrees of freedom
  Multiple R-squared:  0.8575,    Adjusted R-squared:  0.8553 
  F-statistic: 386.2 on 6 and 385 DF,  p-value: < 2.2e-16

可以发现具有统计显著关系，残差也有很大的下降。
f)

lm.fit5 = lm(mpg~log(horsepower)+sqrt(horsepower)+horsepower+I(horsepower^2))
summary(lm.fit5)

运行结果：

  Call:
  lm(formula = mpg ~ log(horsepower) + sqrt(horsepower) + horsepower + 
I(horsepower^2))

  Residuals:
       Min       1Q   Median       3Q      Max 
  -15.3450  -2.4725  -0.1594   2.1068  16.2564 

  Coefficients:
                     Estimate Std. Error t value Pr(>|t|)   
  (Intercept)      -6.839e+02  2.439e+02  -2.804  0.00530 **
  log(horsepower)   6.515e+02  2.111e+02   3.085  0.00218 **
  sqrt(horsepower) -3.385e+02  1.092e+02  -3.101  0.00207 **
  horsepower        1.165e+01  3.898e+00   2.988  0.00299 **
  I(horsepower^2)  -7.425e-03  2.796e-03  -2.655  0.00825 **
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 4.331 on 387 degrees of freedom
  Multiple R-squared:  0.6952,    Adjusted R-squared:  0.692 
  F-statistic: 220.6 on 4 and 387 DF,  p-value: < 2.2e-16

诊断回归：

  par(mfrow=c(2,2))
  plot(lm.fit5)

10.Carseats数据集
a)

  summary(Carseats)

运行结果：

     Sales          CompPrice       Income        Advertising    
   Min.   : 0.000   Min.   : 77   Min.   : 21.00   Min.   : 0.000  
   1st Qu.: 5.390   1st Qu.:115   1st Qu.: 42.75   1st Qu.: 0.000  
   Median : 7.490   Median :125   Median : 69.00   Median : 5.000  
   Mean   : 7.496   Mean   :125   Mean   : 68.66   Mean   : 6.635  
   3rd Qu.: 9.320   3rd Qu.:135   3rd Qu.: 91.00   3rd Qu.:12.000  
   Max.   :16.270   Max.   :175   Max.   :120.00   Max.   :29.000  
     Population        Price        ShelveLoc        Age          Education   
   Min.   : 10.0   Min.   : 24.0   Bad   : 96   Min.   :25.00   Min.   :10.0  
   1st Qu.:139.0   1st Qu.:100.0   Good  : 85   1st Qu.:39.75   1st Qu.:12.0  
   Median :272.0   Median :117.0   Medium:219   Median :54.50         Median :14.0  
   Mean   :264.8   Mean   :115.8                Mean   :53.32   Mean   :13.9  
   3rd Qu.:398.5   3rd Qu.:131.0                3rd Qu.:66.00   3rd Qu.:16.0  
   Max.   :509.0   Max.   :191.0                Max.   :80.00   Max.   :18.0  
   Urban       US     
   No :118   No :142  
   Yes:282   Yes:258

多元线性回归：

  attach(Carseats)
  lm.fit=lm(Sales~Price+Urban+US)
  summary(lm.fit)

运行结果：

  Call:
  lm(formula = Sales ~ Price + Urban + US)

  Residuals:
      Min      1Q  Median      3Q     Max 
  -6.9206 -1.6220 -0.0564  1.5786  7.0581 

  Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
  (Intercept) 13.043469   0.651012  20.036  < 2e-16 ***
  Price       -0.054459   0.005242 -10.389  < 2e-16 ***
  UrbanYes    -0.021916   0.271650  -0.081    0.936    
  USYes        1.200573   0.259042   4.635 4.86e-06 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 2.472 on 396 degrees of freedom
  Multiple R-squared:  0.2393,    Adjusted R-squared:  0.2335 
  F-statistic: 41.52 on 3 and 396 DF,  p-value: < 2.2e-16

b)
随着价格的升高销量下降
商场是否在郊区与销量无关
商场在美国销量会更多
c)Sales = 13.04 + -0.05 Price + -0.02 UrbanYes + 1.20 USYes
d)Priece和USYES可以，根据p值和F-statistic可以拒绝零假设。
e)

lm.fit2=lm(Sales~Price+US)
summary(lm.fit2)

输出结果：

  Call:
  lm(formula = Sales ~ Price + US)

  Residuals:
      Min      1Q  Median      3Q     Max 
  -6.9269 -1.6286 -0.0574  1.5766  7.0515 

  Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
  (Intercept) 13.03079    0.63098  20.652  < 2e-16 ***
  Price       -0.05448    0.00523 -10.416  < 2e-16 ***
  USYes        1.19964    0.25846   4.641 4.71e-06 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 2.469 on 397 degrees of freedom
  Multiple R-squared:  0.2393,    Adjusted R-squared:  0.2354 
  F-statistic: 62.43 on 2 and 397 DF,  p-value: < 2.2e-16

f）a)和e)RSE相近，但是e)稍微好一点
g)

  confint(lm.fit2)

输出结果：

                    2.5 %      97.5 %
  (Intercept) 11.79032020 14.27126531
  Price       -0.06475984 -0.04419543
  USYes        0.69151957  1.70776632

h)

  plot(predict(lm.fit2),rstudent(lm.fit2))

输出结果

所有归一化的残差都在-3到3之间，没有明显的离群值

  par(mfrow=c(2,2))
  plot(lm.fit2)

没有权重值超过(p+1)/n，说明没有明显重要的点。

11.研究t-statistic

a)

  lm.fit=lm(y~x+0)
  summary(lm.fit)

输出结果：

  Call:
  lm(formula = y ~ x + 0)

  Residuals:
       Min       1Q   Median       3Q      Max 
  -2.92110 -0.43210  0.04155  0.67849  2.64495 

  Coefficients:
    Estimate Std. Error t value Pr(>|t|)    
  x   1.9454     0.1083   17.96   <2e-16 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 1.033 on 99 degrees of freedom
  Multiple R-squared:  0.7651,    Adjusted R-squared:  0.7627 
  F-statistic: 322.4 on 1 and 99 DF,  p-value: < 2.2e-16

p值接近0，拒绝零假设
b)

  lm.fit2=lm(x~y+0)
  summary(lm.fit2)

输出结果：

  Call:
  lm(formula = x ~ y + 0)

  Residuals:
       Min       1Q   Median       3Q      Max 
  -1.05835 -0.30952 -0.01945  0.34313  1.15854 

  Coefficients:
    Estimate Std. Error t value Pr(>|t|)    
  y   0.3933     0.0219   17.96   <2e-16 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 0.4646 on 99 degrees of freedom
  Multiple R-squared:  0.7651,    Adjusted R-squared:  0.7627 
  F-statistic: 322.4 on 1 and 99 DF,  p-value: < 2.2e-16

同样p值接近0，拒绝零假设
c）a）和b）拟合的是同一条直线
d）
e）x与y地位相当，交换x，y位置t结果不变
f）

lm.fit3=lm(x~y)
summary(lm.fit3)

输出结果：

  Call:
  lm(formula = x ~ y)

  Residuals:
      Min      1Q  Median      3Q     Max 
  -1.0381 -0.2899  0.0005  0.3628  1.1782 

  Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
  (Intercept) -0.01975    0.04667  -0.423    0.673    
  y            0.39308    0.02200  17.868   <2e-16 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 0.4666 on 98 degrees of freedom
  Multiple R-squared:  0.7651,    Adjusted R-squared:  0.7627 
  F-statistic: 319.3 on 1 and 98 DF,  p-value: < 2.2e-16

x对y线性回归

  lm.fit4=lm(y~x)
  summary(lm.fit4)

输出结果：

  Call:
  lm(formula = y ~ x)

  Residuals:
       Min       1Q   Median       3Q      Max 
  -2.94807 -0.46147  0.01291  0.65020  2.61739 

  Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
  (Intercept)  0.02765    0.10391   0.266    0.791    
  x            1.94651    0.10894  17.868   <2e-16 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  Residual standard error: 1.038 on 98 degrees of freedom
  Multiple R-squared:  0.7651,    Adjusted R-squared:  0.7627 
  F-statistic: 319.3 on 1 and 98 DF,  p-value: < 2.2e-16

发现t值不变