StatQuest学习笔记|用R实现随机森林

Random Forests通过构建一系列决策树,来实现对结果的预测。
在大多数机器学习中,我们需要划分训练集和测试集。但是随机森林会自动帮我们划分"train"和"test",因为随意森林的原理是先随机选中某些数据(可重复)构成bootstrapped data,所以不是所有的原始数据都用来构建决策树,未选中构成bootstrapped data的数据叫做the "Out-Of-Bag" (OOB) data,可以用来做测试集。

加载包

library(ggplot2)
library(cowplot)
library(randomForest)

ggplot2是画图包,cowplot可以让图片更美丽,randomForest帮助我们 随机森林。

获取数据

从UCI machine learning(网站)上获取有关心脏病的数据

url <- "http://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/processed.cleveland.data"
 
data <- read.csv(url, header=FALSE)

数据清理

现在的数据没有列名,我们需要从网站上获取列名并将其导入,网上对数据介绍的很详细。

colnames(data) <- c(
  "age",
  "sex",# 0 = female, 1 = male
  "cp", # chest pain
          # 1 = typical angina,
          # 2 = atypical angina,
          # 3 = non-anginal pain,
          # 4 = asymptomatic
  "trestbps", # resting blood pressure (in mm Hg)
  "chol", # serum cholestoral in mg/dl
  "fbs",  # fasting blood sugar greater than 120 mg/dl, 1 = TRUE, 0 = FALSE
  "restecg", # resting electrocardiographic results
          # 1 = normal
          # 2 = having ST-T wave abnormality
          # 3 = showing probable or definite left ventricular hypertrophy
  "thalach", # maximum heart rate achieved
  "exang",   # exercise induced angina, 1 = yes, 0 = no
  "oldpeak", # ST depression induced by exercise relative to rest
  "slope", # the slope of the peak exercise ST segment
          # 1 = upsloping
          # 2 = flat
          # 3 = downsloping
  "ca", # number of major vessels (0-3) colored by fluoroscopy
  "thal", # this is short of thalium heart scan
          # 3 = normal (no cold spots)
          # 6 = fixed defect (cold spots during rest and exercise)
          # 7 = reversible defect (when cold spots only appear during exercise)
  "hd" # (the predicted attribute) - diagnosis of heart disease
          # 0 if less than or equal to 50% diameter narrowing
          # 1 if greater than 50% diameter narrowing
  )

str()可以帮助我们看数据结构,我们需要针对数据结构进行清理,比如说:
sex对应的数据类型应该是factor
出现'?'需要用NA代替。

#替换NA
data[data == "?"] <- NA
#将sex替换成Famle,male
data[data$sex == 0,]$sex <- "F"
data[data$sex == 1,]$sex <- "M"
#转换数据类型为factor
data$sex <- as.factor(data$sex)
 
data$cp <- as.factor(data$cp)
data$fbs <- as.factor(data$fbs)
data$restecg <- as.factor(data$restecg)
data$exang <- as.factor(data$exang)
data$slope <- as.factor(data$slope)
 #有些列在将'?'转化成NA的时候数据类型变成了字符串,但是这一列本身是整数,所以先转化成整数,再转化为factor,这两列分别是ca和thal
data$ca <- as.integer(data$ca) # since this column had "?"s in it (which
                               # we have since converted to NAs) R thinks that
                               # the levels for the factor are strings, but
                               # we know they are integers, so we'll first
                               # convert the strings to integiers...
data$ca <- as.factor(data$ca)  # ...then convert the integers to factor levels
 
data$thal <- as.integer(data$thal) # "thal" also had "?"s in it.
data$thal <- as.factor(data$thal)
 #hd代表心脏病,将其由0或一转换为健康或不健康,然后再转为factor
data$hd <- ifelse(test=data$hd == 0, yes="Healthy", no="Unhealthy")
data$hd <- as.factor(data$hd) # Now convert to a factor
清理之前
清理之后

数据填充

将缺失值填充

#设置随机数
set.seed(42)
#对数据进行imputed
#hd代表心脏病,‘hd ~ .’代表我们希望预测hd
#建立六个森林来预测缺失值“ iter=6”,一般4~6个就够了
#通过imputed我们将data中原有的NA替换成可能的
data.imputed <- rfImpute(hd ~ ., data = data, iter=6)

#随机森林
```r
#我们仍然需要预测hd
#proximity=TRUE可以返回矩阵,帮助我们cluster这些数据(可以做热土,MDS等等)
model <- randomForest(hd ~ ., data=data.imputed, proximity=TRUE)
model
model结果

在这个例子中,主要用来分类;
每个森林中有500个决策树;每次构建nodel的时候,会随机抽取三个变量来选择最合适的,回归树和分类树都有默认值,这里面是默认的3,但是我们不知道每次抽3个合不合适,所以可以在接下来的过程中调整参数;
预测的错误率为16.5%
预测正确了141个健康人群和112个不健康的,有27个假阳性(原本无病被预测称有病)和23个假阴性(原本有病但是预测无病)。

将错误率通过ggplot2()画出来,来判断我们建立的森林如何

error.rate

第一列是OOB错误率,第二列是判断healthy错误率,第三列是判断unhealthy错误率
第一行是一个决策树导致的错误率,第二行是前两个树导致的错误率,第500行是前五百个决策树导致的错误率
将这些数据通过如下代码转化为如下矩阵


树,错误类型,错误率
oob.error.data <- data.frame(
  Trees=rep(1:nrow(model$err.rate), times=3),
  Type=rep(c("OOB", "Healthy", "Unhealthy"), each=nrow(model$err.rate)),
  Error=c(model$err.rate[,"OOB"],
    model$err.rate[,"Healthy"],
    model$err.rate[,"Unhealthy"]))
ggplot(data=oob.error.data, aes(x=Trees, y=Error)) +
 geom_line(aes(color=Type))
树越多,错误率越低

如果设置成1000个决策树呢?

model <- randomForest(hd ~ ., data=data.imputed, ntree=1000, proximity=TRUE)
model
 
oob.error.data <- data.frame(
  Trees=rep(1:nrow(model$err.rate), times=3),
  Type=rep(c("OOB", "Healthy", "Unhealthy"), each=nrow(model$err.rate)),
  Error=c(model$err.rate[,"OOB"],
    model$err.rate[,"Healthy"],
    model$err.rate[,"Unhealthy"]))
 
ggplot(data=oob.error.data, aes(x=Trees, y=Error)) +
  geom_line(aes(color=Type))
错误率跟以前一样

似乎500~1000之后没有什么差别

如果在每次决定nodel的时候考虑更多因素而不是3个呢?

写一个循环来看每次考虑1,2,3,...,10个因素的时候,错误率分别是多少,通过mtry指定树的个数,将结果保存在obb.values

oob.values <- vector(length=10)
for(i in 1:10) {
  temp.model <- randomForest(hd ~ ., data=data.imputed, mtry=i, ntree=1000)
  oob.values[i] <- temp.model$err.rate[nrow(temp.model$err.rate),1]
}
oob.values
可以看到默认的3就是最好的

最后用随机森林的结果来画MDS

因为随机森林用到一个概念proximity,而画MDS需要的距离可以通过1-proximity)所得到

distance.matrix <- dist(1-model$proximity)
mds.stuff <- cmdscale(distance.matrix, eig=TRUE, x.ret=TRUE)
mds.var.per <- round(mds.stuff$eig/sum(mds.stuff$eig)*100, 1)
mds.values <- mds.stuff$points
mds.data <- data.frame(Sample=rownames(mds.values),
  X=mds.values[,1],
  Y=mds.values[,2],
  Status=data.imputed$hd)
 
ggplot(data=mds.data, aes(x=X, y=Y, label=Sample)) +
  geom_text(aes(color=Status)) +
  theme_bw() +
  xlab(paste("MDS1 - ", mds.var.per[1], "%", sep="")) +
  ylab(paste("MDS2 - ", mds.var.per[2], "%", sep="")) +
  ggtitle("MDS plot using (1 - Random Forest Proximities)")
可以看到病人被很好的分类了,有病的在左边,没病的在右边,并且相同距离,在水平上差距更大

如果有新病人处于红圈但是没有心脏病表型记录,很不幸这个人很有可能有心脏病。

你可能感兴趣的:(StatQuest学习笔记|用R实现随机森林)