hadoop-2.5.0-cdh5.3.0 HA在线升级

本文升级到hadoop2.6
1、hadoop升级前准备，namenode元数据和配置备份等信息参考上篇文章：http://blog.csdn.net/linux_ja/article/details/51908466


2、备份信息：
hdfs dfsadmin -rollingUpgrade prepare
hdfs dfsadmin -rollingUpgrade query


3、进入standby nn节点执行如下四个命令：
sudo rm -rf /opt/hadoop
sudo ln -s /home/hadoop/platform/hadoop-2.6.0-cdh5.4.0 /opt/hadoop
hadoop-daemon.sh stop namenode
nohup hdfs namenode -rollingUpgrade started &


注意，目前hdfs namenode -rollingUpgrade started这个命令先以后台运行，过一段时间以正常命令启动nn，
4、进入active nn节点重复执行步骤2的命令，这样就完成nn的在线升级


5、在nn节点上以此执行所有要升级的datanode，我的四个节点(hdfs.beta1,hdfs.beta2,hdfs.beta3,hdfs.beta4)
 a、hdfs dfsadmin -getDatanodeInfo hdfs.beta1:50020
 b、hdfs dfsadmin -shutdownDatanode hdfs.beta1:50020 upgrade
  c、到所要升级的datanode上执行，hadoop-daemon.sh start datanode
以此重复执行abc步骤，这就完成dn的在线升级步骤


6、如果允许一段时间没问题，可以执行：
hdfs dfsadmin -rollingUpgrade finalize，执行后就不能回滚


7、test：
hadoop jar /opt/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.4.0.jar randomwriter /rand 
hadoop jar /opt/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.4.0.jar sort rand /sort-rand 
hadoop jar /opt/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.4.0.jar wordcount  /wordcount/a.txt /wordcount/temp


8、升级之后问题：

org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: 
got premature end-of-file at txid 445259; expected file to go up to 4243581

nn和standby nn的editlog不一致

执行：hdfs namenode -bootstrapStandby