解决方案：datanode无法启动之"Incompatible namespaceIDs in ..."
启动hadoop的时候，datanode无法启动，使用如下命令查看日志信息：
tail -100f /usr/local/hadoop/libexec/../logs/hadoop-hadoop-datanode-master.log
提示以下错误：
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /data/hadoop/hdfs/data: namenode namespaceID = 1265744122; datanode namespaceID = 908607799
原因:每次namenode format会重新创建一个namenodeId,而/data/hadoop/hdfs/data下包含了上次format下的id,namenode format清空了namenode下的数据,但是没有晴空datanode下的数据,导致启动时失败,所要做的就是每次fotmat前,清空/data/hadoop/hdfs/data以下 的所有目录.
[hadoop@master name]$ cd /data/hadoop/hdfs [hadoop@master hdfs]$ ll total 16 drwxr-xr-x 6 hadoop hadoop 4096 Apr 22 18:42 data drwxrwxr-x 5 hadoop hadoop 4096 Apr 22 18:42 name
[root@master hdfs]# rm -rf data/* [root@master hdfs]# su hadoop
[hadoop@master root]$ jps 12426 Jps 11532 JobTracker 11278 NameNode 11675 TaskTracker [hadoop@master root]$ stop-all.sh Warning: $HADOOP_HOME is deprecated. stopping jobtracker master: stopping tasktracker node1: stopping tasktracker stopping namenode master: no datanode to stop node1: no datanode to stop node1: stopping secondarynamenode [hadoop@master root]$ start-all.sh Warning: $HADOOP_HOME is deprecated. starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-namenode-master.out node1: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-datanode-node1.out master: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-datanode-master.out node1: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-secondarynamenode-node1.out starting jobtracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-jobtracker-master.out master: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-tasktracker-master.out node1: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-tasktracker-node1.out [hadoop@master root]$ jps 13035 JobTracker 13243 Jps 13157 TaskTracker 12911 DataNode
此时发现DataNode启动起来了，但是NameNode却不见了，莫慌，重新格式化以下namenode即可。
[hadoop@master hadoop]$ stop-all.sh Warning: $HADOOP_HOME is deprecated. stopping jobtracker master: stopping tasktracker node1: stopping tasktracker no namenode to stop master: stopping datanode node1: stopping datanode node1: stopping secondarynamenode [hadoop@master hadoop]$ jps 13581 Jps [hadoop@master hadoop]$ hadoop namenode -format Warning: $HADOOP_HOME is deprecated. 14/04/22 18:52:54 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = master/192.168.1.100 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 1.1.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1 -r 1440782; compiled by 'hortonfo' on Thu Jan 31 02:03:24 UTC 2013 ************************************************************/ 14/04/22 18:52:54 INFO util.GSet: VM type = 32-bit 14/04/22 18:52:54 INFO util.GSet: 2% max memory = 0.61875 MB 14/04/22 18:52:54 INFO util.GSet: capacity = 2^17 = 131072 entries 14/04/22 18:52:54 INFO util.GSet: recommended=131072, actual=131072 14/04/22 18:52:54 INFO namenode.FSNamesystem: fsOwner=hadoop 14/04/22 18:52:54 INFO namenode.FSNamesystem: supergroup=supergroup 14/04/22 18:52:54 INFO namenode.FSNamesystem: isPermissionEnabled=false 14/04/22 18:52:54 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100 14/04/22 18:52:54 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s) 14/04/22 18:52:54 INFO namenode.NameNode: Caching file names occuring more than 10 times 14/04/22 18:52:55 INFO common.Storage: Image file of size 112 saved in 0 seconds. 14/04/22 18:52:55 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/data/hadoop/hdfs/name/current/edits 14/04/22 18:52:55 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/data/hadoop/hdfs/name/current/edits 14/04/22 18:52:55 INFO common.Storage: Storage directory /data/hadoop/hdfs/name has been successfully formatted. 14/04/22 18:52:55 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at master/192.168.1.100 ************************************************************/ [hadoop@master hadoop]$ jps 13631 Jps [hadoop@master hadoop]$ start-all.sh Warning: $HADOOP_HOME is deprecated. starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-namenode-master.out node1: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-datanode-node1.out master: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-datanode-master.out node1: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-secondarynamenode-node1.out starting jobtracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-jobtracker-master.out node1: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-tasktracker-node1.out master: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-tasktracker-master.out [hadoop@master hadoop]$ jps 13699 NameNode 13972 JobTracker 14107 TaskTracker 13823 DataNode 14179 Jps
发现，所有的节点已经启动起来了，OK，大功告成！