最近遇到个问题,每次重启Hadoop发现HDFS控制台都无法访问,并且查看nameNode的JOBTracker日志,如下
2015-02-05 09:36:18,935 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.net.ConnectException: Call to haier002/10.255.254.3:9099 failed on connection exception: java.net.ConnectException: Connection refused
secondNameNode讲解
http://blog.chinaunix.net/uid-20577907-id-3524135.html
SecondNameNode,会备份文件到edit.new文件中,然后重启的时候加载此文件,详细功能间上面网址
突然我就想到是不是由于edit.new文件太大导致加载缓慢,服务一直处于加载中。
结果查看edit.new 100+G,原来是这样。
下面edit.new的存放路径查找方法。
在nameNode服务器
查看Hadoop目录conf下,core-site.xml或者在hdfs-site.xml找到
<property> <name>dfs.data.dir</name> <value>/opt/liveEpg/hadoop/working/data</value> <description>Determines where on the local filesystem an DFS data node should store its blocks. If this is a comma-delimited list of directories, then data will be stored in all named directories, typically on different devices. Directories that do not exist are ignored. </description> </property>
就可以看到edit.new文件,使用命令 du -sh ./* 我们就可以查看文件大小,将此文件删除掉即可。
[hadoop@haier002 current]$ du -sh ./* 106G ./edits 24M ./fsimage 4.0K ./fstime 4.0K ./VERSION [hadoop@haier002 current]$ ls edits fsimage fstime VERSION [hadoop@haier002 current]$ > edits