搭建hadoop平台时遇到的问题及解决方法

笔者最近在搭建伪分布式hadoop平台的时候，遇到了一些问题，在解决了这些问题之后就将问题及解决方法记录了下来，以供参考。

出现的问题1

hadoop@daixiang-virtual-machine:/usr/local/hadoop/etc/hadoop$ start-dfs.sh
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hadoop-namenode-daixiang-virtual-machine.out
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hadoop-datanode-daixiang-virtual-machine.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is 65:d4:b8:4c:c1:c6:eb:b6:f8:02:7b:fd:52:80:f8:84.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hadoop-secondarynamenode-daixiang-virtual-machine.out

解决方法

删除 tmp 目录 ，重新格式化

---

出现的问题2

使用start-dfs.sh启动HDFS时发现datanode没有启动：

hadoop@daixiang-virtual-machine:/usr/local/hadoop/etc/hadoop$ jps
15158 Jps
15049 SecondaryNameNode
14751 NameNode

解决方法

查看etc/hadoop/hdfs-site.xml配置文件是否写错，一般是该配置文件写错导致的。

---

出现的问题3

hadoop@daixiang-virtual-machine:/usr/local/hadoop/etc/hadoop$ hadoop fs -mkdir input
mkdir: `input’: No such file or directory

无法stop-dfs.sh，进入到/usr/local/hadoop/sbin 执行./stop-dfs.sh可以关闭

解决方法：

权限问题，未对hadoop用户进行授权，使用命令sudo chmod -R a+w /usr/local/hadoop进行授权。

---

出现问题4：

修改文件时，出现提示：E45: ‘readonly’ option is set (add ! to override) 。无法报错更改。

解决方法：

权限问题，该错误为当前用户没有权限对文件作修改。进入到/usr/local目录，使用命令修改hadoop文件夹以及子文件夹的所属用户为hadoop用户

$ sudo chown -R hadoop ./hadoop

---

出现的问题5：

启动hdfs时报错：Error: Cannot find configuration directory: /etc/hadoop
找不到配置目录，并且没有成功启动进程。

hadoop@daixiang-virtual-machine:/usr/local$ start-dfs.sh
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
Error: Cannot find configuration directory: /etc/hadoop
Error: Cannot find configuration directory: /etc/hadoop
Starting secondary namenodes [0.0.0.0]
Error: Cannot find configuration directory: /etc/hadoop
hadoop@daixiang-virtual-machine:/usr/local$ jps
15782 Jps

解决方法：

将文件/usr/local/hadoop/etc/hadoop/hadoop-env.sh中的

export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}

改为绝对路径：

export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop