HDFS NameNode DataNode 多目录配置
现在目录结构
NameNode
NameNode 的本地目录可以配置成多个，且每个目录存放内容相同，增加了可靠性。
配置 hdfs-site.xml
<property>
<name>dfs.namenode.name.dirname>
<value>file:///${hadoop.tmp.dir}/dfs/name1,file:///${hadoop.tmp.dir}/dfs/name2value>
property>
分发到其他节点。
停止集群
[root@hadoop103 hadoop]# stop-yarn.sh
stopping yarn daemons
stopping resourcemanager
hadoop102: stopping nodemanager
hadoop104: stopping nodemanager
hadoop103: stopping nodemanager
no proxyserver to stop
[root@hadoop102 dfs]# stop-dfs.sh
Stopping namenodes on [hadoop102]
hadoop102: stopping namenode
hadoop103: stopping datanode
hadoop104: stopping datanode
hadoop102: stopping datanode
hadoop105: ssh: connect to host hadoop105 port 22: No route to host
Stopping secondary namenodes [hadoop104]
hadoop104: stopping secondarynamenode
删除 data 和 logs 中所有数据。
[root@hadoop102 hadoop-2.7.2]# rm -rf data/ logs/ [root@hadoop103 hadoop-2.7.2]# rm -rf data/ logs/ [root@hadoop104 hadoop-2.7.2]# rm -rf data/ logs/
格式化集群并启动
格式化
[root@hadoop102 hadoop-2.7.2]# hdfs namenode -format
启动
[root@hadoop102 hadoop-2.7.2]# start-dfs.sh
Starting namenodes on [hadoop102]
hadoop102: starting namenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-namenode-hadoop102.out
hadoop102: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-datanode-hadoop102.out
hadoop103: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-datanode-hadoop103.out
hadoop104: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-datanode-hadoop104.out
Starting secondary namenodes [hadoop104]
hadoop104: starting secondarynamenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-secondarynamenode-hadoop104.out
查看
[root@hadoop102 dfs]# pwd
/opt/module/hadoop-2.7.2/data/tmp/dfs
[root@hadoop102 dfs]# ll
总用量 12
drwx------. 3 root root 4096 4月 3 13:21 data
drwxr-xr-x. 3 root root 4096 4月 3 13:21 name1
drwxr-xr-x. 3 root root 4096 4月 3 13:21 name2
[root@hadoop102 dfs]#
Datanode
datanode 配置成多个目录，每个目录存储的数据不一样。即:数据不是副本。
配置 hdfs-site.xml
<property>
<name>dfs.datanode.data.dirname>
<value>file:///${hadoop.tmp.dir}/dfs/data1,file:///${hadoop.tmp.dir}/dfs/data2value>
property>
分发到各节点。
停止集群
[root@hadoop102 dfs]# stop-dfs.sh
Stopping namenodes on [hadoop102]
hadoop102: stopping namenode
hadoop102: stopping datanode
hadoop103: stopping datanode
hadoop104: stopping datanode
Stopping secondary namenodes [hadoop104]
hadoop104: stopping secondarynamenode
删除 data 和 logs 中所有数据。
[root@hadoop102 hadoop-2.7.2]# rm -rf data/ logs/ [root@hadoop103 hadoop-2.7.2]# rm -rf data/ logs/ [root@hadoop104 hadoop-2.7.2]# rm -rf data/ logs/
格式化集群并启动
格式化
[root@hadoop102 hadoop-2.7.2]# hdfs namenode -format
启动
[root@hadoop102 hadoop-2.7.2]# start-dfs.sh
Starting namenodes on [hadoop102]
hadoop102: starting namenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-namenode-hadoop102.out
hadoop103: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-datanode-hadoop103.out
hadoop102: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-datanode-hadoop102.out
hadoop104: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-datanode-hadoop104.out
Starting secondary namenodes [hadoop104]
hadoop104: starting secondarynamenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-root-secondarynamenode-hadoop104.out
查看
[root@hadoop102 dfs]# pwd
/opt/module/hadoop-2.7.2/data/tmp/dfs
[root@hadoop102 dfs]# ll
总用量 16
drwx------. 3 root root 4096 4月 3 13:31 data1
drwx------. 3 root root 4096 4月 3 13:31 data2
drwxr-xr-x. 3 root root 4096 4月 3 13:31 name1
drwxr-xr-x. 3 root root 4096 4月 3 13:31 name2
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-C2tJlOS0-1595569036879)(https://mweb.yljphp.com/2019-04-03-15542695530213.jpg)]
Web UI
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-erB21CXb-1595569036894)(https://mweb.yljphp.com/2019-04-03-15542697700758.jpg)]