hadoop缩减datanode

1、配置NameNode的hdfs-site.xml，适当减小dfs.replication副本数，增加dfs.hosts.exclude配置

 
    dfs.hosts.exclude
    /usr/local/hadoop2/etc/hadoop/excludes
  

2、在对应路径（/etc/hadoop/）下新建excludes文件，并写入待删除DataNode的ip或域名
3、在NameNode上刷新所有DataNode

..bin/hdfs dfsadmin -refreshNodes
..sbin/start-balancer.sh

4、此时，可以在web检测界面（ip:50070）上可以观测到DataNode逐渐变为Dead。或者也可以用以下命令查看

hdfs dfsadmin -report
Configured Capacity: 870772527104 (810.97 GB)
Present Capacity: 869556375552 (809.84 GB)
DFS Remaining: 865522323456 (806.08 GB)
DFS Used: 4034052096 (3.76 GB)
DFS Used%: 0.46%
Under replicated blocks: 195
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0

-------------------------------------------------
Live datanodes (1):

Name: 10.153.204.19:8011 (XXXXXXXXXXXX)
Hostname: XXXXXXXXXXXX
Decommission Status : Normal
Configured Capacity: 870772527104 (810.97 GB)
DFS Used: 4034052096 (3.76 GB)
Non DFS Used: 1216151552 (1.13 GB)
DFS Remaining: 865522323456 (806.08 GB)
DFS Used%: 0.46%
DFS Remaining%: 99.40%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Apr 04 14:45:01 CST 2019


Dead datanodes (1):

Name: 10.153.204.20:8011 (XXXXXXXXXX)
Hostname: XXXXXXXXX
Decommission Status : Decommissioned
Configured Capacity: 0 (0 B)
DFS Used: 0 (0 B)
Non DFS Used: 0 (0 B)
DFS Remaining: 0 (0 B)
DFS Used%: 100.00%
DFS Remaining%: 0.00%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 0
Last contact: Wed Apr 03 11:11:12 CST 2019

主要是看Decommission Status 字段，Decommissioned（退役的），如果只是节点的datanode没有启动，或者服务器就没启动，这个字段会是Normal并且提示Dead datanodes (1):