hadoop2.6.0控制集群节点个数

要控制集群节点的个数,只需要更改主控节点(master)上Hadoop安装目录下面./etc/hadoop下面的slaves文件,通过添加或删除slave的ip或ip映射的名字,然后在主控节点执行start-all.sh就可以了.

1.slaves里面放一个slave

slave1
启动集群后,查看状态:

hadoop@master:/opt/hadoop$ jps
2735 NameNode
3389 Jps
3103 ResourceManager

2948 SecondaryNameNode

hadoop@master:/opt/hadoop$ bin/hdfs dfsadmin -report
Configured Capacity: 72812462080 (67.81 GB)
Present Capacity: 64238071808 (59.83 GB)
DFS Remaining: 64236359680 (59.82 GB)
DFS Used: 1712128 (1.63 MB)
DFS Used%: 0.00%
Under replicated blocks: 8
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Live datanodes (1):

Name: 192.168.0.14:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 72812462080 (67.81 GB)
DFS Used: 1712128 (1.63 MB)
Non DFS Used: 8574390272 (7.99 GB)
DFS Remaining: 64236359680 (59.82 GB)
DFS Used%: 0.00%
DFS Remaining%: 88.22%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sun Apr 05 13:25:18 CST 2015


2.slaves里面添加一个slave

slave1
slave2


启动集群:
hadoop@master:/opt/hadoop$ sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [master]
master: namenode running as process 2735. Stop it first.
slave1: datanode running as process 2598. Stop it first.
slave2: starting datanode, logging to /opt/hadoop/logs/hadoop-hadoop-datanode-slave2.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: secondarynamenode running as process 2948. Stop it first.
resourcemanager running as process 3103. Stop it first.
slave1: nodemanager running as process 2701. Stop it first.
slave2: starting nodemanager, logging to /opt/hadoop/logs/yarn-hadoop-nodemanager-slave2.out


查看HDFS状态:

hadoop@master:/opt/hadoop$ bin/hdfs dfsadmin -report
Configured Capacity: 126670000128 (117.97 GB)
Present Capacity: 109199339520 (101.70 GB)
DFS Remaining: 109195915264 (101.70 GB)
DFS Used: 3424256 (3.27 MB)
DFS Used%: 0.00%
Under replicated blocks: 8
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Live datanodes (2):

Name: 192.168.0.14:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 72812462080 (67.81 GB)
DFS Used: 1712128 (1.63 MB)
Non DFS Used: 8574320640 (7.99 GB)
DFS Remaining: 64236429312 (59.82 GB)
DFS Used%: 0.00%
DFS Remaining%: 88.22%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sun Apr 05 13:28:45 CST 2015


Name: 192.168.0.12:50010 (slave2)
Hostname: slave2
Decommission Status : Normal
Configured Capacity: 53857538048 (50.16 GB)
DFS Used: 1712128 (1.63 MB)
Non DFS Used: 8896339968 (8.29 GB)
DFS Remaining: 44959485952 (41.87 GB)
DFS Used%: 0.00%
DFS Remaining%: 83.48%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sun Apr 05 13:28:45 CST 2015