I setup a hadoop cluster with two nodes hadoop01(master- 10.0.0.151) and hadoop02(slaves- 10.0.0.152) when a type start-dfs.sh and then visit website my_ip(just 10.0.0.151 above):50070. It's successful.
But when i type start-yarn.sh then visit website my_ip:8088. It's failed.
my yarn-site.xml:
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.resourcemanager.address
hadoop01:8032
yarn.resourcemanager.scheduler.address
hadoop01:8030
yarn.resourcemanager.resource-tracker.address
hadoop01:8031
yarn.resourcemanager.admin.address
hadoop01:8033
yarn.resourcemanager.webapp.address
hadoop01:8088
core-site.xml:
fs.defaultFS
hdfs://hadoop01:8020
the same set as hadoop02(slave)
my hadoop01 hadoop-2.2.0/etc/hadoop/slaves set
hadoop01
hadoop02
After typing start-dfs.sh & start-yarn.sh then i type jps
hadoop01:
21594 NameNode
22345 NodeManager
22007 SecondaryNameNode
22171 ResourceManager
23147 Jps
21762 DataNode
hadoop02:
29861 NodeManager
30358 Jps
29665 DataNode
my /etc/hosts in hadoop01:
localhost hadoop01
10.0.0.151 hadoop01
10.0.0.152 hadoop02
my /etc/hosts in hadoop02:
localhost hadoop02
10.0.0.151 hadoop01
10.0.0.152 hadoop02
This bellow link is my yarn-nodemanager.log I upload to google dirvehttps://drive.google.com/file/d/0B7nCJ_XJWSrQN1BZVTVyOEgxd1E/edit?usp=sharingbut yarn-nodemanager.log doesn't appear ERROR if i didn't miss some information..
Please help me improve the problem why I can't visit the websitehttp://10.0.0.151:8088
**if need other information (such as hdfs-site.xml...etc) just tell me. I'll update..
netstat -tunalp | grep LISTEN
tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN 17442/java
tcp 0 0 0.0.0.0:50075 0.0.0.0:* LISTEN 17442/java
tcp 0 0 0.0.0.0:50020 0.0.0.0:* LISTEN 17442/java
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 17693/java
tcp 0 0 10.0.0.151:8020 0.0.0.0:* LISTEN 17267/java
tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN 17267/java
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp6 0 0 :::13562 :::* LISTEN 21061/java
tcp6 0 0 10.0.0.151:8030 :::* LISTEN 20881/java
tcp6 0 0 10.0.0.151:8031 :::* LISTEN 20881/java
tcp6 0 0 10.0.0.151:8032 :::* LISTEN 20881/java
tcp6 0 0 10.0.0.151:8033 :::* LISTEN 20881/java
tcp6 0 0 :::33762 :::* LISTEN 21061/java
tcp6 0 0 :::8040 :::* LISTEN 21061/java
tcp6 0 0 :::8042 :::* LISTEN 21061/java
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 10.0.0.151:8088 :::* LISTEN 20881/java
After disable ipv6 , then i type netstat -tunalp | grep LISTEN
tcp 0 0 0.0.0.0:13562 0.0.0.0:* LISTEN 30608/java
tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN 29967/java
tcp 0 0 0.0.0.0:50075 0.0.0.0:* LISTEN 29967/java
tcp 0 0 10.0.0.151:8030 0.0.0.0:* LISTEN 30424/java
tcp 0 0 10.0.0.151:8031 0.0.0.0:* LISTEN 30424/java
tcp 0 0 0.0.0.0:52992 0.0.0.0:* LISTEN 30608/java
tcp 0 0 10.0.0.151:8032 0.0.0.0:* LISTEN 30424/java
tcp 0 0 10.0.0.151:8033 0.0.0.0:* LISTEN 30424/java
tcp 0 0 0.0.0.0:50020 0.0.0.0:* LISTEN 29967/java
tcp 0 0 0.0.0.0:8040 0.0.0.0:* LISTEN 30608/java
tcp 0 0 0.0.0.0:8042 0.0.0.0:* LISTEN 30608/java
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 30222/java
tcp 0 0 10.0.0.151:8020 0.0.0.0:* LISTEN 29790/java
tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN 29790/java
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 10.0.0.151:8088 0.0.0.0:* LISTEN 30424/java
tcp6 0 0 :::22 :::* LISTEN -
SOLVED:
In hdfs-site.xml, setthe value ofdfs.namenode.rpc-bind-hostto0.0.0.0and Hadoop will listen on both the private and public network interfaces allowing remote access and datanode access.
This is related tohttps://issues.apache.org/jira/browse/HADOOP-605; Hadoop configuration scripts have added the-Djava.net.preferIPv4Stack=trueflag to force IPv4 binding, but this is missing from the Yarn configuration script. You can fix this by adding at the end of bin/yarn (before the exec):
YARN_OPTS="$YARN_OPTS -Djava.net.preferIPv4Stack=true"
the reason is you should change 10.0.0.151:8088 to 0.0.0.0:8088. the method is open the yarn-site.xml, add: yarn.resourcemanager.webapp.address 0.0.0.0:8088