hadoop常见问题

1 Error:/bin/bash: /bin/java: No such file or directory

/bin/java不存在，则创建
sudo ln -s /opt/module/jdk1.8.0_162/bin/java /bin/java

running 219601408B beyond the 'VIRTUAL' memory limit.Container exited with a non-zero exit code 143

    
        yarn.nodemanager.vmem-check-enabled
        false
        Whether virtual memory limits will be enforced for containers
    

    
        yarn.nodemanager.vmem-pmem-ratio
        4
        Ratio between virtual memory to physical memory when setting memory limits for containers
    

最后重启虚拟机，重新执行实例就好了

2 使用sbin/start-dfs.sh，datanode不启动

清空datanode所在主机的tmp目录：

rm -fr data/tmp/dfs/data/current/
bin/hdfs namenode -format

3 集群时间同步

• 安装ntp

#安装
sudo yum install ntp
sudo yum install ntpdate
#启动
sudo service ntpd start
#设置开启启动
sudo chkconfig ntpd on

• 编辑 /etc/ntp.conf
  修改这几行

# Hosts on local network are less restricted.
restrict 192.168.176.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html). #server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst

添加两行

server 127.127.1.0      #local clock
fudqe 127.127.1.0 stratum 10

配置如下内容，保证BIOS与系统时间同步
编辑 /etc/sysconfig/ntpd
添加

SYNC_HWCLOCK=YES 

• 重启服务
  sudo service ntpd restart
  至此时间同步服务器设置完成
• 同步其他机器到时间服务器
  sudo ntpdate namenode.hay.com
• 创建定时任务
  在root用户下执行

crontab -e

输入

0-59/10 * * * * /usr/sbin/ntpdate namenode.hay.com