CDH5.8.4-Hadoop2.6.0安常见报错及解决方法
1.cdh2.6.0安装完成时报错
启动hadoop集群时报错，如下:
20/01/23 22:27:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
原因：导致该问题的改版本是因为${HADOOP_PREFIX}/lib/native目录没有lib库
2.检查命令
[root@host150 bigdata]# hadoop checknative -a
20/01/23 22:42:31 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop: false
zlib: false
snappy: false
lz4: false
bzip2: false
openssl: false
20/01/23 22:42:31 INFO util.ExitUtil: Exiting with status 1
或者进入Hadoop下的/lib/native发现是空文件夹
3.解决办法
下载对应版本的native库，下载地址:
http://dl.bintray.com/sequenceiq/sequenceiq-bin/
下载完成之后解压到/lib/native目录:
[root@host152 native]# tar -xvf hadoop-native-64-2.6.0.tar
4.重启hdfs，再checknative查看
[root@host150 sbin]# hadoop checknative -a
20/01/23 23:16:35 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
20/01/23 23:16:35 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop: true /root/bigdata/hadoop/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: false
lz4: true revision:99
bzip2: false
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!
20/01/23 23:16:35 INFO util.ExitUtil: Exiting with status 1
5.还是报错libcrypto.so错
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!
解决:创建libcrypto.so软连接即可
[root@host150 sbin]# cd /usr/lib64/
[root@host150 lib64]# ln -s libcrypto.so.1.0.1e libcrypto.so
修改后再次checknative ，结果如下：
[root@host150 bigdata]# hadoop checknative -a
20/01/23 23:49:39 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
20/01/23 23:49:39 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop: true /root/bigdata/hadoop/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: false
lz4: true revision:99
bzip2: false
openssl: true /lib64/libcrypto.so
20/01/23 23:49:39 INFO util.ExitUtil: Exiting with status 1
说明
软链接：也称为符号链接,新建的文件以“路径”的形式来表示另一个文件，linux下的软链接类似于windows下的快捷方式。
举例:
ln -s httpd.conf confighttp
其中 httpd.conf 就是源文件，confighttp是链接文件名,其作用是当编辑confighttp，实际上是链接编辑httpd.conf。
如果要删除软链接：
rm -rf confighttp
这样只会删除confighttp链接文件,而不会删除源文件httpd.conf。
硬链接：新建的文件是已经存在的文件的一个别名,当原文件删除时，新建的文件仍然可以使用,硬链接文件相当于一个文件存储在两个位置，可以有效防止误删。
ln httpd.conf confighttp
6.yarn启动之后页面看不到默认队列资源，使用资源显示NAN%
查看后台yarn日志报错如下：
2020-01-24 11:19:22,540 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCount
WithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-01-24 11:19:23,541 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCount
WithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
解决办法:编辑yarn-site.xml，然后重启yarn集群即可。
[root@host152 hadoop]# vim yarn-site.xml