解决unable to load native-library问题
系统配置:win7 64位机上安装的VMWARE10
Ubuntu14.10
Hadoop2.7.0
从hadoop官网下载了最新的hadoop-2.7.0-src.tar.gz,经过漫长的等待,终于编译成功,然后按照网上的教程(http://www.cnblogs.com/kinglau/p/3794433.html),安装单机模式,配置好之后,通过执行hadoop自带实例WordCount验证是否安装成功。结果发现报unable to load native-hadoop library for your platform … 如下图所示:
在网上查了很多资料,有的让更改lib/native里的链接、有的说是hadoop2以后版本自带的本地库是32位版本的,让从网上下载64位的,然后覆盖lib/native里的所有文件、有的说让重新编译,我很认真的实现了所有网上的有可能的做法,结果很沮丧的发现还是unable to …
最后一不小心就到hadoop官网(http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html)里去找答案,发现可以首先用NativeLibraryChecker 工具来检查native libraries是否被正确的下载。原文如下:
NativeLibraryChecker is a tool to check whether nativelibraries are loaded correctly. You can launch NativeLibraryChecker as follows:
我用sbin/start-all.sh打开hadoop,然后用hadoopchecknative –a 指令检查,发现全是false,如下所示:
继续看hadoop官方资料,发现原文里有关于hadoopRuntime的描述,原文如下:
发现最后的If something goeswrong…正是我现在的问题,再看一下前边的描述,再仔细看,发现需要更改-Djava.library.path=
exportJAVA_HOME=/home/brian/hadoop/jdk
exportPATH=$JAVA_HOME/bin:$PATH
exportM2_HOME=/home/brian/hadoop/maven
PATH=$M2_HOME/bin:$PATH
exportLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
exportANT_HOME=/home/brian/hadoop/ant
exportPATH=$ANT_HOME/bin:$PATH
exportFINDBUGS_HOME=/home/brian/hadoop/findbugs
exportPATH=$FINDBUGS_HOME/bin:$PATH
#HADOOP VARIABLESSTART
exportHADOOP_INSTALL=/home/brian/hadoop/hadoop
exportPATH=$PATH:$HADOOP_INSTALL/bin
exportPATH=$PATH:$HADOOP_INSTALL/sbin
exportHADOOP_MAPRED_HOME=$HADOOP_INSTALL
exportHADOOP_COMMON_HOME=$HADOOP_INSTALL
exportHADOOP_HDFS_HOME=$HADOOP_INSTALL
exportYARN_HOME=$HADOOP_INSTALL
exportHADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
#HADOOPVARIABLES END
于是突然心血来潮,给lib后边加了native,更给完后如下所示:
exportJAVA_HOME=/home/brian/hadoop/jdk
exportPATH=$JAVA_HOME/bin:$PATH
exportM2_HOME=/home/brian/hadoop/maven
PATH=$M2_HOME/bin:$PATH
exportLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
exportANT_HOME=/home/brian/hadoop/ant
exportPATH=$ANT_HOME/bin:$PATH
export FINDBUGS_HOME=/home/brian/hadoop/findbugs
exportPATH=$FINDBUGS_HOME/bin:$PATH
#HADOOPVARIABLES START
exportHADOOP_INSTALL=/home/brian/hadoop/hadoop
exportPATH=$PATH:$HADOOP_INSTALL/bin
exportPATH=$PATH:$HADOOP_INSTALL/sbin
exportHADOOP_MAPRED_HOME=$HADOOP_INSTALL
exportHADOOP_COMMON_HOME=$HADOOP_INSTALL
exportHADOOP_HDFS_HOME=$HADOOP_INSTALL
exportYARN_HOME=$HADOOP_INSTALL
exportHADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
exportHADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib/native"
#HADOOPVARIABLES END
本来没有报任何希望(因为native本来就包括在lib文件夹里),只是机械的再次check了一下,结果奇迹出现了:
然后重新hadoop自带实例WordCount,结果看到了讨厌的WARN一去不返!!!