安装hadoop-lzo压缩库

 

Ubuntu或debain可以使用以下命令进行安装即可。

 

sudo apt-get install liblzo2-dev lzop

 CentOS或RedHat需要从以下地址去下载：

liblzo2-2.06-alt1.i586.rpm

 

liblzo2-devel-2.06-alt1.i586.rpm

 

2、下载apache ant

3、 https://github.com/kevinweil/hadoop-lzo
     a, git clone https://github.com/kevinweil/hadoop-lzo 
     b, download *.tar.gz

4、修改 hadoop-lzo的build.xml
    搜索javah
    加上<classpath refid="classpath"/>
        <javah classpath="${build.classes}"
           destdir="${build.native}/src/com/hadoop/compression/lzo"
           force="yes"
           verbose="yes">
      <class name="com.hadoop.compression.lzo.LzoCompressor" />
      <class name="com.hadoop.compression.lzo.LzoDecompressor" />
      <classpath refid="classpath"/>
    </javah>

5、编译hadoop-lzo

export CFLAGS=-m32 && export CXXFLAGS=-m32 &&  ant compile-native tar

 

6、复制libgplcompression.so

 

 cp build/native/Linux-i386-32/lib/libgplcompression.so $HADOOP_HOME/lib/native/Linux-i386-32/

  
7、拷贝 build/hadoop-lzo-0.4.15.jar 到 $HADOOP_HOME/lib

cp build/hadoop-lzo-0.4.15.jar $HADOOP_HOME/lib

8、 修改core-site.xml

<property> 
	<name>io.compression.codecs</name> 
	<value>org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.LzopCodec</value>
</property>
<property> 
	<name>io.compression.codec.lzo.class</name> 
	<value>com.hadoop.compression.lzo.LzoCodec</value> 
</property>