Hadoop的Snappy安装配置

1.snappy编译安装

下载snappy,编译。
前置条件：gcc c++, autoconf, automake, libtool, Java 6, JAVA_HOME set, Maven 3
1）运行：autogen.sh
2）运行：
./configure
make
make check
make install
3）验证：查看/usr/local/lib下面是否有libsnappy.*

4)拷贝到其他集群：scp -r ./google-snappy-2d99bd1/ bigdata@xxx:/ssd/bigdata/install/

2. Hadoop Snappy 编译安装

1、下载
2、The built tarball is at target/hadoop-snappy-0.0.1-SNAPSHOT.tar.gz. The tarball includes snappy native library。
解压拷贝：
cp -r * $HADOOP_HOME/lib
scp -r * bigdata@xxx:/ssd/bigdata/software/hadoop/lib
3、配置环境变量：
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/
export
HBASE_LIBRARY_PATH=$HBASE_LIBRARY_PATH:$HBASE_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/

2°、修改$HADOOP_HOME/etc/hadoop/core-site.xml：

io.compression.codecs

org.apache.hadoop.io.compress.GzipCodec,
org.apache.hadoop.io.compress.DefaultCodec,
org.apache.hadoop.io.compress.BZip2Codec,
org.apache.hadoop.io.compress.SnappyCodec


3°、修改$HADOOP_HOME/etc/hadoop/mapred-site.xml中有关压缩属性，测试snappy：

mapreduce.map.output.compress
true


mapreduce.map.output.compress.codec
org.apache.hadoop.io.compress.SnappyCodec

4°、验证
hadoop checknative -a

5、最终编译了hadoop2.7.3的源码：
https://www.58jb.com/html/113.html 来支持了snappy。
编译进行一半，只要把snappy编译出来即可。
参考：
http://www.cnblogs.com/zlzhoulei/p/5735698.html
http://blog.csdn.net/bbaiggey/article/details/53583923
http://www.micmiu.com/bigdata/hadoop/hadoop-snappy-install-config/