源码编译-Build Source

1.环境准备

想要更进一步了解和使用Trafodion,难免不得不自己尝试一番编译源码，此为之前做的一个小任务，踩过一些坑，现分享出来并做个总结。[参考](https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source)

1.1 虚拟机

源码编译准备搭建自己的一个虚拟机，系统CentOS6.X即可，7版本会编译错误不支持。
配置参考：
硬盘：30G 最好不要默认，否则多次编译会使磁盘空间不足
内存：2G
网卡：NAT

Tips:避免出错需要重新新建虚拟机，最好每做一步做好快照，方便恢复。

查看系统版本：
cat /etc/redhat-release
CentOS release 6.7 (Final)

1.2 安装编译需要的包

# Install first to avoid dependency problems for the steps below
sudo yum install epel-release

sudo yum install alsa-lib-devel ant ant-nodeps apr-devel apr-util-devel \
         boost-devel cmake device-mapper-multipath dhcp doxygen flex \
         gcc-c++ gd git glibc-devel glibc-devel.i686 graphviz-perl gzip \
         java-1.7.0-openjdk-devel libX11-devel libXau-devel libaio-devel \
         libcurl-devel libibcm.i686 libibumad-devel libibumad-devel.i686 \
         libiodbc libiodbc-devel librdmacm-devel librdmacm-devel.i686 \
         libxml2-devel lua-devel lzo-minilzo \
         net-snmp-devel net-snmp-perl openldap-clients openldap-devel \
         openldap-devel.i686 openmotif openssl-devel openssl-devel.i686 \
         perl-Config-IniFiles perl-Config-Tiny \
         perl-DBD-SQLite perl-Expect perl-IO-Tty perl-Math-Calc-Units \
         perl-Params-Validate perl-Parse-RecDescent perl-TermReadKey \
         perl-Time-HiRes protobuf-compiler protobuf-devel \
         readline-devel rpm-build saslwrapper sqlite-devel \
         unixODBC unixODBC-devel uuid-perl wget xerces-c-devel xinetd

Tips: 避免因为包安装失败造成编译错误，请确保每一个包均安装成功

1.3 配置

配置java的环境变量，通过java -version 验证成功

#新建编译目录trafodion
mkdir trafodion
cd trafodion
#下载源码
git clone git://git.apache.org/incubator-trafodion.git
# 新建trafodion 编译目录
mkdir ~/trafodion-download ~/trafodion-build-tools

cd ~/mysource/incubator-trafodion/install
# 利用脚本安装
./traf_tools_setup.sh -d ~/trafodion-download -i ~/trafodion-build-tools
# 安装成功信息基本如下
INFO: Starting tools build on Mon Jan 25 12:30:41 PST 2016
Tools install directory /home/trafdeveloper/trafodion-build-tools does not exist, do you want to to create it? y/n : y
INFO: Created directory /home/trafdeveloper/trafodion-build-tools
INFO: Tar download location: /home/trafdeveloper/download
INFO: Tool install directory location: /home/trafdeveloper/trafodion-build-tools
INFO: LogFile location: /home/trafdeveloper/mysource/incubator-trafodion/install/traf_tools_setup.log
 ***********************************************************

INFO: Installing MPI on Mon Jan 25 12:30:45 PST 2016
INFO:   downloaded tar file: mpich-3.0.4.tar.gz
.
.
.
INFO:   downloaded tar file:  apache-maven-3.3.3-bin.tar.gz
INFO: Maven installation complete
***********************************************************
INFO: Completed tools build on Mon Jan 25 13:18:31 PST 2016
INFO: List of tools directory:

apache-log4cxx-0.10.0
apache-maven-3.3.3
bison_3_linux
dest-llvm-3.2
dest-mpich-3.0.4
icu4c_4.4
thrift-0.9.0
udis86-1.7.2
zookeeper-3.4.5

Tips : 此过程需要从网络上下载包并解压安装，由于网络限制，速度较慢，一般在1-1.5h

安装完成导入环境变量，后续需要使用，否则因为找不到变量而发生错误
export TOOLSDIR=$HOME/trafodion-build-tools
echo -en "export TOOLSDIR=$HOME/trafodion-build-tools\n" >> $HOME/.bashrc

本步骤结束，请验证
$JAVA_HOME
$TOOLSDIR
是否有值

2.编译

源码编译一般来说有debug和release两种方式，但是debug多用于开发百衲衣测试，在正式的环境中编译请推荐使用release方式
Tips：debug方式我没有编译成功过，具体问题需要待研究

# 在你的trafodion 目录
d source directory>/core/sqf
source ./sqenvr.sh
cd ../..
执行 make all
或者make package-all(后续就不需要安装local_hadoop)
# 需要等待2-3h 没有任何编译错误，出现success 就算成功
[INFO] Building jar: /home/trafdeveloper/mysource/incubator-trafodion/dcs/target/dcs-1.3.0.jar  ##(DCS)
[INFO] Building tar: /home/trafdeveloper/mysource/incubator-trafodion/dcs/target/dcs-1.3.0.tar.gz  ##(DCS)
[INFO] BUILD SUCCESS  ##(DCS)
mkdir -p ../distribution  ##(DCS)
mv target/dcs-1.3.0.tar.gz ../distribution  ##(DCS)
rm -f trafodion-dcs.jar.versions  ##(DCS)
make[2]: Leaving directory `/home/trafdeveloper/mysource/incubator-trafodion/dcs'  ##(DCS)
make[1]: Leaving directory `/home/trafdeveloper/mysource/incubator-trafodion/core'

Tips:这部分完成需要验证编译情况
sqvers -u

sqcers -u
TRAF_HOME=/home/trafdeveloper/mysource/incubator-trafodion/core/sqf
who@host=trafdeveloper@box01
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64
SQ_MBTYPE=64d (64-debug)
linux=2.6.32-504.el6.x86_64
redhat=6.6
NO patches
Most common Apache_Trafodion Release 1.3.0 (Build debug [trafdeveloper], branch -, date 25Jan16)
UTT count is 1
[6]     Release 1.3.0 (Build debug [trafdeveloper], branch -, date 25Jan16)
          export/lib/hbase-trx-cdh5_3-1.3.0.jar
          export/lib/hbase-trx-hbase_98_4-1.3.0.jar
          export/lib/hbase-trx-hdp2_2-1.3.0.jar
          export/lib/sqmanvers.jar
          export/lib/trafodion-dtm-1.3.0.jar
          export/lib/trafodion-sql-1.3.0.jar

3.搭建测试环境

编译完成，我们需要配置SSH免登录和测试一下环境服务

3.1 ssh 免登录

ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys;

3.2 修改默认配置

edit /etc/security/limits.conf and reboot the server

# Changes added for the Trafodion Test Environment
*                -       core            1000000
*                -       nproc           267263
*                -       nofile          32000
*                -       sigpending      515196
*                -       memlock         49595556
*                -       stack           10240

3.3 安装local_hadoop

如果前面编译make package-all，这步可不需要

# 选择编译源
d /core/sqf
source ./sqenvr.sh
cd ../..
# 进入脚本目录
cds
install_local_hadoop
# 大约2.5-1h
Checking for existing Hadoop processes...
The testware tpcds_kit.zip does not exist and will not be installed
This testware is needed to run developer HIVE regression tests
.
.
.
Installed directory size and name = 2.7G        /home/trafdeveloper/mysource/incubator-trafodion/core/sqf/sql/local_hadoop

Setup is complete. You can use the convenience scripts starting with sw... located in /home/trafdeveloper/mysource/incubator-trafodion/core/sqf/sql/scripts.
$ ./install_traf_components

Installing and configuring DCS, REST, TRAFCI & Phoenix tests for Trafodion...

Environment used for core, DCS, REST and Phonenix ...
.
.
.
Configuration scripts for DCS, REST, TRAFCI and Phoenix test are set up

   Open a new session and start Trafodion by executing sqgen and sqstart scripts
   # 编译成功执行
./install_traf_components

3.4 测试服务

测试服务状态

swstatus
# 无报错，打印两行字大致是所有服务以启动即可
# 一般HBase 没有正常启动
执行
swstarthbase 启动HMaster
# 
sqgen
# 重新打开终端，引入源，执行
sqstart
所有服务启动正常，没有down的服务即大功告成

4.总结

1.以上Tips都是常遇到的错误，请注意
2.Trafodion 基于hadoop的基础

最后欢迎一起讨论学习Trafodion，支持企业版Esgyn