centos 系统
------------------------------------------------------------------
首先必须安装 rpmbuild
yum install rpm-build
----------------------------
接着运行编译命令，到datax源码的rpm目录下面
rpmbuild --ba t_dp_datax_engine.spec
------------报错提示缺少 dos2unix--------------
yum install dos2unix
------------接着继续执行编译命令出现一下错误-------------------------------------------------------------
Processing files: t_dp_datax_engine-1.0.0-1.noarch
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/bin
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/conf
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/engine
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/common
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/libs
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/logs
error: File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/jobs
RPM build errors:
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/bin
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/conf
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/engine
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/common
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/libs
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/logs
File not found: /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.i386/home/taobao/datax/jobs
-----------------------------------
这个问题 网上已经有人提出了解决方案：
http://yangyoupeng-cn-fujitsu-com.iteye.com/blog/1832143
请直接点击跳转即可
------------------------------------------------------------------------------------------
-----------------接下来继续执行 出现如下错误----------------
Checking for unpackaged file(s): /usr/lib/rpm/check-files /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.x86_64
error: Installed (but unpackaged) file(s) found:
/home/taobao/datax/jobs/HdfsReader_to_OracleWriter.xml
/home/taobao/datax/jobs/MysqlReader_to_HdfsWriter.xml
/home/taobao/datax/jobs/OracleReader_to_HdfsWriter.xml
/home/taobao/datax/jobs/OracleReader_to_MysqlWriter.xml
/home/taobao/datax/jobs/sample.xml
RPM build errors:
Installed (but unpackaged) file(s) found:
/home/taobao/datax/jobs/HdfsReader_to_OracleWriter.xml
/home/taobao/datax/jobs/MysqlReader_to_HdfsWriter.xml
/home/taobao/datax/jobs/OracleReader_to_HdfsWriter.xml
/home/taobao/datax/jobs/OracleReader_to_MysqlWriter.xml
/home/taobao/datax/jobs/sample.xml
------------------------
该问题的方案是：
进入 cd /usr/lib/rpm
vi macros
------------------------ ----------------------------------
#==============================================================================
# ---- Build configuration macros.
#
# Script gets packaged file list on input and buildroot as first parameter.
# Returns list of unpackaged files, i.e. files in $RPM_BUILD_ROOT not packaged.
#
# Note: Disable (by commenting out) for legacy compatibility.
%__check_files %{_rpmconfigdir}/check-files %{buildroot}
#
# Should unpackaged files in a build root terminate a build?
#
# Note: The default value should be 0 for legacy compatibility.
%_unpackaged_files_terminate_build 1
------------------------------把红色的这一行注释掉即可----------------------------------------
再运行编译打包命令
------------------------------------------------------------
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/brp-strip
+ /usr/lib/rpm/brp-strip-static-archive
+ /usr/lib/rpm/brp-strip-comment-note
Processing files: t_dp_datax_engine-1.0.0-1.noarch
Wrote: /root/rpmbuild/SRPMS/t_dp_datax_engine-1.0.0-1.src.rpm
Wrote: /root/rpmbuild/RPMS/noarch/t_dp_datax_engine-1.0.0-1.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.OYRna1
+ umask 022
+ cd /root/rpmbuild/BUILD
+ /bin/rm -rf /root/rpmbuild/BUILDROOT/t_dp_datax_engine-1.0.0-1.x86_64
+ exit 0
-----------------------------------------------------------------------------------------------------------
成功生成 rpm包。
-----------------------------------------------------------------------------------------------------------
安装生成的rpm包，执行以下命令
rpm -ivh t_dp_datax_engine-1.0.0-1.noarch.rpm
-----------------------出现一下日志表示已经成功安装-------------------------------------------------------
Preparing... ########################################### [100%]
1:t_dp_datax_engine-1.0.0-1.noarch ########################################### [100%]
-------------------------------------------------------------------------------------------------------------------------
进入 cd /home/taobao/datax/bin 目录
运行命令 ： ./datax.py -e true
----------------------------------------------出现空指针错误---------------------------------------------------------
[root@hadoop3 bin]# ./datax.py -e true
Taobao DataX V1.0
Exception in thread "main" java.lang.NullPointerException
at java.util.Arrays$ArrayList.
at java.util.Arrays.asList(Arrays.java:2828)
at com.taobao.datax.engine.tools.JobConfGenDriver.getPluginDirAsList(JobConfGenDriver.java:36)
at com.taobao.datax.engine.tools.JobConfGenDriver.filterPluginType(JobConfGenDriver.java:43)
at com.taobao.datax.engine.tools.JobConfGenDriver.produceXmlConf(JobConfGenDriver.java:166)
at com.taobao.datax.engine.schedule.Engine.main(Engine.java:389)
-----------------------------------------------------------------------------------------------------------------------------
出现该问题的原因是因为 datax安装好以后，还没有安装各种读写插件(mysql，oracle等)
所以我们现在需要安装读写插件
首先安装 mysqlreader插件，还是和本文开头遇见的第一个错误一样，需要修改 t_dp_datax_mysqlreader.spec 文件
-------------------------------------------红色字体就是修改后的--------------------------------------------------------
[root@hadoop3 rpm]# vi t_dp_datax_mysqlreader.spec
summary: DataX mysqlreader can read data from mysql
Name: t_dp_datax_mysqlreader
Version: 1.0.0
Release: 1
Group: System
License: GPL
AutoReqProv: no
BuildArch: noarch
Requires: t_dp_datax_engine
%define dataxpath %{buildroot}/home/taobao/datax
%define vdataxpath /home/taobao/datax (添加这一行)
%description
DataX mysqlreader can read data from mysql
%prep
cd ${OLDPWD}/../
export LANG=zh_CN.UTF-8
ant dist
%build
%install
mkdir -p %{dataxpath}/plugins/reader/mysqlreader
cp ${OLDPWD}/../src/com/taobao/datax/plugins/reader/mysqlreader/ParamKey.java %{dataxpath}/plugins/reader/mysqlreader
cp ${OLDPWD}/../build/plugins/mysqlreader-1.0.0.jar %{dataxpath}/plugins/reader/mysqlreader
cp ${OLDPWD}/../build/plugins/plugins-common-1.0.0.jar %{dataxpath}/plugins/reader/mysqlreader
cp -r ${OLDPWD}/../libs/mysql-connector-java-5.1.18-bin.jar %{dataxpath}/plugins/reader/mysqlreader
cp -r ${OLDPWD}/../libs/commons-dbcp-1.4.jar %{dataxpath}/plugins/reader/mysqlreader
cp -r ${OLDPWD}/../libs/commons-pool-1.5.4.jar %{dataxpath}/plugins/reader/mysqlreader
cp -r ${OLDPWD}/../libs/commons-logging-1.1.1.jar %{dataxpath}/plugins/reader/mysqlreader
%files
%defattr(0755,root,root)
%{vdataxpath}/plugins/reader/mysqlreader （替换后的）
%changelog
* Fri Aug 20 2010 meining
- Version 1.0.0
-------------------------------------------------------------------------------------------------------------------
接着再运行
rpmbuild --ba t_dp_datax_mysqlreader.spec
-----------------------------------------------------------------
Processing files: t_dp_datax_mysqlreader-1.0.0-1.noarch
Wrote: /root/rpmbuild/SRPMS/t_dp_datax_mysqlreader-1.0.0-1.src.rpm
Wrote: /root/rpmbuild/RPMS/noarch/t_dp_datax_mysqlreader-1.0.0-1.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.EhHiFf
+ umask 022
+ cd /root/rpmbuild/BUILD
+ /bin/rm -rf /root/rpmbuild/BUILDROOT/t_dp_datax_mysqlreader-1.0.0-1.x86_64
+ exit 0
-------------------------------成功生成rpm包---------------------------------------------
安装该插件
------------------------------------
[root@hadoop3 sa]# rpm -ivh t_dp_datax_mysqlreader-1.0.0-1.noarch.rpm
Preparing... ########################################### [100%]
1:t_dp_datax_mysqlreader ########################################### [100%]
------------------------------------------------------------------------------------------------------------------------
接着mysqlwriter也是按照同样的方法进行安装即可。
datax必须同时存在 一个reader插件和一个writer插件 才能正常运行 bin/datax.py -e true 命令，不然都会报错空指针的异常。
----------------------------------------------------------------------------------------------------------
[root@hadoop3 datax]# bin/datax.py -e true
Taobao DataX V1.0
Data Source List :
0 mysql
Please choose [0-0]: 0
Data Destination List :
0 mysql
Please choose [0-0]: 0
Generate /home/taobao/datax/jobs/mysqlreader_to_mysqlwriter_1426588439516.xml successfully .
------------------------------------------------成功生成job的xml配置文件----------------------------------------------------