元数据与数据治理|apache-atlas2.0.0 源码编译和安装部署(CHD版本)

1.文档地址：

    官网地址：https://atlas.apache.org/

    文档地址：https://atlas.apache.org/2.0.0/index.html

    下载地址：https://www.apache.org/dyn/closer.cgi/atlas/2.0.0/apache-atlas-2.0.0-sources.tar.gz

 

 

2.Atlas源码编译打包

2.1 我的编译环境:

    • JDK:1.8.0_221

    • Maven: 3.6.1

    • macOS : 10.13.2

    说明：起初在Linux服务器编译各种问题，但是在本地环境编译较顺。

 

2.2 编译Atlas源码

tar xvfz apache-atlas-2.0.0-sources.tar.gz
cd apache-atlas-sources-2.0.0/
export MAVEN_OPTS="-Xms2g -Xmx2g"

mvn clean -DskipTests install    提示：执行过程比较长，会下载很多依赖，大约需要半个小时，期间如果报错很有可能是因为TimeOut造成的网络中断，重试即可。若编译成功，则会提示下面的内容

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Atlas Server Build Tools 1.0 ................ SUCCESS [ 31.976 s]
[INFO] apache-atlas 2.0.0 ................................. SUCCESS [01:11 min]
[INFO] Apache Atlas Test Utility Tools 2.0.0 .............. SUCCESS [01:29 min]
[INFO] Apache Atlas Integration 2.0.0 ..................... SUCCESS [01:10 min]
[INFO] Apache Atlas Common 2.0.0 .......................... SUCCESS [ 12.093 s]
[INFO] Apache Atlas Client 2.0.0 .......................... SUCCESS [  1.130 s]
[INFO] atlas-client-common 2.0.0 .......................... SUCCESS [  5.776 s]
[INFO] atlas-client-v1 2.0.0 .............................. SUCCESS [  5.696 s]
[INFO] Apache Atlas Server API 2.0.0 ...................... SUCCESS [  5.121 s]
[INFO] Apache Atlas Notification 2.0.0 .................... SUCCESS [ 20.069 s]
[INFO] atlas-client-v2 2.0.0 .............................. SUCCESS [  4.396 s]
[INFO] Apache Atlas Graph Database Projects 2.0.0 ......... SUCCESS [  0.352 s]
[INFO] Apache Atlas Graph Database API 2.0.0 .............. SUCCESS [ 10.243 s]
[INFO] Graph Database Common Code 2.0.0 ................... SUCCESS [  5.203 s]
[INFO] Apache Atlas JanusGraph-HBase2 Module 2.0.0 ........ SUCCESS [ 56.983 s]
[INFO] Apache Atlas JanusGraph DB Impl 2.0.0 .............. SUCCESS [01:58 min]
[INFO] Apache Atlas Graph Database Implementation Dependencies 2.0.0 SUCCESS [  1.709 s]
[INFO] Apache Atlas Authorization 2.0.0 ................... SUCCESS [  5.636 s]
[INFO] Apache Atlas Repository 2.0.0 ...................... SUCCESS [01:18 min]
[INFO] Apache Atlas UI 2.0.0 .............................. SUCCESS [02:12 min]
[INFO] Apache Atlas Web Application 2.0.0 ................. SUCCESS [03:48 min]
[INFO] Apache Atlas Documentation 2.0.0 ................... SUCCESS [ 35.325 s]
[INFO] Apache Atlas FileSystem Model 2.0.0 ................ SUCCESS [  2.271 s]
[INFO] Apache Atlas Plugin Classloader 2.0.0 .............. SUCCESS [  4.831 s]
[INFO] Apache Atlas Hive Bridge Shim 2.0.0 ................ SUCCESS [01:27 min]
[INFO] Apache Atlas Hive Bridge 2.0.0 ..................... SUCCESS [ 43.768 s]
[INFO] Apache Atlas Falcon Bridge Shim 2.0.0 .............. SUCCESS [ 45.987 s]
[INFO] Apache Atlas Falcon Bridge 2.0.0 ................... SUCCESS [  7.513 s]
[INFO] Apache Atlas Sqoop Bridge Shim 2.0.0 ............... SUCCESS [01:26 min]
[INFO] Apache Atlas Sqoop Bridge 2.0.0 .................... SUCCESS [ 21.666 s]
[INFO] Apache Atlas Storm Bridge Shim 2.0.0 ............... SUCCESS [ 22.885 s]
[INFO] Apache Atlas Storm Bridge 2.0.0 .................... SUCCESS [  6.413 s]
[INFO] Apache Atlas Hbase Bridge Shim 2.0.0 ............... SUCCESS [  5.225 s]
[INFO] Apache Atlas Hbase Bridge 2.0.0 .................... SUCCESS [01:05 min]
[INFO] Apache HBase - Testing Util 2.0.0 .................. SUCCESS [  5.144 s]
[INFO] Apache Atlas Kafka Bridge 2.0.0 .................... SUCCESS [ 21.402 s]
[INFO] Apache Atlas Distribution 2.0.0 .................... SUCCESS [  1.847 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  23:57 min
[INFO] Finished at: 2019-10-18T12:09:59+08:00
[INFO] ------------------------------------------------------------------------
MacBook-Pro:apache-atlas-sources-2.0.0 luomingkui$

 

 

2.4 打包

2.4.1 有多种选择，这种方法是比较简单的内嵌方式打包，单机部署，不适用于生产环境。

mvn clean -DskipTests package -Pdist,embedded-hbase-solr

    说明：在打包该文件的时候，产生如下问题：

Download HBase:

    [mkdir] Created dir: /Users/luomingkui/workspace/atlas_only/apache-atlas-sources-2.0.0/distro/target/hbase

    [mkdir] Created dir: /Users/luomingkui/workspace/atlas_only/apache-atlas-sources-2.0.0/distro/hbase

      [get] Getting: http://archive.apache.org/dist/hbase/2.0.2/hbase-2.0.2-bin.tar.gz

      [get] To: /Users/luomingkui/workspace/atlas_only/apache-atlas-sources-2.0.0/distro/hbase/hbase-2.0.2.tar.gz

2.4.2 也可以使用以下几种方式进行打包（使用外部的hbase和solr，我们所使用的）

mvn clean -DskipTests package -Pdist

 //编译完成显示如下表示打包完成

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Atlas Server Build Tools 1.0 ................ SUCCESS [  0.730 s]
[INFO] apache-atlas 2.0.0 ................................. SUCCESS [  3.434 s]
[INFO] Apache Atlas Test Utility Tools 2.0.0 .............. SUCCESS [  4.295 s]
[INFO] Apache Atlas Integration 2.0.0 ..................... SUCCESS [  4.525 s]
[INFO] Apache Atlas Common 2.0.0 .......................... SUCCESS [  1.739 s]
[INFO] Apache Atlas Client 2.0.0 .......................... SUCCESS [  0.206 s]
[INFO] atlas-client-common 2.0.0 .......................... SUCCESS [  0.784 s]
[INFO] atlas-client-v1 2.0.0 .............................. SUCCESS [  0.947 s]
[INFO] Apache Atlas Server API 2.0.0 ...................... SUCCESS [  1.346 s]
[INFO] Apache Atlas Notification 2.0.0 .................... SUCCESS [  2.283 s]
[INFO] atlas-client-v2 2.0.0 .............................. SUCCESS [  0.785 s]
[INFO] Apache Atlas Graph Database Projects 2.0.0 ......... SUCCESS [  0.166 s]
[INFO] Apache Atlas Graph Database API 2.0.0 .............. SUCCESS [  0.902 s]
[INFO] Graph Database Common Code 2.0.0 ................... SUCCESS [  0.925 s]
[INFO] Apache Atlas JanusGraph-HBase2 Module 2.0.0 ........ SUCCESS [  1.317 s]
[INFO] Apache Atlas JanusGraph DB Impl 2.0.0 .............. SUCCESS [  4.606 s]
[INFO] Apache Atlas Graph Database Implementation Dependencies 2.0.0 SUCCESS [  1.407 s]
[INFO] Apache Atlas Authorization 2.0.0 ................... SUCCESS [  1.533 s]
[INFO] Apache Atlas Repository 2.0.0 ...................... SUCCESS [  9.664 s]
[INFO] Apache Atlas UI 2.0.0 .............................. SUCCESS [ 57.481 s]
[INFO] Apache Atlas Web Application 2.0.0 ................. SUCCESS [ 37.800 s]
[INFO] Apache Atlas Documentation 2.0.0 ................... SUCCESS [  9.862 s]
[INFO] Apache Atlas FileSystem Model 2.0.0 ................ SUCCESS [  2.134 s]
[INFO] Apache Atlas Plugin Classloader 2.0.0 .............. SUCCESS [  0.814 s]
[INFO] Apache Atlas Hive Bridge Shim 2.0.0 ................ SUCCESS [  2.656 s]
[INFO] Apache Atlas Hive Bridge 2.0.0 ..................... SUCCESS [  8.271 s]
[INFO] Apache Atlas Falcon Bridge Shim 2.0.0 .............. SUCCESS [  0.897 s]
[INFO] Apache Atlas Falcon Bridge 2.0.0 ................... SUCCESS [  1.165 s]
[INFO] Apache Atlas Sqoop Bridge Shim 2.0.0 ............... SUCCESS [  0.127 s]
[INFO] Apache Atlas Sqoop Bridge 2.0.0 .................... SUCCESS [  6.439 s]
[INFO] Apache Atlas Storm Bridge Shim 2.0.0 ............... SUCCESS [  0.280 s]
[INFO] Apache Atlas Storm Bridge 2.0.0 .................... SUCCESS [  2.139 s]
[INFO] Apache Atlas Hbase Bridge Shim 2.0.0 ............... SUCCESS [  1.960 s]
[INFO] Apache Atlas Hbase Bridge 2.0.0 .................... SUCCESS [  5.678 s]
[INFO] Apache HBase - Testing Util 2.0.0 .................. SUCCESS [  4.384 s]
[INFO] Apache Atlas Kafka Bridge 2.0.0 .................... SUCCESS [  1.711 s]
[INFO] Apache Atlas Distribution 2.0.0 .................... SUCCESS [ 51.147 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:57 min
[INFO] Finished at: 2019-10-18T14:56:33+08:00
[INFO] ------------------------------------------------------------------------
MacBook-Pro:apache-atlas-sources-2.0.0 luomingkui$

// 打包之后的内容在如下位置

// 打包之后的内容在如下位置
MacBook-Pro:target luomingkui$ pwd
/Users/luomingkui/workspace/apache-atlas-sources-2.0.0/distro/target
MacBook-Pro:target luomingkui$ ls -l
total 1558888
drwxr-xr-x   5 luomingkui  staff        160 10 18 14:55 META-INF
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:56 apache-atlas-2.0.0-bin
-rw-r--r--   1 luomingkui  staff  376550017 10 18 14:56 apache-atlas-2.0.0-bin.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 apache-atlas-2.0.0-falcon-hook
-rw-r--r--   1 luomingkui  staff    9211807 10 18 14:55 apache-atlas-2.0.0-falcon-hook.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 apache-atlas-2.0.0-hbase-hook
-rw-r--r--   1 luomingkui  staff   11079516 10 18 14:55 apache-atlas-2.0.0-hbase-hook.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 apache-atlas-2.0.0-hive-hook
-rw-r--r--   1 luomingkui  staff   16267174 10 18 14:55 apache-atlas-2.0.0-hive-hook.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 apache-atlas-2.0.0-kafka-hook
-rw-r--r--   1 luomingkui  staff    9219776 10 18 14:55 apache-atlas-2.0.0-kafka-hook.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:56 apache-atlas-2.0.0-server
-rw-r--r--   1 luomingkui  staff  271886858 10 18 14:56 apache-atlas-2.0.0-server.tar.gz
-rw-r--r--   1 luomingkui  staff   11077883 10 18 14:56 apache-atlas-2.0.0-sources.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 apache-atlas-2.0.0-sqoop-hook
-rw-r--r--   1 luomingkui  staff    9200619 10 18 14:55 apache-atlas-2.0.0-sqoop-hook.tar.gz
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 apache-atlas-2.0.0-storm-hook
-rw-r--r--   1 luomingkui  staff   58914515 10 18 14:55 apache-atlas-2.0.0-storm-hook.tar.gz
drwxr-xr-x   2 luomingkui  staff         64 10 18 14:55 archive-tmp
-rw-r--r--   1 luomingkui  staff      97485 10 18 14:55 atlas-distro-2.0.0.jar
drwxr-xr-x  13 luomingkui  staff        416 10 18 14:55 bin
drwxr-xr-x  11 luomingkui  staff        352 10 18 14:55 conf
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 maven-archiver
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 maven-shared-archive-resources
-rw-r--r--   1 luomingkui  staff       4174 10 18 14:55 rat.txt
drwxr-xr-x   3 luomingkui  staff         96 10 18 14:55 test-classes
MacBook-Pro:target luomingkui$

 

 

3.Atlas 部署

tar -xzvf apache-atlas-2.0.0-server.tar.gz -C /opt/module/
cd apache-atlas-2.0.0

3.1 和Hbase 集成

// vim /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties
# 修改atlas存储数据主机
atlas.graph.storage.hostname=pdcbdptmp03p.glprop.com:2181,pdcbdptmp04p.glprop.com:2181,pdcbdptmp05p.glprop.com:2181
// 软连接
ln -s /etc/hbase/conf/ /opt/module/apache-atlas-2.0.0/conf/hbase/
# 添加HBase配置文件路径
vim /opt/module/apache-atlas-2.0.0/conf/atlas-env.sh
export HBASE_CONF_DIR=/opt/module/apache-atlas-2.0.0/conf/hbase/

3.2 和solr 集成  http://pdcbdptmp01p:8983/solr/#/

// vim /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties
#修改如下配置
atlas.graph.index.search.solr.zookeeper-url=pdcbdptmp03p.glprop.com:2181,pdcbdptmp04p.glprop.com:2181,pdcbdptmp05p.glprop.com:2181
 将Atlas自带的Solr文件夹拷贝到外部Solr集群的各个节点。
[luomk@hadoop102 conf]$ cp -r /opt/module/apache-atlas-2.0.0/conf/solr /opt/cloudera/parcels/CDH/lib/solr/


mv solr apache-atlas-conf 


scp -r /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf mkluo02@root@[email protected]:/opt/cloudera/parcels/CDH/lib/solr/


scp -r /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf mkluo02@root@[email protected]:/opt/cloudera/parcels/CDH/lib/solr/


scp -r /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf mkluo02@root@[email protected]:/opt/cloudera/parcels/CDH/lib/solr/


scp -r /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf mkluo02@root@[email protected]:/opt/cloudera/parcels/CDH/lib/solr/




// SOLR_HOME/bin/solr start -c -z  -p 8983
cd /opt/cloudera/parcels/CDH/lib/solr/
bin/solr start -c -force -zookeper localhost:2182 -p 8983




// cd /opt/cloudera/parcels/CDH/lib/solr


bin/solr create -c vertex_index -d /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf -force -shards 5 -replicationFactor 1


bin/solr create -c edge_index -d /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf  -force -shards 5 -replicationFactor 1


bin/solr create -c fulltext_index -d /opt/cloudera/parcels/CDH/lib/solr/apache-atlas-conf -force -shards 5 -replicationFactor 1

创建成功如下：http://pdcbdptmp02p:8983/solr/#/~cloud

3.3 和kafka 集成

vim /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties


#########  Notification Configs  #########
atlas.notification.embedded=false
atlas.kafka.zookeeper.connect=pdcbdptmp03p.glprop.com:2181,pdcbdptmp04p.glprop.com:2181,pdcbdptmp05p.glprop.com:2181
atlas.kafka.bootstrap.servers=pdcbdptmp03p.glprop.com:9092,pdcbdptmp04p.glprop.com:9092,pdcbdptmp05p.glprop.com:9092
atlas.kafka.zookeeper.session.timeout.ms=4000
atlas.kafka.zookeeper.connection.timeout.ms=2000


atlas.kafka.enable.auto.commit=true




//kafka主题 
--查看当前服务器中的所有topic（查看主题）
cd /opt/cloudera/parcels/CDH/lib/kafka


bin/kafka-topics.sh --zookeeper pdcbdptmp03p:2181 --list


—创建topic（创建主题）
bin/kafka-topics.sh --zookeeper pdcbdptmp03p:2181,pdcbdptmp04p:2181,pdcbdptmp05p:2181 --create --replication-factor 3 --partitions 3 --topic _HOATLASOK


bin/kafka-topics.sh --zookeeper pdcbdptmp03p:2181,pdcbdptmp04p:2181,pdcbdptmp05p:2181 --create --replication-factor 3 --partitions 3 --topic ATLAS_ENTITIES


—删除topic（删除主题）
bin/kafka-topics.sh --zookeeper pdcbdptmp03p:2181,pdcbdptmp04p:2181,pdcbdptmp05p:2181 -delete --topic _HOATLASOK
bin/kafka-topics.sh --zookeeper pdcbdptmp03p:2181,pdcbdptmp04p:2181,pdcbdptmp05p:2181 -delete --topic ATLAS_ENTITIES


说明：需要server.properties中设置delete.topic.enable=true否则只是标记删除或者直接重启。

3.4 Atlas其他基本配置    进入/opt/module/atlas/conf/目录，修改配置文件atlas-application.properties

[luomk@hadoop102 conf]$ vim atlas-application.properties 
#########  Server Properties  #########
atlas.rest.address=http://pdcbdptmp01p:21000
# If enabled and set to true, this will run setup steps when the server starts
atlas.server.run.setup.on.start=false
#########  Entity Audit Configs  #########
atlas.audit.hbase.zookeeper.quorum=pdcbdptmp03p.glprop.com:2181,pdcbdptmp04p.glprop.com:2181,pdcbdptmp05p.glprop.com:2181

    到这里，apache atlas 独立部署编译，solr集群部署，集成到apache atlas，集成已有HBase集群，集成已有kafka集群

    可启动apache atlas 查看是否配置正确，若启动不成功，可在目录：$ATALS_HOME/logs 查看日志。

 

 

 

4.Atlas和hive的元数据集成

4.1 进入/opt/module/atlas/conf/目录，修改配置文件atlas-application.properties

[luomk@hadoop102 conf]$ vim atlas-application.properties


######### Hive Hook Configs #######
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.queueSize=10000
atlas.cluster.name=primary

4.2 将atlas-application.properties配置文件加入到atlas-plugin-classloader-1.0.0.jar中

[root@pdcbdptmp01p hive]# cp /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties /opt/module/apache-atlas-2.0.0/apache-atlas-hive-hook-2.0.0/hook/hive/


[root@pdcbdptmp01p hive]# zip -u /opt/module/apache-atlas-2.0.0/apache-atlas-hive-hook-2.0.0/hook/hive/atlas-plugin-classloader-2.0.0.jar atlas-application.properties


[root@pdcbdptmp01p conf]# cp /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties /etc/hive/conf
或者
[root@pdcbdptmp01p conf]# ln -s /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties /etc/hive/conf

4.3 在/opt/module/hive/conf/hive-site.xml文件中设置Atlas hook

[luomk@hadoop102 conf]$ vim hive-site.xml

      hive.exec.post.hooks
      org.apache.atlas.hive.hook.HiveHook





[luomk@hadoop102 conf]$ vim hive-env.sh
export HIVE_AUX_JARS_PATH=/opt/module/apache-atlas-2.0.0/apache-atlas-hive-hook-2.0.0/hook/hive

在CM中配置具体如下：

 

 

 

 

 

 

 

4.4 启动 import-hive.sh脚本

export HIVE_HOME="/opt/cloudera/parcels/CDH/lib/hive”


[root@pdcbdptmp01p hook-bin]# /opt/module/apache-atlas-2.0.0/apache-atlas-hive-hook-2.0.0/hook-bin/import-hive.sh

4.5 导入结果如下则导入成功：

[root@pdcbdptmp01p hook-bin]# ./import-hive.sh
Using Hive configuration directory [/opt/cloudera/parcels/CDH/lib/hive/conf]
/opt/cloudera/parcels/CDH/lib/hive/conf:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//*:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop-yarn/lib/*:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/lib/hadoop/libexec/../../hadoop-yarn/.//*
Log file for import is /opt/module/apache-atlas-2.0.0/apache-atlas-hive-hook-2.0.0/logs/import-hive.log
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.1-1.cdh6.0.1.p0.590678/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 internal initialization logging.
Enter username for atlas :- admin
Enter password for atlas :-




Hive Meta Data imported successfully!!!

4.6 刷新页面后可见：http://pdcbdptmp01p.glprop.com:21000

 

 

 

5 Atlas和sqoop的元数据集成5.1 进入/opt/module/atlas/conf/目录，修改配置文件atlas-application.properties

[luomk@hadoop102 conf]$ vim atlas-application.properties


######### sqoop Hook Configs #######
atlas.hook.sqoop.synchronous=false
atlas.hook.sqoop.numRetries=3
atlas.hook.sqoop.queueSize=10000
atlas.cluster.name=primary

5.2 将atlas-application.properties配置文件加入到atlas-plugin-classloader-1.0.0.jar中

[root@pdcbdptmp01p sqoop]# cp /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties /opt/module/apache-atlas-2.0.0/apache-atlas-sqoop-hook-2.0.0/hook/sqoop/




[root@pdcbdptmp01p sqoop]# zip -u /opt/module/apache-atlas-2.0.0/apache-atlas-sqoop-hook-2.0.0/hook/sqoop/atlas-plugin-classloader-2.0.0.jar atlas-application.properties




[root@pdcbdptmp01p conf]# cp /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties /etc/sqoop/conf
或者
[root@pdcbdptmp01p conf]# ln -s /opt/module/apache-atlas-2.0.0/conf/atlas-application.properties /etc/sqoop/conf

5.3  修改 sqoop-site.xml 文件

     sqoop.job.data.publish.class
     org.apache.atlas.sqoop.hook.SqoopHook





[root@pdcbdptmp01p sqoop]# cp /opt/module/apache-atlas-2.0.0/apache-atlas-sqoop-hook-2.0.0/hook/sqoop/*.jar /opt/cloudera/parcels/CDH/lib/sqoop/lib


[root@pdcbdptmp01p sqoop]# cp /opt/module/apache-atlas-2.0.0/apache-atlas-sqoop-hook-2.0.0/hook/sqoop/atlas-sqoop-plugin-impl/*.jar /opt/cloudera/parcels/CDH/lib/sqoop/lib/

在CM中：