【sqoop】sqoop1和sqoop2环境安装

• sqoop2

  版本：sqoop-1.99.6
  安装目录：/usr/local/sqoop
  参考博文
  https://blog.csdn.net/zhexcel/article/details/52192770
  https://blog.csdn.net/gamer_gyt/article/details/55225700

  • 安装步骤

    • 解压软件包

    tar -zxvf sqoop-1.99.6-bin-hadoop200.tar.gz -C /usr/local/sqoop

    • 配置

    
    # 配置hadoop的proxyuser
    
    
    # core-site.xml中的hadoop.proxyuser.$SERVER_USER.hosts和hadoop.proxyuser.$ SERVER_USER.groups中两个属性
    
    <property>
      <name>hadoop.proxyuser.sqoop2.hostsname>
      *
    property>
    <property>
      <name>hadoop.proxyuser.sqoop2.groupsname>
      *
    property>
    
    
    
    # 设置sqoop的环境变量
    
    
    # etc/profile
    
    export SQOOP_HOME=/usr/local/sqoop
    
    # sqoop使用的第三方组件的路径,此目录需手动创建，mysql的jdbc驱动的jar包需复制到此目录
    
    export SQOOP_SERVER_EXTRA_LIB=$SQOOP_HOME/extra
    export PATH=$PATH:$SQOOP_HOME/bin
    
    
    # 配置server/conf目录下的sqoop.properties
    
    
    # 路径为hadoop的配置文件目录
    
    org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/etc/hadoop 
    
    
    
    # 配置server/conf目录下的catalina.properties
    
    
    # 将其中引用jar的目录地址改成hadoop中的地址(刚开始没有配置此项，sqoop2-tool verify报错，配置后运行正常)
    
    common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop/share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar

    • 初始化（第一次启动时）

      
      # 脚本在$SQOOP_HOME/bin目录下
      
      sqoop2-tool upgrade

    • 验证配置

       # 脚本在$SQOOP_HOME/bin目录下
       sqoop2-tool verify

    • 启动服务器

       # 脚本在$SQOOP_HOME/bin目录下
       sqoop.sh server start
       sqoop2-server start

• sqoop1

  版本：sqoop-1.4.6
  安装目录：/usr/local/sqoop

  参考博文

  https://blog.csdn.net/dwld_3090271/article/details/50747639  
  https://www.cnblogs.com/mmzs/p/8149921.html

  • 安装步骤

    • 解压软件包

      tar -zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /usr/local/sqoop

    • 设置环境变量

      
      # etc/profile
      
      
      # sqoop路径
      
      export SQOOP_HOME=usr/local/sqoop
      export PATH=$PATH:$SQOOP_HOME

    • 将mysql的jdbc驱动包复制到$SQOOP_HOME/lib目录下

    • 启动hadoop的JobHistoryServer进程

      
      # 在执行导入时，会自动连接JobHistoryServer如果进程没有启动会显示错误信息。
      
      $HADOOP_HOME/sbin/mr-jobhistory-daemon.sh start historyserver