sqoop1将mysql数据导入数据库

环境：
hbase 1.1.9集群
zookeeper-3.4.9 集群
hadoop 2.7.1集群
注：sqoop2不支持直接将mysql数据导入hbase,所以选择sqoop1

一、sqoop1安装

1 解压安装

sudo tar xvfz  sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

 sudo mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha home/zhonglan/sqoop1/
 sudo chmod -R 775 /home/zhonglan/sqoop1/
 sudo chown -R zhonglan /home/zhonglan/sqoop1/

2 修改环境变量

 sudo vim /etc/profile
#sqoop
export SQOOP_HOME=/ home/zhonglan/sqoop1/
export PATH=$SQOOP_HOME/bin:$PATH

source /etc/profile

3 配置sqoop的环境变量

下载mysql驱动包，mysql-connector-java-5.1.27.jar

把jar包丢到到$SQOOP_HOME/lib下面

 cp sqoop-env-template.sh sqoop-env.sh

# 指定各环境变量的实际配置

#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/home/zhonglan/hadoop

#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/home/zhonglan/hadoop

#set the path to where bin/hbase is available
export HBASE_HOME=/home/zhonglan/hbase

#Set the path to where bin/hive is available
#export HIVE_HOME=

#Set the path for where zookeper config dir is
export ZOOCFGDIR=/home/zhonglan/zookeeper/zookeeperdata

4 测试
使用 sqoop version 查看

不用管warning部分，可在bin/configure-sqoop中注释掉HCAT_HOME 、ACCUMULO_HOME

二、数据导入

1.启动环境

1)启动hadoop集群

start-dfs.sh 
start-yarn.sh

2)分别启动zk(注意每个节点的zk都要启动)

zkServer.sh start

3)启动hbase，在主节点上运行：

start-hbase.sh

jps

2.准备数据库

以mysql为例子
IP:222.31.81.214
用户名:root
密码:****
数据库:big_data_service
表:song_topn

3.sqoop导入

sqoop help
Available commands:
  codegen            Generate code to interact with database records
  create-hive-table  Import a table definition into Hive
  eval               Evaluate a SQL statement and display the results
  export             Export an HDFS directory to a database table
  help               List available commands
  import             Import a table from a database to HDFS
  import-all-tables  Import tables from a database to HDFS
  import-mainframe   Import datasets from a mainframe server to HDFS
  job                Work with saved jobs
  list-databases     List available databases on a server
  list-tables        List available tables in a database
  merge              Merge results of incremental imports
  metastore          Run a standalone Sqoop metastore
  version            Display version information

查看mysql上的表

sqoop list-tables --connect jdbc:mysql://222.31.81.214/big_data_service --username root -P

在hbase上创建表以及列簇

create 'song_test','song' 

导入对于的mysql数据，id为row_key

sqoop import --connect jdbc:mysql://222.31.81.214/big_data_service --table song_topn  --hbase-table song_test --column-family song --hbase-row-key id --username root -P

启动mapreduce任务

hbase上查看结果

scan 'song_test'

参考网址：
https://chu888chu888.gitbooks.io/hadoopstudy/content/Content/9/chapter0902.html