Hive中metastore（元数据存储）的三种方式：
- 内嵌Derby方式
- Local方式
- Remote方式
[一]、内嵌Derby方式
这个是Hive默认的启动模式，一般用于单元测试，这种存储方式有一个缺点：在同一时间只能有一个进程连接使用数据库。
hive-site.xml 中jdbc URL、驱动、用户名、密码等的配置信息如下：
javax.jdo.option.ConnectionURL jdbc:derby:;databaseName=metastore_db;create=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName org.apache.derby.jdbc.EmbeddedDriver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName APP username to use against metastore database javax.jdo.option.ConnectionPassword mine password to use against metastore database hive.metastore.warehouse.dir file:///Users/micmiu/tmp/hive/warehouse unit test data goes in here on your local filesystem
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
javax.jdo.option.ConnectionURL
jdbc:derby:;databaseName=metastore_db;create=true
JDBC connect string for a JDBC metastore
javax.jdo.option.ConnectionDriverName
org.apache.derby.jdbc.EmbeddedDriver
Driver class name for a JDBC metastore
javax.jdo.option.ConnectionUserName
APP
username to use against metastore database
javax.jdo.option.ConnectionPassword
mine
password to use against metastore database
hive.metastore.warehouse.dir
file:///Users/micmiu/tmp/hive/warehouse
unit test data goes in here on your local filesystem
|
执行初始化命令：schematool -dbType derby -initSchema
查看初始化后的信息： schematool -dbType derby -info
配置完成后就可在shell中以CLI的方式访问hive 进行操作验证。
[二]、Local方式
以本地Mysql数据库为例：创建好用户：hive；database：hive。
配置文件 hive-site.xml 中jdbc URL、驱动、用户名、密码等属性值配置如下：
javax.jdo.option.ConnectionURL jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName hive username to use against metastore database javax.jdo.option.ConnectionPassword micmiu password to use against metastore database hive.metastore.warehouse.dir /user/hive/warehouse location of default database for the warehouse
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
javax.jdo.option.ConnectionURL
jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true
JDBC connect string for a JDBC metastore
javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver
Driver class name for a JDBC metastore
javax.jdo.option.ConnectionUserName
hive
username to use against metastore database
javax.jdo.option.ConnectionPassword
micmiu
password to use against metastore database
hive.metastore.warehouse.dir
/user/hive/warehouse
location of default database for the warehouse
|
ps：需要把mysql的驱动包copy到目录 /lib 中
如果是第一次需要执行初始化命令：schematool -dbType mysql -initSchema
micmiu-mbp:mysql micmiu$ schematool -dbType mysql -initSchema 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 14/02/17 14:46:08 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative Metastore connection URL: jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: hive Starting metastore schema initialization to 0.12.0 Initialization script hive-schema-0.12.0.mysql.sql Initialization script completed schemaTool completeted
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
micmiu
-
mbp
:
mysql
micmiu
$
schematool
-
dbType
mysql
-
initSchema
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
input
.
dir
.
recursive
is
deprecated
.
Instead
,
use
mapreduce
.
input
.
fileinputformat
.
input
.
dir
.
recursive
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
max
.
split
.
size
is
deprecated
.
Instead
,
use
mapreduce
.
input
.
fileinputformat
.
split
.
maxsize
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
min
.
split
.
size
is
deprecated
.
Instead
,
use
mapreduce
.
input
.
fileinputformat
.
split
.
minsize
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
min
.
split
.
size
.
per
.
rack
is
deprecated
.
Instead
,
use
mapreduce
.
input
.
fileinputformat
.
split
.
minsize
.
per
.
rack
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
min
.
split
.
size
.
per
.
node
is
deprecated
.
Instead
,
use
mapreduce
.
input
.
fileinputformat
.
split
.
minsize
.
per
.
node
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
reduce
.
tasks
is
deprecated
.
Instead
,
use
mapreduce
.
job
.
reduces
14
/
02
/
17
14
:
46
:
08
INFO
Configuration
.
deprecation
:
mapred
.
reduce
.
tasks
.
speculative
.
execution
is
deprecated
.
Instead
,
use
mapreduce
.
reduce
.
speculative
Metastore
connection
URL
:
jdbc
:
mysql
:
//localhost/hive?createDatabaseIfNotExist=true
Metastore
Connection
Driver
:
com
.
mysql
.
jdbc
.
Driver
Metastore
connection
User
:
hive
Starting
metastore
schema
initialization
to
0.12.0
Initialization
script
hive
-
schema
-
0.12.0.mysql.sql
Initialization
script
completed
schemaTool
completeted
|
查看初始化后信息 schematool -dbType mysql -info
初始化后查看mysql中表情况：show tables;
mysql> show tables; +---------------------------+ | Tables_in_hive | +---------------------------+ | BUCKETING_COLS | | CDS | | COLUMNS_V2 | | DATABASE_PARAMS | | DBS | | DB_PRIVS | | DELEGATION_TOKENS | | GLOBAL_PRIVS | | IDXS | | INDEX_PARAMS | | MASTER_KEYS | | NUCLEUS_TABLES | | PARTITIONS | | PARTITION_EVENTS | | PARTITION_KEYS | | PARTITION_KEY_VALS | | PARTITION_PARAMS | | PART_COL_PRIVS | | PART_COL_STATS | | PART_PRIVS | | ROLES | | ROLE_MAP | | SDS | | SD_PARAMS | | SEQUENCE_TABLE | | SERDES | | SERDE_PARAMS | | SKEWED_COL_NAMES | | SKEWED_COL_VALUE_LOC_MAP | | SKEWED_STRING_LIST | | SKEWED_STRING_LIST_VALUES | | SKEWED_VALUES | | SORT_COLS | | TABLE_PARAMS | | TAB_COL_STATS | | TBLS | | TBL_COL_PRIVS | | TBL_PRIVS | | TYPES | | TYPE_FIELDS | | VERSION | +---------------------------+ 41 rows in set (0.00 sec)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
|
mysql
&
gt
;
show
tables
;
+
--
--
--
--
--
--
--
--
--
--
--
--
--
-
+
|
Tables_in_hive
|
+
--
--
--
--
--
--
--
--
--
--
--
--
--
-
+
|
BUCKETING_COLS
|
|
CDS
|
|
COLUMNS_V2
|
|
DATABASE_PARAMS
|
|
DBS
|
|
DB_PRIVS
|
|
DELEGATION_TOKENS
|
|
GLOBAL_PRIVS
|
|
IDXS
|
|
INDEX_PARAMS
|
|
MASTER_KEYS
|
|
NUCLEUS_TABLES
|
|
PARTITIONS
|
|
PARTITION_EVENTS
|
|
PARTITION_KEYS
|
|
PARTITION_KEY_VALS
|
|
PARTITION_PARAMS
|
|
PART_COL_PRIVS
|
|
PART_COL_STATS
|
|
PART_PRIVS
|
|
ROLES
|
|
ROLE_MAP
|
|
SDS
|
|
SD_PARAMS
|
|
SEQUENCE_TABLE
|
|
SERDES
|
|
SERDE_PARAMS
|
|
SKEWED_COL_NAMES
|
|
SKEWED_COL_VALUE_LOC_MAP
|
|
SKEWED_STRING_LIST
|
|
SKEWED_STRING_LIST_VALUES
|
|
SKEWED_VALUES
|
|
SORT_COLS
|
|
TABLE_PARAMS
|
|
TAB_COL_STATS
|
|
TBLS
|
|
TBL_COL_PRIVS
|
|
TBL_PRIVS
|
|
TYPES
|
|
TYPE_FIELDS
|
|
VERSION
|
+
--
--
--
--
--
--
--
--
--
--
--
--
--
-
+
41
rows
in
set
(
0.00
sec
)
|
配置完成后就可在shell中以CLI的方式访问hive 进行操作验证。
[三]、Remote方式
以Mysql数据库(192.168.6.77)为例：创建好用户：hive；database：hive_meta。Remote方式需要分别配置服务端和客户端的配置文件：
服务端的 hive-site.xml 中jdbc URL、驱动、用户名、密码等属性值配置如下：
javax.jdo.option.ConnectionURL jdbc:mysql://192.168.6.77/hive_meta?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName hive username to use against metastore database javax.jdo.option.ConnectionPassword micmiu password to use against metastore database hive.metastore.warehouse.dir /user/hive/warehouse base hdfs path :location of default database for the warehouse
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
javax.jdo.option.ConnectionURL
jdbc:mysql://192.168.6.77/hive_meta?createDatabaseIfNotExist=true
JDBC connect string for a JDBC metastore
javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver
Driver class name for a JDBC metastore
javax.jdo.option.ConnectionUserName
hive
username to use against metastore database
javax.jdo.option.ConnectionPassword
micmiu
password to use against metastore database
hive.metastore.warehouse.dir
/user/hive/warehouse
base hdfs path :location of default database for the warehouse
|
ps：需要把mysql的驱动包copy到目录 /lib 中
如果是第一次需要执行初始化命令：schematool -dbType mysql -initSchema
客户端中配置内容修改如下：
hive.metastore.uris thrift://192.168.6.77:9083 Thrift uri for the remote metastore. Used by metastore client to connect to remote metastore. hive.metastore.warehouse.dir /user/hive/warehouse location of default database for the warehouse
1
2
3
4
5
6
7
8
9
10
11
12
13
|
hive.metastore.uris
thrift://192.168.6.77:9083
Thrift uri for the remote metastore. Used by metastore client to connect to remote metastore.
hive.metastore.warehouse.dir
/user/hive/warehouse
location of default database for the warehouse
|
hive metastore 服务端启动命令：
hive --service metastore -p
如果不加端口默认启动：hive --service metastore，则默认监听端口是：9083 ，注意客户端中的端口配置需要和启动监听的端口一致。服务端启动正常后，客户端就可以执行hive操作了。
