hive 支持行级update、delete操作权限

1. 找到对应的hive-site.xml文件,加上以下配置:

 
    hive.support.concurrency
    true


    hive.enforce.bucketing
    true


    hive.exec.dynamic.partition.mode
    nonstrict


    hive.txn.manager
    org.apache.hadoop.hive.ql.lockmgr.DbTxnManager


    hive.compactor.initiator.on
    true


    hive.compactor.worker.threads
    1

进行重启hive元数据服务metastore,再通过beeline到hive客户端进行操作。

2. 进行更新表数据,成功!

0: jdbc:hive2://demo3.leap.com:2181,demo1.lea> select * from test_cdc;
INFO  : Compiling command(queryId=hive_20180607104040_34af3962-1273-4a35-9725-c0e653d2b31d): select * from test_cdc
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:test_cdc.id, type:int, comment:null), FieldSchema(name:test_cdc.name, type:string, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20180607104040_34af3962-1273-4a35-9725-c0e653d2b31d); Time taken: 1.781 seconds
INFO  : Executing command(queryId=hive_20180607104040_34af3962-1273-4a35-9725-c0e653d2b31d): select * from test_cdc
INFO  : Completed executing command(queryId=hive_20180607104040_34af3962-1273-4a35-9725-c0e653d2b31d); Time taken: 0.006 seconds
INFO  : OK
+--------------+----------------+--+
| test_cdc.id  | test_cdc.name  |
+--------------+----------------+--+
| 1            | aaa            |
+--------------+----------------+--+
1 row selected (2.334 seconds)
0: jdbc:hive2://demo3.leap.com:2181,demo1.lea> update test_cdc set name = 'ccc' where id =1; 
INFO  : Compiling command(queryId=hive_20180607104040_2eee4dba-43ec-4e49-b559-1548ef9dbb05): update test_cdc set name = 'ccc' where id =1
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:row__id, type:struct, comment:null), FieldSchema(name:id, type:int, comment:null), FieldSchema(name:_c2, type:string, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20180607104040_2eee4dba-43ec-4e49-b559-1548ef9dbb05); Time taken: 0.703 seconds
INFO  : Executing command(queryId=hive_20180607104040_2eee4dba-43ec-4e49-b559-1548ef9dbb05): update test_cdc set name = 'ccc' where id =1
INFO  : Query ID = hive_20180607104040_2eee4dba-43ec-4e49-b559-1548ef9dbb05
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Number of reduce tasks determined at compile time: 8
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=
INFO  : number of splits:8
INFO  : Submitting tokens for job: job_1528335916181_0001
INFO  : The url to track the job: http://demo1.leap.com:8088/proxy/application_1528335916181_0001/
INFO  : Starting Job = job_1528335916181_0001, Tracking URL = http://demo1.leap.com:8088/proxy/application_1528335916181_0001/
INFO  : Kill Command = /usr/bin/hadoop job  -kill job_1528335916181_0001
INFO  : Hadoop job information for Stage-1: number of mappers: 8; number of reducers: 8
INFO  : 2018-06-07 10:40:51,464 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-06-07 10:40:58,846 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 4.2 sec
INFO  : 2018-06-07 10:40:59,889 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 8.44 sec
INFO  : 2018-06-07 10:41:00,932 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 24.08 sec
INFO  : 2018-06-07 10:41:08,860 Stage-1 map = 100%,  reduce = 50%, Cumulative CPU 34.19 sec
INFO  : 2018-06-07 10:41:09,903 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 44.77 sec
INFO  : MapReduce Total cumulative CPU time: 44 seconds 770 msec
INFO  : Ended Job = job_1528335916181_0001
INFO  : Starting task [Stage-0:MOVE] in serial mode
INFO  : Loading data to table default.test_cdc from hdfs://demo1.leap.com:8020/apps/hive/warehouse/test_cdc/.hive-staging_hive_2018-06-07_10-40-42_440_5275939439062405091-1/-ext-10000
INFO  : Starting task [Stage-2:STATS] in serial mode
INFO  : Table default.test_cdc stats: [numFiles=9, numRows=1, totalSize=1234, rawDataSize=91]
INFO  : MapReduce Jobs Launched: 
INFO  : Stage-Stage-1: Map: 8  Reduce: 8   Cumulative CPU: 44.77 sec   HDFS Read: 70685 HDFS Write: 979 SUCCESS
INFO  : Total MapReduce CPU Time Spent: 44 seconds 770 msec
INFO  : Completed executing command(queryId=hive_20180607104040_2eee4dba-43ec-4e49-b559-1548ef9dbb05); Time taken: 30.418 seconds
INFO  : OK
No rows affected (31.262 seconds)
0: jdbc:hive2://demo3.leap.com:2181,demo1.lea> select * from test_cdc;
INFO  : Compiling command(queryId=hive_20180607104141_8986f857-f02e-4c32-854b-d224b3414f04): select * from test_cdc
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:test_cdc.id, type:int, comment:null), FieldSchema(name:test_cdc.name, type:string, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20180607104141_8986f857-f02e-4c32-854b-d224b3414f04); Time taken: 0.22 seconds
INFO  : Executing command(queryId=hive_20180607104141_8986f857-f02e-4c32-854b-d224b3414f04): select * from test_cdc
INFO  : Completed executing command(queryId=hive_20180607104141_8986f857-f02e-4c32-854b-d224b3414f04); Time taken: 0.001 seconds
INFO  : OK
+--------------+----------------+--+
| test_cdc.id  | test_cdc.name  |
+--------------+----------------+--+
| 1            | ccc            |
+--------------+----------------+--+
1 row selected (0.415 seconds)


你可能感兴趣的:(大数据学习及分享)