HBase伪分布式集群的复制
0. prerequisite
Suppose 2 hbase pseudo distributed clusters have both started as folowing
| relevant parameters in hbase-site.xml | source | destnation |
|---|---|---|
| hbase.zookeeper.quorum | ubuntu | centos2 |
| hbase.zookeeper.property.clientPort | 2181 | 2181 |
| zookeeper.znode.parent | /hbase | /hbase |
1. Create table for replication
1) start hbase shell on source cluster and create a table
$ cd $HOME_HBASE
$ bin/hbase shell
> create 'manga:fruit', 'cf'
> describe 'manga:fruit'
Table manga:fruit is ENABLED
manga:fruit
COLUMN FAMILIES DESCRIPTION
{NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'F
ALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2) do excatly same thing on destination cluster
$ cd $HOME_HBASE
$ bin/hbase shell
> create 'manga:fruit', 'cf'2. In source cluster hbase shell, add the destination cluster as a peer
> add_peer 'centos2', CLUSTER_KEY => "centos2:2181:/hbase"
> list_peer
hbase:028:0> list_peers
PEER_ID CLUSTER_KEY ENDPOINT_CLASSNAME STATE REPLICATE_ALL NAMESPACES TABLE_CFS BANDWIDTH SERIAL
centos2 centos2:2181:/hbase ENABLED true 0 false
1 row(s)we can see following message in source hbase master log
2023-06-23 17:32:32,551 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Client=sunxo//192.168.55.250 creating replication peer, id=centos2, config=clusterKey=centos2:2181:/hbase,replicationEndpointImpl=null,replicateAllUserTables=true,bandwidth=0,serial=false, state=ENABLED
Note, the peer can disable and remove by using commands
> disable_peer 'centos2'
> remove_peer 'centos2'3. In source cluster hbase shell, enable the table for replication
> enable_table_replication 'manga:fruit'we can see following message in source hbase master log
2023-06-23 17:42:27,353 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Client=sunxo//192.168.55.250 modify table manga:fruit from 'manga:fruit', {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'manga:fruit', {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}Note, the replication can disable by
> disable_table_replication 'manga:fruit'4. try replication
1) in source cluster hbase shell, put data
> put 'manga:fruit', 103, 'cf:name', '苹果'
> put 'manga:fruit', 103, 'cf:price', 120.0
> put 'manga:fruit', 104, 'cf:name', '柠檬'
> put 'manga:fruit', 104, 'cf:price', 200.02) in destination cluster hbase shell, check
> scan 'manga:fruit'
scan 'manga:fruit'
ROW COLUMN+CELL
103 column=cf:name, timestamp=2023-06-23T17:45:20.969, value=\xE8\x8B\xB9\xE6\x9E\x9C
103 column=cf:price, timestamp=2023-06-23T17:45:21.010, value=120.0
104 column=cf:name, timestamp=2023-06-23T17:45:21.036, value=\xE6\x9F\xA0\xE6\xAA\xAC
104 column=cf:price, timestamp=2023-06-23T17:45:21.049, value=200.0 we can see following message in destination hbase region log
2023-06-23 17:45:33,436 INFO [centos2:16020Replication Statistics #0] regionserver.Replication: Sink: age in ms of last applied edit: 0, total replicated edits: 4Reference:
https://hbase.apache.org/book.html#_cluster_replication
https://blog.csdn.net/sun_xo/article/details/131256870