kudu 添加master节点不像其他大数据组件直接添加即可,操作比较复杂,步骤如下:
1.kudu添加master节点,集群不可用
2.Master节点数为奇数
3.在已存在master节点查看wal和数据目录,rpc_bind_addresses 端口,默认7051
sudo -u kudu kudu fs dump uuid --fs_wal_dir=
sudo -u kudu kudu fs dump uuid --fs_wal_dir=/data/data1/kmaster 2>/dev/null
当 fs_wal_dir 和--fs_data_dirs不在同一目录下时使用如下命令
sudo -u kudu kudu fs dump uuid --fs_wal_dir=/data/data1/kmaster
-- fs_data_dirs=/data/data2/kmdata,/data/data3/kmdata,/data/data4/kmdata 2>/dev/null
4. 使用impala访问kudu,需要更新hive metastore数据库,在impala shell中执行如下语句
ALTER TABLE table_name
SET TBLPROPERTIES
('kudu.master_addresses' = 'master-1,master-2,master-3');
5. 停止kudu集群
6. 在新master节点,执行以下命令
Sudo -u kudu kudu fs format --fs_wal_dir=
$ sudo -u kudu kudu fs dump uuid --fs_wal_dir=
sudo -u kudu kudu fs format --fs_wal_dir=/data/data3/kmaster/wal
-- fs_data_dirs=/data/data1/kmaster/data,/data/data2/kmaster/data,/data/data4/kmaster/data
sudo -u kudu kudu fs dump uuid --fs_wal_dir=/data/data3/kmaster/wal
--fs_data_dirs=/data/data1/kmaster/data,/data/data2/kmaster/data,/data/data4/kmaster/data 2>/dev/null
7. 在新服务上添加master角色实例不启动
8. 在已有的master节点上执行
sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=
sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=/data/data3/kmaster/wal --fs_data_dirs=/data/data1/kmaster/data,/data/data2/kmaster/data,/data/data4/kmaster/data 00000000000000000000000000000000 5cd17814d4e748e0b52b776daaff21bf:cdh-15-170:7051 164623e4b7ee45139ad00f64ee84be84:cdh-15-171:7051 85534db90b604fe38323041af882363b:cdh-15-172:7051
9. Cm界面启动原kudu master,新添加的不启动
10. 复制master数据
sudo -u kudu kudu local_replica copy_from_remote --fs_wal_dir=/data/data1/kmaster --fs_data_dirs=/data/data1/kmdata,/data/data2/kmdata,/data/data4/kmdata 00000000000000000000000000000000 cdh-15-170:7051
11. 启动新添加的master
12. 启动tablet server
UPDATE TABLE_PARAMS
SET PARAM_VALUE =
'master-1.example.com,master-2.example.com,master-3.example.com'
WHERE PARAM_KEY = 'kudu.master_addresses' AND PARAM_VALUE = 'old-master'; 这步操作根据需要执行,跟第四步操作一样。
遇到异常:
Kudu启动异常:
F1225 17:40:53.693409 11791 master_main.cc:77] Check failed: _s.ok() Bad status: Invalid argument: Unable to initialize catalog manager: Failed to initialize sys tables async: on-disk master list (cdh-15-170:7051, cdh-15-171:7051, cdh-15-172:7051)
and provided master list (cdh-15-170.test.com:7051, cdh-15-171.test.com:7051, cdh-15-172.test.com:7051) differ.
将第8部的命令ip 改为全域名
sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=/data/data1/kmaster --fs_data_dirs=/data/data3/kmdata,/data/data2/kmdata,/data/data4/kmdata 00000000000000000000000000000000 423d009cfd6348ddbec66bb9873b6209:cdh-15-170.test.com:7051 f558ae92dce9426386298a1dbb358f68:cdh-15-171.test.com:7051 15e3d8c401964f45a355fdf6ae00e415:cdh-15-172.test.com:7051