kudu 添加master

 kudu 添加master节点不像其他大数据组件直接添加即可,操作比较复杂,步骤如下:

1.kudu添加master节点,集群不可用

2.Master节点数为奇数

3.在已存在master节点查看wal和数据目录,rpc_bind_addresses 端口,默认7051

       sudo -u kudu kudu fs dump uuid --fs_wal_dir= [--fs_data_dirs=] 2>/dev/null

       sudo -u kudu kudu fs dump uuid --fs_wal_dir=/data/data1/kmaster 2>/dev/null

       当 fs_wal_dir 和--fs_data_dirs不在同一目录下时使用如下命令

          sudo -u kudu kudu fs dump uuid --fs_wal_dir=/data/data1/kmaster

         -- fs_data_dirs=/data/data2/kmdata,/data/data3/kmdata,/data/data4/kmdata 2>/dev/null

4. 使用impala访问kudu,需要更新hive metastore数据库,在impala shell中执行如下语句

    ALTER TABLE table_name

SET TBLPROPERTIES

('kudu.master_addresses' = 'master-1,master-2,master-3');

5. 停止kudu集群

6. 在新master节点,执行以下命令

   Sudo -u kudu kudu fs format --fs_wal_dir= [--fs_data_dirs=]

   $ sudo -u kudu kudu fs dump uuid --fs_wal_dir= [--fs_data_dirs=] 2>/dev/null

   sudo -u kudu kudu fs format --fs_wal_dir=/data/data3/kmaster/wal 

    -- fs_data_dirs=/data/data1/kmaster/data,/data/data2/kmaster/data,/data/data4/kmaster/data

    sudo -u kudu kudu fs dump uuid --fs_wal_dir=/data/data3/kmaster/wal 

    --fs_data_dirs=/data/data1/kmaster/data,/data/data2/kmaster/data,/data/data4/kmaster/data 2>/dev/null

7. 在新服务上添加master角色实例不启动

8. 在已有的master节点上执行

   sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir= [--fs_data_dirs=]

sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=/data/data3/kmaster/wal --fs_data_dirs=/data/data1/kmaster/data,/data/data2/kmaster/data,/data/data4/kmaster/data 00000000000000000000000000000000 5cd17814d4e748e0b52b776daaff21bf:cdh-15-170:7051 164623e4b7ee45139ad00f64ee84be84:cdh-15-171:7051 85534db90b604fe38323041af882363b:cdh-15-172:7051

9. Cm界面启动原kudu master,新添加的不启动

10. 复制master数据

 sudo -u kudu kudu local_replica copy_from_remote --fs_wal_dir=/data/data1/kmaster --fs_data_dirs=/data/data1/kmdata,/data/data2/kmdata,/data/data4/kmdata  00000000000000000000000000000000 cdh-15-170:7051

11. 启动新添加的master

12. 启动tablet server

  1.  如果您有从Impala访问的Kudu表并且未设置DNS别名,请在为HMS提供存储的基础数据库中手动更新HMS数据库。

 UPDATE TABLE_PARAMS

SET PARAM_VALUE =

  'master-1.example.com,master-2.example.com,master-3.example.com'

WHERE PARAM_KEY = 'kudu.master_addresses' AND PARAM_VALUE = 'old-master';  这步操作根据需要执行,跟第四步操作一样。

遇到异常:

Kudu启动异常:

F1225 17:40:53.693409 11791 master_main.cc:77] Check failed: _s.ok() Bad status: Invalid argument: Unable to initialize catalog manager: Failed to initialize sys tables async: on-disk master list (cdh-15-170:7051, cdh-15-171:7051, cdh-15-172:7051) 
and provided master list (cdh-15-170.test.com:7051, cdh-15-171.test.com:7051, cdh-15-172.test.com:7051) differ.

 将第8部的命令ip 改为全域名

 sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=/data/data1/kmaster --fs_data_dirs=/data/data3/kmdata,/data/data2/kmdata,/data/data4/kmdata 00000000000000000000000000000000 423d009cfd6348ddbec66bb9873b6209:cdh-15-170.test.com:7051 f558ae92dce9426386298a1dbb358f68:cdh-15-171.test.com:7051 15e3d8c401964f45a355fdf6ae00e415:cdh-15-172.test.com:7051

 

你可能感兴趣的:(kudu,个人总结)