安装Kudu集群必须是奇数的集群,建议用一个主节点(无容错)或者三个主节点(可以允许容错一个),双主机部署无容错功能。
集群环境:
节点 | 主机名 | 角色 |
---|---|---|
192.168.11.70 | hnode1 | Master Server、Tablet Server |
192.168.11.71 | hnode2 | Master Server、Tablet Server |
192.168.11.72 | hnode3 | Master Server、Tablet Server |
在hnode1下载
http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.13.1/RPMS/x86_64/
kudu-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
kudu-client-devel-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
kudu-client0-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
kudu-debuginfo-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
kudu-master-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
kudu-tserver-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
在hnode1使用yum 本地安装模式,可以关联安装依赖包
yum localinstall -y kudu-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
yum localinstall -y kudu-client0-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
yum localinstall -y kudu-debuginfo-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
yum localinstall -y kudu-master-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
yum localinstall -y kudu-tserver-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
yum localinstall -y kudu-client-devel-1.5.0+cdh5.13.1+0-1.cdh5.13.1.p0.3.el6.x86_64.rpm
通过scp命令将安装包拷贝到hnode2、hnode3,并执行上面的安装步骤
scp -r [email protected]:/opt/modules/kudu/*.rpm /opt/modules/kudu
kudu配置项分为稳定配置(stable),高级配置(advanced)
稳定配置可用于配置常见任务;
高级配置包含有一些正在进行尝试的调试,可能会引起bug;
采用rpm安装之后,配置文件默认存在/etc/kudu/conf
节点配置说明:
----master.gflagfile
----block_cache_capacity_mb 块高速缓存容量(MB)(选配)
----log_force_fsync_all 每次写入后,Log/WAL 是否应显式调用(选配)
----fs_data_dirs Master 将放置其数据块的目录的逗号分隔列表(必须配置)
----master-addresses kudu-master的rpc通讯地址,不指定默认为单节点(多master必须配置,单master不能配置)
----fs_wal_dir Master 将其预写日志的目录(必须配置)
----log_dir 存储主日志文件的目录。(必须配置)
----tserver.gflagfile
----fs_data_dirs Tablet Server 将放置其数据块的目录的逗号分隔列表(必须配置)
----fs_wal_dir Tablet Server 将放置预写日志的目录(必须配置)
----log_dir 存储 Tablet Server 日志文件的目录(必须配置)
----tserver_master_addrs tablet server 应连接的主机的逗号分隔地址(必须配置)
----block_cache_capacity_mb 分配给 Kudu Tablet Server 块缓存的最大内存量(选配)
----memory_limit_hard_bytes Tablet Server 预存传入并未写入时消耗的最大内存量(选配)
Master配置操作如下:
vi /etc/default/kudu-master
hnode1
export FLAGS_log_dir=/var/log/kudu
export FLAGS_rpc_bind_addresses=hnode1:7051
hnode2
export FLAGS_log_dir=/var/log/kudu
export FLAGS_rpc_bind_addresses=hnode2:7051
hnode3
export FLAGS_log_dir=/var/log/kudu
export FLAGS_rpc_bind_addresses=hnode3:7051
保存退出
vi /etc/kudu/conf/master.gflagfile
hnode1、hnode2、hnode3配置相同,文件内容如下:
# Do not modify these two lines. If you wish to change these variables,
# modify them in /etc/default/kudu-tserver.
--fromenv=rpc_bind_addresses
--fromenv=log_dir
--fs_wal_dir=/opt/modules/kudu/tserver/fs_wal_dir
--fs_data_dirs=/opt/modules/kudu/tserver/fs_data_dirs
--tserver_master_addrs=hnode1:7051,hnode2:7051,hnode3:7051
Server配置如下:
vi /etc/default/kudu-tserver
hnode1
export FLAGS_log_dir=/var/log/kudu
export FLAGS_rpc_bind_addresses=hnode1:7050
hnode2
export FLAGS_log_dir=/var/log/kudu
export FLAGS_rpc_bind_addresses=hnode2:7050
hnode3
export FLAGS_log_dir=/var/log/kudu
export FLAGS_rpc_bind_addresses=hnode3:7050
保存退出
vi /etc/kudu/conf/tserver.gflagfile
hnode1、hnode2、hnode3文件内容如下:
# Do not modify these two lines. If you wish to change these variables,
# modify them in /etc/default/kudu-tserver.
--fromenv=rpc_bind_addresses
--fromenv=log_dir
--fs_wal_dir=/opt/modules/kudu/tserver/fs_wal_dir
--fs_data_dirs=/opt/modules/kudu/tserver/fs_data_dirs
--tserver_master_addrs=hnode1:7051,hnode2:7051,hnode3:7051
NTp服务用于节点之间进行时间同步,如果只有一台主机,可以不用安装,如果有一台以上,则需要安装NTP服务进行时间同步。
安装参考:https://blog.csdn.net/weixin_43865893/article/details/86678065
给kudu用户设置目录的访问权限
chown -R kudu:kudu /opt/modules/kudu
service kudu-master start
service kudu-tserver start
查看所有Master
http://hnode1:8051/masters
查看所有Tablet
http://hnode1:8051/tablet-servers