hgdw安装教程
Greenplum docs:http://gpdb.docs.pivotal.io/580/utility_guide/admin_utilities/gpstate.html
找三台centos7的虚拟机，我使用的是7.8
1台主机(Master)
1台备用主机(standby master)
1台计算节点(segment)
一般segment(计算节点)要大于主机的个数
修改hostname方法， hostnamectl set-hostname xxxxx ，然后su - 切换
配置主机的hosts
[root@baidu /]# vi /etc/hosts
#master
192.168.90.123 baidu
#standby master
192.168.90.124 google
#segment
192.168.90.106 sougou尝试去ping各个节点，通了说明成功
ping google
ping sougou
每个节点关闭防火墙
systemctl disable firewalld.service
hosts配置文件配置到子节点
scp /etc/hosts google:/etc/
scp /etc/hosts sougou:/etc/
所有节点创建hgadmin用户，修改用户密码
useradd hgadmin
passwd hgadmin
所有节点创建data目录，把所有者给hgadmin
创建master数据目录
mkdir -p /data/hgdata/master
chown hgadmin:hgadmin /data -R
创建standby master数据目录
mkdir -p /data/hgdata
chown hgadmin:hgadmin /data -R
创建segment数据目录
mkdir -p /data/hgdata/hgseg
chown hgadmin:hgadmin /data -R
所有节点创建highgo目录，用于后期存放应用的目录
mkdir -p /highgo
chown hgadmin:hgadmin /highgo -R
安装包程序上传到主机(子节点会通过scp传输安装包)，解压到/highgo目录下，修改owner
[root@baidu highgo]# pwd
/highgo
[root@baidu highgo]# mv /opt/hgdw_3.0_gis_x64.zip ./
[root@baidu highgo]# unzip hgdw_3.0_gis_x64.zip
...
[root@baidu highgo]# chown hgadmin:hgadmin hgdw -R切换到hgadmin
su - hgadmin
生成hgxxxx可执行命令
[root@baidu hgdw]# pwd
/highgo/hgdw
[root@baidu hgdw]# vi hgdw_path.sh
#HGDWHOME=/usr/local/hgdw
HGDWHOME=/highgo/hgdw
...
#保存文件
[root@baidu hgdw]# source hgdw_path.sh
[root@baidu hgdw]# echo $HGDWHOME
/highgo/hgdw修改db节点配置文件(用户hgssh发送命令)
标识全部节点
[hgadmin@baidu hgdw]$ vi /highgo/hgdw/conf/hostlist
baidu
sougou标识计算节点
[hgadmin@baidu hgdw]$ vi /highgo/hgdw/conf/seglist
sougou
建立免密互信
一直按回车，或者yes
ssh-keygen -t rsa
cd ~/.ssh/
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys
ssh-copy-id -i id_rsa.pub hgadmin@google
ssh-copy-id -i id_rsa.pub hgadmin@sougou
验证免密互信
[hgadmin@baidu hgdw]$ pwd
/usr/local/hgdw
[hgadmin@baidu hgdw]$ hgssh-exkeys -f ./conf/hostlist
[STEP 1 of 5] create local ID and authorize on local host
...
[STEP 5 of 5] copy authentication files to all remote hosts
... finished key exchange with google
... finished key exchange with sougou
[INFO] completed successfully如果报：hgssh-exkeys command not found ，说明source hgdw_path.sh 这一步出错了
使用hgssh命令验证，实际上就是发送的ssh命令，比如pwd，或者touch 一个文件
[hgadmin@baidu hgdw]$ hgssh -f conf/hostlist
=> pwd
[google] /home/hgadmin
[sougou] /home/hgadmin
[ baidu] /home/hgadmin设置主机环境变量，都非常重要
[hgadmin@baidu hgdw]$ vi ~/.bash_profile
...
source /highgo/hgdw/hgdw_path.sh
export MASTER_DATA_DIRECTORY=/data/hgdata/master/gpseg-1
export PGPORT=2345
export PGUSER=hgadmin
#设置PGDATABASE,涉及psql/gpdeletesystem等涉及database的命令
export PGDATABASE=gpdb
[hgadmin@baidu hgdw]$ source ~/.bash_profile 设置时间同步
查看时间是否同步
[hgadmin@baidu hgdw]$ hgssh -f conf/hostlist 'date'
[google] Fri Aug 14 18:47:16 CST 2020
[sougou] Fri Aug 14 18:47:16 CST 2020
[ baidu] Fri Aug 14 18:47:15 CST 2020如果不同步，启用master节点上的ntp，并在Segment节点上配置和启用NTP：（时钟同步）
由于时间有限，这一步没有实践过，有空补上
连通性检查，节点之间文件读取时间(意义不大，警告可以继续往下走)
[hgadmin@baidu hgdw]$ gpcheckperf -f conf/hostlist -r N -d /tmp
/highgo/hgdw/bin/gpcheckperf -f conf/hostlist -r N -d /tmp
...
-- NETPERF TEST
...
Summary:
sum = 687.55 MB/sec
min = 166.83 MB/sec
max = 177.68 MB/sec
avg = 171.89 MB/sec
median = 174.91 MB/sec
[hgadmin@baidu hgdw]$ 拷贝应用到子节点并解压，standby master也需要拷贝
[hgadmin@baidu hgdw]$ tar -cvf ../hgdw.tar ../hgdw
[hgadmin@baidu hgdw]$ scp ../hgdw.tar google:/highgo/
[hgadmin@baidu hgdw]$ scp ../hgdw.tar sougou:/highgo/
[hgadmin@baidu hgdw]$ hgssh -f conf/seglist
=> tar -xvf hgdw.tar
...
=> exit
[hgadmin@baidu hgdw]$ hgssh -h google
=> pwd
[google] /home/hgadmin
=> tar -xvf /highgo/hgdw.tar
...
-> exit拷贝环境变量到子节点(如果不打算去segment使用psql，这一步不需要做)
[hgadmin@baidu hgdw]$ scp ~/.bash_profile sougou:$HOME
用的时候自己去source吧，就不在这执行了
初始化数据库配置文件
[hgadmin@baidu hgdw]$ cp docs/cli_help/hgconfigs/hginitsystem_config ./conf/
#对以下配置进行修改
[hgadmin@baidu hgdw]$ vi conf/hginitsystem_config
ARRAY_NAME="Highgo DW"
SEG_PREFIX=gpseg
PORT_BASE=3000
#每个segment(计算节点)创建几个data目录
declare -a DATA_DIRECTORY=(/data/hgdata/hgseg /data/hgdata/hgseg)
#主机名
MASTER_HOSTNAME=baidu
#主库目录
MASTER_DIRECTORY=/data/hgdata/master/
#对应环境变量PGPORT
MASTER_PORT=2345
TRUSTED_SHELL=ssh
CHECK_POINT_SEGMENTS=18
ENCODING=UNICODE
#对应环境变量的PGDATABASE,前面的#一定要打开，不打开默认是postgres
DATABASE_NAME=hgdb初始化数据库(standby master等计算节点完成在创建)
[hgadmin@baidu hgdw]$ hginitsystem -c ./conf/hginitsystem_config -h ./conf/seglist
...
#中间需要输入y
..
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-Review options for gpinitstandby
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-------------------------------------------------------
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-The Master /data/hgdata/master/gpseg-1/pg_hba.conf post gpinitsystem
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-has been configured to allow all hosts within this new
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-array to intercommunicate. Any hosts external to this
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-new array must be explicitly added to this file
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-Refer to the HGDW Admin support guide which is
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-located in the /highgo/hgdw/docs directory
20200814:19:27:58:103224 hginitsystem:baidu:hgadmin-[INFO]:-------------------------------------------------------hginitsystem：创建一个HGDW Database实例，一个主机和1个计算节点实例
-c：指定初始化文件。
-h：指定segment主机文件。
如果不报ERROR，说明HG数据库就安装好了，并且各个节点都启动了
查看主机和segment(计算节点)进程
ps -ef |grep postgres
数据库已启动
hgstart：启动集群
hgstop：关闭集群
hgstate：查看集群状态
失败情况，删除后重新安装
gpdeletesystem -d /data/hgdata/master/gpseg-1 -f
如果配置了环境变量，这里直接gpdeletesystem 即可
-d 后面跟 MASTER_DATA_DIRECTORY（master 的数据目录）
-f force， 终止所有进程，强制删除
配置standby master(备用主机) 高可用
[hgadmin@baidu hgdw]$ gpinitstandby -s google -S /data/hgdata/standby -P 2345
...
20200817:10:57:06:023216 gpinitstandby:baidu:hgadmin-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.
20200817:10:57:06:023216 gpinitstandby:baidu:hgadmin-[INFO]:-Successfully created standby master on google会出现[ERROR]:-Unable to update pg_hba.conf bash: /highgo/hgdw/lib/python/gppylib/operations/initstandby.py: Permission denied的错误，这不影响。
查看高可用状态
[hgadmin@baidu hgdw]$ hgstate -f
...
20200817:10:57:11:023283 hgstate:baidu:hgadmin-[INFO]:-Obtaining Segment details from master...
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:-Standby master details
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:-----------------------
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:- Standby address = google
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:- Standby data directory = /data/hgdata/standby
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:- Standby port = 2345
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:- Standby PID = 12061
20200817:10:57:12:023283 hgstate:baidu:hgadmin-[INFO]:- Standby status = Standby host passive
...查看计算节点状态
[hgadmin@baidu hgdw]$ hgstate -s
20200817:11:00:06:023348 hgstate:baidu:hgadmin-[INFO]:-Starting hgstate with args: -s
20200817:11:00:06:023348 hgstate:baidu:hgadmin-[INFO]:-local HGDW Version: 'postgres (HGDW Database) 3.0'
20200817:11:00:06:023348 hgstate:baidu:hgadmin-[INFO]:-master HGDW Version: 'PostgreSQL 9.4.24 (HGDW Database 3.0 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit compiled on Mar 26 2020 17:28:51'
20200817:11:00:06:023348 hgstate:baidu:hgadmin-[INFO]:-Obtaining Segment details from master...
20200817:11:00:06:023348 hgstate:baidu:hgadmin-[INFO]:-Gathering data from segments...
...
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:-Segment Instance Status Report
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:-----------------------------------------------------
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Segment Info
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Hostname = sougou
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Address = sougou
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Datadir = /data/hgdata/hgseg/gpseg0
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Port = 3000
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Status
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- PID = 14702
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Configuration reports status as = Up
20200817:11:03:40:025648 hgstate:baidu:hgadmin-[INFO]:- Database status = Up
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:-----------------------------------------------------
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Segment Info
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Hostname = sougou
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Address = sougou
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Datadir = /data/hgdata/hgseg/gpseg1
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Port = 3001
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Status
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- PID = 16997
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Configuration reports status as = Up
20200817:12:55:20:030038 hgstate:baidu:hgadmin-[INFO]:- Database status = Up
...扩展segment(计算节点)
虚拟机有限，先给备机添加计算节点
在google(备用主机)创建计算节点的data目录
mkdir /data/hgdata/hgseg
输入y，后面回车，生成inputfile文件
这里省略了-D hgadmin(数据库名称)
因为环境变量里配置了PGDATABASE
[hgadmin@baidu hgdw]$ vi conf/expand_file
google
[hgadmin@baidu hgdw]$ gpexpand -f conf/expand_file
20200817:11:20:13:028857 gpexpand:baidu:hgadmin-[INFO]:-local HGDW Version: 'postgres (HGDW Database) 3.0'
20200817:11:20:13:028857 gpexpand:baidu:hgadmin-[INFO]:-master HGDW Version: 'PostgreSQL 9.4.24 (HGDW Database 3.0 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit compiled on Mar 26 2020 17:28:51'
20200817:11:20:13:028857 gpexpand:baidu:hgadmin-[INFO]:-Querying gpexpand schema for current expansion state
System Expansion is used to add segments to an existing GPDB array.
gpexpand did not detect a System Expansion that is in progress.
Before initiating a System Expansion, you need to provision and burn-in
the new hardware. Please be sure to run gpcheckperf to make sure the
new hardware is working properly.
Please refer to the Admin Guide for more information.
Would you like to initiate a new System Expansion Yy|Nn (default=N):
> y
By default, new hosts are configured with the same number of primary
segments as existing hosts. Optionally, you can increase the number
of segments per host.
For example, if existing hosts have two primary segments, entering a value
of 2 will initialize two additional segments on existing hosts, and four
segments on new hosts. In addition, mirror segments will be added for
these new primary segments if mirroring is enabled.
How many new primary segments per host do you want to add? (default=0):
>
Generating configuration file...
20200817:11:20:36:028857 gpexpand:baidu:hgadmin-[INFO]:-Generating input file...
Input configuration file was written to 'gpexpand_inputfile_20200817_112036'.
Please review the file and make sure that it is correct then re-run
with: gpexpand -i gpexpand_inputfile_20200817_112036
20200817:11:20:36:028857 gpexpand:baidu:hgadmin-[INFO]:-Exiting...此处为每个Segment主机添加的节点个数，0就是只有新的主机添加一个Segment，如果输入1的话，将会在所有主机再添加一个Segment，包括新添加的SDW4主机，也就是 如果这里输入1的话，会在sdw4主机上添加两个Segment
挂载inputfile文件，来初始化新段并在默认数据库中创建扩展模式
[hgadmin@baidu hgdw]$ gpexpand -i gpexpand_inputfile_20200817_112036
将节点重新分布，最长运行1小时
$ gpexpand -d 01:00:00
该命令会对所有的数据库和表进行重分布。按照对应表的分布键，把数据打散到各个节点，包括新增加的机器。从而实现了扩容操作
查看hgstate -s