GreenPlum版本升级

哈喽！大家好，我是【IT邦德】，江湖人称jeames007，10余年DBA及大数据工作经验
一位上进心十足的【大数据领域博主】！
中国DBA联盟(ACDU)成员，目前服务于工业互联网
擅长主流Oracle、MySQL、PG、高斯及Greenplum运维开发，备份恢复，安装迁移，性能优化、故障应急处理等。
✨ 如果有对【数据库】感兴趣的【小可爱】，欢迎关注【IT邦德】
❤️❤️❤️感谢各位大可爱小可爱！❤️❤️❤️

前言

Greenplum版本升级的详细过程，此博客做了详细的介绍。

---

1.升级简介

greenplum 升级分两种：

大版本升级：跨越主版本的升级，比如 4.3 or 5 升级到 6，6升级到7
小版本升级：主版本内的升级，比如6.x to 6.newer

✨ 1.1 大版本升级

Greenplum 从5.28版本开始可以直接使用gpupgrade工具升级到6.9及以后的版本
https://github.com/greenplum-db/gpupgrade

gpupgrade 是一个用于 Greenplum 数据库版本升级的工具，它提供了一种简单的方法来将 Greenplum 数据库集群从一个主要版本升级到另一个主要版本。
对于GP4.3版本的数据库，如果升级到6，需要通过备份恢复的方式，可以使用gpbackup、gprestore或gpcopy 工具来实现，这种方式同样适用于GP5 to GP6

以gpcopy命令，以4.3 升级到 6 为例，升级流程如下：
1）对原集群做好备份，并保证备份可用，如果升级失败需要原集群继续提供服务
2）准备GP6集群，保证集群状态可用
3）确保有足够的磁盘空间可以保留备份文件，建议数据库的5倍大小
4）如果磁盘不够，使用gpcopy（4.3.26及以上版本 or 5.9及以上版本）命令，不建议使用–truncate-source-after参数
5）对于GP6不兼容的扩展模块，需要在备份之前或者在restore的时候需要排除相关的表
6）GP6 的系统表、系统视图及函数变化比较大，留好操作日志，便于修复


gpupgrade 的升级流程大致如下：
1）确认环境：在开始升级之前，需要确认源和目标 Greenplum 数据库版本的兼容性，并确保升级环境满足一些先决条件，如空间和内存等资源。
2）预升级检查：运行 gpupgrade 工具的 pre-upgrade 子命令，它会检查是否满足升级所需的各种条件，例如必备软件版本、配置文件设置、备份等。
3）备份源数据库：在升级之前，需要对源数据库进行完整备份，以便在升级过程中发生意外情况时进行还原。
4）执行升级：运行 gpupgrade 工具的 upgrade 子命令，它会执行升级过程，包括关闭源数据库、备份源数据库、替换二进制文件、启动目标数据库等。
5）后升级检查：在升级完成后，运行 gpupgrade 工具的 post-upgrade 子命令，它会检查目标数据库是否成功启动，升级是否完成，并验证新版本的 Greenplum 数据库是否工作正常。
6）完成升级：在确认升级成功后，删除源数据库备份，清理 gpupgrade 工具的工作目录以及其他不必要的文件。

✨ 1.2 小版本升级

小版本升级步骤很简单，如下：
1、各个主机下载最新版本的安装包
2、master执行gpcheckcat -A检查所有库
3、master停库gpstop -a -M fast
4、直接安装新版本软件包：rpm -U open-source-greenplum-db-XXXXXX-rhel7-x86_64.rpm --nodeps
5、启库 gpstart -a
6、检查：gpstate -i

2. 6.23升级到6.24

✨ 2.1 升级前检查

[gpadmin@master ~]$ gpcheckcat -A

Usage: gpcheckcat [<option>] [dbname]

    -? | --help              : help menu
    -B parallel              : number of worker threads
    -g dir                   : generate SQL to rectify catalog corruption, put it in dir
    -p port                  : DB port number
    -P passwd                : DB password
    -U uname                 : DB User Name
    -v                       : verbose
    -A                       : all databases
    -S option                : shared table options (none, only)
    -O                       : Online
    -l                       : list all tests
    -R test | 'test1, test2' : run this particular test(s) (quoted, comma seperated list for multiple tests)
    -s test | 'test1, test2' : skip this particular test(s) (quoted, comma seperated list for multiple tests)
    -C catname               : run cross consistency, FK and ACL tests for this catalog table

    Test subset options are mutually exclusive, use only one of '-R', '-s', or '-C'.

[gpadmin@master ~]$ gpcheckcat -p5432 -A
Truncated batch size to number of primaries: 3

Connected as user 'gpadmin' to database 'postgres', port '5432', gpdb version '6.23'
-------------------------------------------------------------------
Batch size: 3
Performing test 'unique_index_violation'
Total runtime for test 'unique_index_violation': 0:00:01.84
Performing test 'duplicate'
Total runtime for test 'duplicate': 0:00:01.62
Performing test 'missing_extraneous'
Total runtime for test 'missing_extraneous': 0:00:00.57
Performing test 'inconsistent'
Total runtime for test 'inconsistent': 0:00:01.59
Performing test 'foreign_key'
Total runtime for test 'foreign_key': 0:00:01.27
Performing test 'pgclass'
Total runtime for test 'pgclass': 0:00:00.06
Performing test 'namespace'
Total runtime for test 'namespace': 0:00:00.00
Performing test 'distribution_policy'
Total runtime for test 'distribution_policy': 0:00:00.00
Performing test 'dependency'
Total runtime for test 'dependency': 0:00:03.45
Performing test 'part_integrity'
Total runtime for test 'part_integrity': 0:00:00.05
Performing test 'part_constraint'
Total runtime for test 'part_constraint': 0:00:00.15
Performing test 'orphaned_toast_tables'
Total runtime for test 'orphaned_toast_tables': 0:00:00.77
Performing test 'aoseg_table'
Total runtime for test 'aoseg_table': 0:00:00.00

SUMMARY REPORT
===================================================================
Completed 13 test(s) on database 'postgres' at 2023-06-04 19:06:05 with elapsed time 0:00:11
Found no catalog issue


Connected as user 'gpadmin' to database 'rptdb', port '5432', gpdb version '6.23'
-------------------------------------------------------------------
Batch size: 3
Performing test 'unique_index_violation'
Total runtime for test 'unique_index_violation': 0:00:03.71
Performing test 'duplicate'
Total runtime for test 'duplicate': 0:00:02.73
Performing test 'missing_extraneous'
Total runtime for test 'missing_extraneous': 0:00:00.51
Performing test 'inconsistent'
Total runtime for test 'inconsistent': 0:00:01.36
Performing test 'foreign_key'
Total runtime for test 'foreign_key': 0:00:01.28
Performing test 'pgclass'
Total runtime for test 'pgclass': 0:00:00.04
Performing test 'namespace'
Total runtime for test 'namespace': 0:00:00.00
Performing test 'distribution_policy'
Total runtime for test 'distribution_policy': 0:00:00.00
Performing test 'dependency'
Total runtime for test 'dependency': 0:00:03.55
Performing test 'part_integrity'
Total runtime for test 'part_integrity': 0:00:00.02
Performing test 'part_constraint'
Total runtime for test 'part_constraint': 0:00:00.10
Performing test 'orphaned_toast_tables'
Total runtime for test 'orphaned_toast_tables': 0:00:00.07
Performing test 'aoseg_table'
Total runtime for test 'aoseg_table': 0:00:00.00

SUMMARY REPORT
===================================================================
Completed 13 test(s) on database 'rptdb' at 2023-06-04 19:06:20 with elapsed time 0:00:13
Found no catalog issue


Connected as user 'gpadmin' to database 'template1', port '5432', gpdb version '6.23'
-------------------------------------------------------------------
Batch size: 3
Performing test 'unique_index_violation'
Total runtime for test 'unique_index_violation': 0:00:03.95
Performing test 'duplicate'
Total runtime for test 'duplicate': 0:00:03.16
Performing test 'missing_extraneous'
Total runtime for test 'missing_extraneous': 0:00:00.46
Performing test 'inconsistent'
Total runtime for test 'inconsistent': 0:00:01.86
Performing test 'foreign_key'
Total runtime for test 'foreign_key': 0:00:01.27
Performing test 'pgclass'
Total runtime for test 'pgclass': 0:00:00.02
Performing test 'namespace'
Total runtime for test 'namespace': 0:00:00.00
Performing test 'distribution_policy'
Total runtime for test 'distribution_policy': 0:00:00.00
Performing test 'dependency'
Total runtime for test 'dependency': 0:00:01.98
Performing test 'part_integrity'
Total runtime for test 'part_integrity': 0:00:00.02
Performing test 'part_constraint'
Total runtime for test 'part_constraint': 0:00:00.11
Performing test 'orphaned_toast_tables'
Total runtime for test 'orphaned_toast_tables': 0:00:00.56
Performing test 'aoseg_table'
Total runtime for test 'aoseg_table': 0:00:00.00

SUMMARY REPORT
===================================================================
Completed 13 test(s) on database 'template1' at 2023-06-04 19:06:34 with elapsed time 0:00:13
Found no catalog issue

✨ 2.2 所有节点软件升级

[root@master opt]# cd /usr/local/
[root@master local]# ll
total 0
drwxr-xr-x.  2 root    root      6 Apr 11  2018 bin
drwxr-xr-x.  2 root    root      6 Apr 11  2018 etc
drwxr-xr-x.  2 root    root      6 Apr 11  2018 games
lrwxrwxrwx   1 gpadmin gpadmin  30 Jun  4 12:08 greenplum-db -> /usr/local/greenplum-db-6.23.1
drwxr-xr-x  11 gpadmin gpadmin 238 Jun  4 12:08 greenplum-db-6.23.1
drwxr-xr-x.  2 root    root      6 Apr 11  2018 include
drwxr-xr-x.  2 root    root      6 Apr 11  2018 lib
drwxr-xr-x.  2 root    root      6 Apr 11  2018 lib64
drwxr-xr-x.  2 root    root      6 Apr 11  2018 libexec
drwxr-xr-x.  2 root    root      6 Apr 11  2018 sbin
drwxr-xr-x.  5 root    root     49 Jun  4 09:11 share
drwxr-xr-x.  2 root    root      6 Apr 11  2018 src


##关闭集群
[root@master ~]# su - gpadmin
[gpadmin@master ~]$ gpstop -a
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Starting gpstop with args: -a
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Gathering information and validating the environment...
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Obtaining Segment details from master...
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4 Open Source'
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='smart'
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Master segment instance directory=/greenplum/data/master/gpseg-1
20230604:19:24:30:024309 gpstop:master:gpadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ...
server shutting down
20230604:19:24:31:024309 gpstop:master:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process
20230604:19:24:31:024309 gpstop:master:gpadmin-[INFO]:-Terminating processes for segment /greenplum/data/master/gpseg-1
20230604:19:24:31:024309 gpstop:master:gpadmin-[INFO]:-No standby master host configured
20230604:19:24:31:024309 gpstop:master:gpadmin-[INFO]:-Targeting dbid [2, 3] for shutdown
20230604:19:24:31:024309 gpstop:master:gpadmin-[INFO]:-Commencing parallel segment instance shutdown, please wait...
20230604:19:24:31:024309 gpstop:master:gpadmin-[INFO]:-0.00% of jobs completed
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-100.00% of jobs completed
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-----------------------------------------------------
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-   Segments stopped successfully      = 2
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-   Segments with errors during stop   = 0
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-----------------------------------------------------
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-Successfully shutdown 2 of 2 segment instances
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-Database successfully shutdown with no errors reported
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-Cleaning up leftover gpmmon process
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-No leftover gpmmon process found
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-Cleaning up leftover gpsmon processes
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-No leftover gpsmon processes on some hosts. not attempting forceful termination on these hosts
20230604:19:24:32:024309 gpstop:master:gpadmin-[INFO]:-Cleaning up leftover shared memory


##安装最新版本的安装包，所有节点均执行
[root@master opt]# rpm -U open-source-greenplum-db-6.24.3-rhel7-x86_64.rpm --nodeps
[root@master opt]# chown -R gpadmin:gpadmin /usr/local/greenplum*

[root@master opt]# cd /usr/local/
[root@master local]# ll
total 0
drwxr-xr-x.  2 root    root      6 Apr 11  2018 bin
drwxr-xr-x.  2 root    root      6 Apr 11  2018 etc
drwxr-xr-x.  2 root    root      6 Apr 11  2018 games
lrwxrwxrwx   1 gpadmin gpadmin  30 Jun  4 19:35 greenplum-db -> /usr/local/greenplum-db-6.24.3
drwxr-xr-x  11 gpadmin gpadmin 238 Jun  4 19:35 greenplum-db-6.24.3
drwxr-xr-x.  2 root    root      6 Apr 11  2018 include
drwxr-xr-x.  2 root    root      6 Apr 11  2018 lib
drwxr-xr-x.  2 root    root      6 Apr 11  2018 lib64
drwxr-xr-x.  2 root    root      6 Apr 11  2018 libexec
drwxr-xr-x.  2 root    root      6 Apr 11  2018 sbin
drwxr-xr-x.  5 root    root     49 Jun  4 09:11 share
drwxr-xr-x.  2 root    root      6 Apr 11  2018 src

3.启动集群

[root@master ~]# su - gpadmin

Last login: Sun Jun  4 19:24:28 CST 2023 on pts/0
[gpadmin@master ~]$ gpstart -a
20230604:19:39:48:029329 gpstart:master:gpadmin-[INFO]:-Starting gpstart with args: -a
20230604:19:39:48:029329 gpstart:master:gpadmin-[INFO]:-Gathering information and validating the environment...
20230604:19:39:48:029329 gpstart:master:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.24.3 build commit:25d3498a400ca5230e81abb94861f23389315213 Open Source'
20230604:19:39:48:029329 gpstart:master:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232'
20230604:19:39:48:029329 gpstart:master:gpadmin-[INFO]:-Starting Master instance in admin mode
20230604:19:39:50:029329 gpstart:master:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20230604:19:39:50:029329 gpstart:master:gpadmin-[INFO]:-Obtaining Segment details from master...
20230604:19:39:50:029329 gpstart:master:gpadmin-[INFO]:-Setting new master era
20230604:19:39:50:029329 gpstart:master:gpadmin-[INFO]:-Master Started...
20230604:19:39:50:029329 gpstart:master:gpadmin-[INFO]:-Shutting down master
20230604:19:39:52:029329 gpstart:master:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait...
......
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-Process results...
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-----------------------------------------------------
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-   Successful segment starts                                            = 2
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-   Failed segment starts                                                = 0
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-   Skipped segment starts (segments are marked down in configuration)   = 0
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-----------------------------------------------------
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-Successfully started 2 of 2 segment instances
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-----------------------------------------------------
20230604:19:39:59:029329 gpstart:master:gpadmin-[INFO]:-Starting Master instance master directory /greenplum/data/master/gpseg-1
20230604:19:40:00:029329 gpstart:master:gpadmin-[INFO]:-Command pg_ctl reports Master master instance active
20230604:19:40:00:029329 gpstart:master:gpadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15
20230604:19:40:00:029329 gpstart:master:gpadmin-[INFO]:-No standby master configured.  skipping...
20230604:19:40:00:029329 gpstart:master:gpadmin-[INFO]:-Database successfully started

##确定版本是否升级成功
[gpadmin@master ~]$ gpstate --version
gpstate version 6.24.3 build commit:25d3498a400ca5230e81abb94861f23389315213 Open Source