系统名称 | 版本 | 备注 |
---|---|---|
RHEL | 7.x | 7.3之前版本会有cgroup 内核错误不建议采用(大并发会导致数据库挂起) |
RHEL | 6.x | 如使用RHEL6系列,请将内核升级至2.6.32-696以上 |
Centos | 7.x | 7.3之前版本会有cgroup 内核错误不建议采用(大并发会导致数据库挂起) |
Centos | 6.x | 如使用Centos6系列,请将内核升级至2.6.32-696以上 |
Ubuntu | 18.04 TLS |
项目 | 要求 |
---|---|
最低CPU | 任何x86_64兼容的CPU |
最小内存 | 16GB |
磁盘要求 | 1.单个主机150M用于Greenplum安装 2.每个Segment实例大约300M元数据 |
网络要求 | 10千兆网络,多个接口进行NIC绑定,greenplum支持ipv6 和ipv4 |
文件系统 | 只支持XFS |
主机名 | IP地址 | CPU | 内存 | 磁盘 | 系统 |
---|---|---|---|---|---|
master | 10.10.10.177 | vCpu8 | 8G | 250G | Centos7.3 |
segment1 | 10.10.10.183 | vCpu8 | 8G | 250G | Centos7.3 |
segment2 | 10.10.10.184 | vCpu8 | 8G | 250G | Centos7.3 |
[root@master ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
[root@master ~]# systemctl stop firewalld
[root@master ~]# systemctl disable firewalld
[root@master ~]# setenforce 0
[root@master ~]# reboot
[root@segment1 ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
[root@segment1 ~]# systemctl stop firewalld
[root@segment1 ~]# systemctl disable firewalld
[root@segment1 ~]# setenforce 0
[root@segment1 ~]# reboot
[root@segment2 ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
[root@segment2 ~]# systemctl stop firewalld
[root@segment2 ~]# systemctl disable firewalld
[root@segment2 ~]# reboot
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
[root@master ~]# cat /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
#
# kernel.shmall = _PHYS_PAGES / 2 # See Shared Memory Pages
kernel.shmall = 1001395
# kernel.shmmax = kernel.shmall * PAGE_SIZE
kernel.shmmax = 4101713920
kernel.shmmni = 4096
vm.overcommit_memory = 2
# See Segment Host Memory
vm.overcommit_ratio = 95
# See Segment Host Memory
#
net.ipv4.ip_local_port_range = 10000 65535
# See Port Settings
kernel.sem = 500 2048000 200 40960
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.swappiness = 10
vm.zone_reclaim_mode = 0
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.dirty_background_ratio = 3
# See System Memory
vm.dirty_ratio = 10
# vm.dirty_background_bytes = 1610612736
# vm.dirty_bytes = 4294967296
使用以下命令计算是你主机得参数值
kernerl.shmall : echo $(expr $(getconf _PHYS_PAGES) / 2)
kernel.shmmax : echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE))
对于64G内存以上得主机建议以下参数
vm.dirty_background_ratio = 0
vm.dirty_ratio = 0
vm.dirty_background_bytes = 1610612736
# 1.5GB
vm.dirty_bytes = 4294967296
# 4GB
执行次命令设置 vm.min_free_kbytes (大内存时使用)
awk 'BEGIN {OFMT = "%.0f";} /MemTotal/ {print "vm.min_free_kbytes =", $2 * .03;}' /proc/meminfo >> /etc/sysctl.conf
64G内存以下得主机建议以下参数
移除以下参数
vm.dirty_background_bytes
vm.dirty_bytes
设置以下得参数
vm.dirty_background_ratio = 3
vm.dirty_ratio = 10
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
在 /etc/security/limits.d/20-nproc.conf文件后追加如下内容
* soft nofile 524288
* hard nofile 524288
* soft nproc 131072
* hard nproc 131072
效果如下
[root@master ~]# cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 4096
root soft nproc unlimited
* soft nofile 524288
* hard nofile 524288
* soft nproc 131072
* hard nproc 131072
使用 /etc/security/limits.d/20-nproc.conf 覆盖/etc/security/limits.conf
[root@master ~]# cp /etc/security/limits.d/20-nproc.conf /etc/security/limits.conf
cp: overwrite ‘/etc/security/limits.conf’? y
重启ssh会话执行如下命令 查看返回是否为131072
[root@master ~]# ulimit -u
131072
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
centos或RHEL挂载使用如下参数
rw,nodev,noatime,nobarrier,inode64
Ubuntu参数如下
rw,nodev,noatime,inode64
Centos /etc/fstab文件示例
/dev/data /data xfs nodev,noatime,nobarrier,inode64 0 0
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
示例
[root@master ~]# /sbin/blockdev --setra 16384 /dev/sdb
将盘符换成自己的
将结果追加进/etc/rc.d/rc.local
[root@master ~]# echo "/sbin/blockdev --setra 16384 /dev/sdb" >> /etc/rc.d/rc.local
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
[root@master ~]# grubby --update-kernel = ALL --args =“ elevator = deadline”
[root@master ~]# grubby --info=ALL
index=0
kernel=/boot/vmlinuz-3.10.0-327.el7.x86_64
args="ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8 transparent_hugepage=never"
root=/dev/mapper/centos-root
initrd=/boot/initramfs-3.10.0-327.el7.x86_64.img
title=CentOS Linux (3.10.0-327.el7.x86_64) 7 (Core)
index=1
kernel=/boot/vmlinuz-0-rescue-a603815b1a9344b2801a77066196d5bb
args="ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet transparent_hugepage=never"
root=/dev/mapper/centos-root
initrd=/boot/initramfs-0-rescue-a603815b1a9344b2801a77066196d5bb.img
title=CentOS Linux (0-rescue-a603815b1a9344b2801a77066196d5bb) 7 (Core)
index=2
non linux entry
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
[root@master ~]# grubby --update-kernel=ALL --args="transparent_hugepage=never"
Master Host执行
禁用删除IPC。在以下位置设置此参数 /etc/systemd/logind.conf 在Greenplum数据库主机系统上。
更改/etc/systemd/logind.con
将RemoveIPC=yes改为RemoveIPC=no
修改后执行下面命令
[root@master ~]# systemctl restart systemd-logind
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
从/etc/ssh/sshd_config后/etc/sshd_config中修改如下参数
MaxStartups 200
MaxSessions 200如果MaxStartups使用start:rate:full语法将值修改为如下
Max Startups 10:30:200
MaxSessions 200
[root@master ~]# systemctl restart sshd
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
自己根据环境设置NTP校时
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
[root@master ~] groupadd gpadmin
[root@master ~] useradd gpadmin -r -m -g gpadmin
[root@master ~] passwd gpadmin
[root@master ~] su gpadmin
[root@master ~] ssh-keygen -t rsa -b 4096 ##直接回车跳过所有
授予gpadmin sudo 权限
修改/etc/sudoers
移除%wheel ALL=(ALL) NOPASSWD: ALL得注释
[root@master ~] usermod -aG wheel gpadmin
注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
将上面所属得主机名加入/etc/hosts
主机将主机名改成和hosts文件中对应得名字
[root@master ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.10.183 segment1 segment1-1
10.10.10.184 segment2 segment2-1
10.10.10.177 master
[root@master ~]# reboot
注意每台主机都要用gpadmin用户设置
注意每台主机都要gpadmin用户设置
注意每台主机都要gpadmin用户设置
[gpadmin@master ~]$ cd
[gpadmin@master ~]$ wget http://47.94.143.103/greenplum-db-6.4.0-rhel7-x86_64.rpm
#等待下载完成
[gpadmin@master ~]$ sudo yum install ./greenplum-db-6.4.0-rhel7-x86_64.rpm
[gpadmin@master ~]$ sudo chown -R gpadmin:gpadmin /usr/local/greenplum*
*Master Host gpadmin用户操作
[gpadmin@master ~]$ source /usr/local/greenplum-db-<version>/greenplum_path.sh
[gpadmin@master ~]$ ssh-copy-id segment1
[gpadmin@master ~]$ ssh-copy-id segment2
创建一个名为 hostfile_exkeys为计算机配置了Greenplum系统中每个主机(主机,备用主机和段主机)的主机名和主机地址(接口名称)。确保没有空白行或多余的空格。检查 / etc/hosts文件在您的系统上,以获取要用于您的环境的正确主机名。
例如
[gpadmin@master ~]$ cat hostfile_exkeys
master
segment1
segment2
[gpadmin@master ~]$ gpssh-exkeys -f hostfile_exkeys
[STEP 1 of 5] create local ID and authorize on local host
... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped
[STEP 2 of 5] keyscan all hosts and update known_hosts file
[STEP 3 of 5] retrieving credentials from remote hosts
... send to segment1
... send to segment2
[STEP 4 of 5] determine common authentication file content
[STEP 5 of 5] copy authentication files to all remote hosts
... finished key exchange with segment1
... finished key exchange with segment2
[INFO] completed successfully
[gpadmin@master ~]$ echo "source /usr/local/greenplum-db/greenplum_path.sh" >> .bashrc
Master Host gpadmin用户设置
[gpadmin@master ~]$ gpssh -f hostfile_exkeys -e 'ls -l /usr/local/greenplum-db-6.4.0'
[ master] ls -l /usr/local/greenplum-db-6.4.0
[ master] total 160
[ master] drwxr-xr-x 7 root root 4096 Feb 17 12:45 bin
[ master] drwxr-xr-x 3 root root 21 Feb 17 12:45 docs
[ master] drwxr-xr-x 2 root root 58 Feb 17 12:45 etc
[ master] drwxr-xr-x 3 root root 19 Feb 17 12:45 ext
[ master] -rwxr-xr-x 1 root root 783 Feb 17 12:45 greenplum_path.sh
[ master] drwxr-xr-x 4 root root 4096 Feb 17 12:45 include
[ master] drwxr-xr-x 6 root root 4096 Feb 17 12:45 lib
[ master] drwxr-xr-x 2 root root 20 Feb 17 12:45 libexec
[ master] -rw-r--r-- 1 root root 135617 Feb 12 05:02 open_source_license_pivotal_greenplum.txt
[ master] drwxr-xr-x 7 root root 87 Feb 17 12:45 pxf
[ master] drwxr-xr-x 2 root root 4096 Feb 17 12:45 sbin
[ master] drwxr-xr-x 5 root root 49 Feb 17 12:45 share
[segment1] ls -l /usr/local/greenplum-db-6.4.0
[segment1] total 160
[segment1] drwxr-xr-x 7 gpadmin gpadmin 4096 Feb 18 14:05 bin
[segment1] drwxr-xr-x 3 gpadmin gpadmin 21 Feb 18 14:05 docs
[segment1] drwxr-xr-x 2 gpadmin gpadmin 58 Feb 18 14:05 etc
[segment1] drwxr-xr-x 3 gpadmin gpadmin 19 Feb 18 14:05 ext
[segment1] -rwxr-xr-x 1 gpadmin gpadmin 783 Feb 18 14:05 greenplum_path.sh
[segment1] drwxr-xr-x 4 gpadmin gpadmin 4096 Feb 18 14:05 include
[segment1] drwxr-xr-x 6 gpadmin gpadmin 4096 Feb 18 14:05 lib
[segment1] drwxr-xr-x 2 gpadmin gpadmin 20 Feb 18 14:05 libexec
[segment1] -rw-r--r-- 1 gpadmin gpadmin 135617 Feb 12 05:02 open_source_license_pivotal_greenplum.txt
[segment1] drwxr-xr-x 7 gpadmin gpadmin 87 Feb 18 14:05 pxf
[segment1] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 18 14:05 sbin
[segment1] drwxr-xr-x 5 gpadmin gpadmin 49 Feb 18 14:05 share
[segment2] ls -l /usr/local/greenplum-db-6.4.0
[segment2] total 160
[segment2] drwxr-xr-x 7 gpadmin gpadmin 4096 Feb 18 14:05 bin
[segment2] drwxr-xr-x 3 gpadmin gpadmin 21 Feb 18 14:05 docs
[segment2] drwxr-xr-x 2 gpadmin gpadmin 58 Feb 18 14:05 etc
[segment2] drwxr-xr-x 3 gpadmin gpadmin 19 Feb 18 14:05 ext
[segment2] -rwxr-xr-x 1 gpadmin gpadmin 783 Feb 18 14:05 greenplum_path.sh
[segment2] drwxr-xr-x 4 gpadmin gpadmin 4096 Feb 18 14:05 include
[segment2] drwxr-xr-x 6 gpadmin gpadmin 4096 Feb 18 14:05 lib
[segment2] drwxr-xr-x 2 gpadmin gpadmin 20 Feb 18 14:05 libexec
[segment2] -rw-r--r-- 1 gpadmin gpadmin 135617 Feb 12 05:02 open_source_license_pivotal_greenplum.txt
[segment2] drwxr-xr-x 7 gpadmin gpadmin 87 Feb 18 14:05 pxf
[segment2] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 18 14:05 sbin
[segment2] drwxr-xr-x 5 gpadmin gpadmin 49 Feb 18 14:05 share
bin-此目录包含Greenplum数据库管理实用程序。该目录还包含PostgreSQL客户端和服务器程序,其中大多数也在Greenplum数据库中使用。
docs/cli_help-此目录包含Greenplum数据库命令行实用程序的帮助文件。
docs/cli_help/gpconfigs-此目录包含示例 gpinitsystem 在安装和初始化Greenplum数据库系统时可以修改和使用的配置文件和主机文件。
ext —一些Greenplum Database实用程序使用的捆绑程序(例如Python)。
include — Greenplum数据库的C头文件。
lib — Greenplum数据库和PostgreSQL库文件。
sbin —支持/内部脚本和程序。
share — Greenplum数据库的共享文件。
Master Host root用户设置
[root@master ~]# mkdir -p /data/master
[root@master ~]# chown gpadmin:gpadmin /data/master
Master Host root用户设置
创建hostfile_gpssh_segonly文件对于每个段主机,该文件应该只有一台机器配置的主机名。
[root@master gpadmin]# cat hostfile_gpssh_segonly
segment1
segment2
[root@master gpadmin]# source /usr/local/greenplum-db/greenplum_path.sh
[root@master gpadmin]# gpssh -f hostfile_gpssh_segonly -e 'mkdir -p /data/primary'
[root@master gpadmin]# gpssh -f hostfile_gpssh_segonly -e 'mkdir -p /data/mirror'
[root@master gpadmin]# gpssh -f hostfile_gpssh_segonly -e 'chown -R gpadmin /data/*'
每台segment Host root用户设置
[root@segment1 ~]# mkdir /data1
[root@segment1 ~]# mkdir /data2
[root@segment1 ~]# chown gpadmin.gpadmin -R /data1
[root@segment1 ~]# chown gpadmin.gpadmin -R /data2
Master Host root用户设置
创建一个名为的主机文件 hostfile_gpcheckperf每个段主机具有一个主机名。不包括主控主机
[gpadmin@master ~]$ cat hostfile_gpcheckperf
segment1
segment2
[gpadmin@master ~]$ gpcheckperf -f hostfile_gpcheckperf -r ds -D \
-d /data1/primary -d /data2/primary \
-d /data1/mirror -d /data2/mirror
要等待一会才能看到测试结果
Master Host gpadmin用户设置
创建一个名为 hostfile_gpinitsystem。在此文件中,添加段主机接口的主机地址名称,每行一个名称,没有多余的行或空格。
[gpadmin@master ~]$ cat hostfile_gpinitsystem
segment1
segment2
[gpadmin@master ~]$ cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpconfigs/gpinitsystem_config
[gpadmin@master ~]$ cat /home/gpadmin/gpconfigs/gpinitsystem_config
# FILE NAME: gpinitsystem_config
# Configuration file needed by the gpinitsystem
################################################
#### REQUIRED PARAMETERS
################################################
#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"
#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg
#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=6000
### 这里需要和sysctl.conf中net.ipv4.ip_local_port_range对比 是否在ip_local_port_range范围内
#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=master
#### File system location where the master data directory
#### will be created.
MASTER_DIRECTORY=/data/master
#### Port number for the master instance.
MASTER_PORT=5432
#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh
#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8
#### Default server-side character set encoding.
ENCODING=UNICODE
####这里往下都是镜像segment设置
################################################
#### OPTIONAL MIRROR PARAMETERS
################################################
#### Base number by which mirror segment port numbers
#### are calculated.
MIRROR_PORT_BASE=7000
#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)
################################################
#### OTHER OPTIONAL PARAMETERS
################################################
#### Create a database of this name after initialization.
#DATABASE_NAME=name_of_database
#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem
初始化程序
- 运行以下命令,以引用初始化配置文件的路径和文件名(gpinitsystem_config)和主机文件(hostfile_gpinitsystem)。
[gpadmin@master ~]$ gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem
- 对于完全冗余的系统(具有备用主服务器和扩展镜像配置),请包括-s 和 -S选项。例如:
[gpadmin@master ~]$ gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem -s standby_master_hostname -S
安装完成能看到
=> Greenplum Database instance successfully created.
Master Host gpadmin用户设置
[gpadmin@master ~]$ gpconfig -s TimeZone Asia/Shanghai
更改时区后,您必须重新启动Greenplum数据库。命令gpstop -ra重新启动Greenplum数据库。目录视图 pg_timezone_names 提供Greenplum数据库时区信息。
Master Host gpadmin用户设置
[gpadmin@master ~]$ echo "export MASTER_DATA_DIRECTORY=/data/master/gpseg-1" >> .bashrc
[gpadmin@master ~]$ echo "export PGPORT=5432" >> .bashrc
[gpadmin@master ~]$ echo "export PGUSER=gpadmin" >> .bashrc
[gpadmin@master ~]$ echo "export PGDATABASE=default_login_database_name" >> .bashrc
[gpadmin@master ~]$ echo "export LD_PRELOAD=/lib64/libz.so.1 ps" >> .bashrc
[gpadmin@master ~]$ source ~/.bashrc
[gpadmin@master ~]$ psql
psql (9.4.24)
Type "help" for help.
test=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+---------+----------+------------+------------+---------------------
postgres | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin +
| | | | | gpadmin=CTc/gpadmin
template1 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin +
| | | | | gpadmin=CTc/gpadmin
test | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 |
(4 rows)
test=#