Chapter 19 Configuring IPsec for Greenplum Database

Chapter 19 Configuring IPsec forGreenplum Database

本主题介绍如何为Greenplum的数据库集群配置Internet协议安全（IPsec）。

•IPsec概述

•安装Openswan

•配置Openswan连接

IPsec概述

Internet协议安全（IPsec）是一种在IP网络层（OSI第3层）的身份验证协议和加密通信的套件。使用IPsec可以帮助Greenplum系统防止网络攻击，如数据包嗅探，改变网络数据包，身份欺骗和中间人攻击。

Greenplum启用IPSec后，就会在集群的各对主机之间建立虚拟专用网（VPN），或隧道，集群中任两个主机的通信都是经过加密并通过隧道来发送。如果您在集群的N台主机，则每个主机都需要n(n-1)/2个VPN的来连接到其他的主机。你也可以设置IPsec，将网络中的其他主机添加进来。

启用IP流量加密是需要付出网络性能代价的。为了确保在启用IPsec后网络的带宽仍能满足需求，使用gpcheckperf来进行测试。详见Greenplum的数据库实用程序指南与gpcheckperf帮助。如果带宽不能满足数据库负载和性能的需求，你可能需要调整配置或则使用更高带宽的网络媒体。

本节将介绍如何在基于Red Hat or CentOS的Greenplum集群上设置和使用的Openswan，一个非常流行的IPsec实现。

Openswan也为用户提供工具，在Linux上启用IPsec。它利用互联网密钥交换（IKE）协议和X.509证书，握手，交换会话密钥，并使用netlink的API内置到Linux内核的支持IPsec连接。

IKE协议允许在隧道中使用两个对等协商认证和加密算法。会话有两个阶段，在第一阶段，会话双方使用Diffie-Hellman密钥交换来建立一个安全的加密通道，这个阶段必须在第二个阶段前顺利完成。在第二阶段，在IPsec隧道中使用对等体协商认证和加密算法。第二阶段协商的结果是安全关联（securityassociation  SA）。它包含的源，目的地，和的指令。 Linux内核使用SA建立连接。

对等体可以使用下列方法中的一个互相认证：

•RSA公钥加密。每台主机都有一个公钥和私钥。公共密钥分发到集群中的所有主机，这样任何主机可以验证任何其他主机。

•预共享密钥。该方法是最简单的配置，但不如使用RSA公钥加密的安全性。

•X.509证书。所发证书为每个主机由证书颁发机构（CA）。主机是根据由CA赋予信任认证。当许多主机连接到中央网关这是最常用的。

RSA公钥加密是首选的方法。

一个连接有两种连接模式：隧道tunnel或传输transport。在隧道模式下，整个IP数据包都被加密，包括IP报头。在传输模式下，IP报头被泄露。隧道模式是更高的安全性的首选。

下列的资源是推荐的关于IPsec和Openswan的附加信息。

•      Internet Key Exchange (IKE) -维基百科的文章，描述用于建立安全关联IKE协议。

•      Security Association - 维基百科的文章中描述的安全关联的属性和用途。

•      AES instruction set - 维基百科的文章，提供集英特尔高级加密标准（AES）指令的概述，并列出支持它的CPU系列。

•                ipsec.conf(5) - 为ipsec.conf文件配置文件手册页。

•      setkey(8) - 对用于管理Linux内核的安全​​关联数据库（SAD）和安全策略数据库（SPD）的setkey的实用手册页。

•                Openswan -红帽的Openswan包概述;同样也适用于CentOS。

•      Host-to-Host VPN Using Openswan - 红帽指导创建使用的Openswan主机到主机的VPN;同样也适用于CentOS。

安装Openswan

Openswan也可以使用系统上的软件包管理器进行安装，或者从openswan的网站下载安装包进行安装，或者下载源码并近编译安装。

Pivotal推荐你使用Openswan version 2.6.43或更高的版本。如果你的包管理器是较早的版本，则你可以从Openswan官网下载最新的Openswan RPM包进行安装。你也可以从Openswan下载源码。

以下说明假定您在运行64位的Red Hat 6.x或CentOS 6.x的主机上安装的Openswan。

首先，确定是否已经安装的Openswan，如果已安装，是哪个版本：

$ sudo yuminfo installed openswan

如果已经安装了推荐的版本，则从继续Openswan安装的配置和验证。

如果安装了旧版本，在继续之前卸载它：

$ sudo yumremove openswan

Installing Openswan with an RPM

输入以下命令来查看哪些的Openswan的版本是在包库可供选择：

$ sudo yumlist available openswan

如果有推荐的版本，在集群中的每个主机上安装：

$ sudo yuminstall -y openswan

如果推荐的版本是不是在库中，可以从的Openswan网站下载https://download.openswan.org。浏览到 /rhel6/x86_64 的目录来寻找RPM.

采用如下命令来安装下载的RPM:

$ sudo rpm -iopenswanX-version.x86_64.rpm

 

 

Enter thefollowing command to see which version of Openswan is available in the packagerepository:

$ sudo yumlist available openswan

If therecommended version is available, install it on each host in the cluster:

$ sudo yuminstall -y openswan

If therecommended version is not in the repository, you can download it from theOpenswan Web site at https://download.openswan.org. Browse to the /rhel6/x86_64 directoryto find the RPM.

Install thedownloaded RPM with a command like the following:

$ sudo rpm -iopenswanX-^ersio^.x86_64.rpm

从源代码安装的Openswan

如果你不能通过RPM安装Openswan，则你也可以下载源码，编译并安装。

1.从Openswan网站下载Openswan的源码。

2.从压缩包中解压并查看README，以确保必要的软件包安装在您的构建机器。例如：

sudo yuminstall gmp gmp-devel gawk flex bison \

iproute2iptables sed awk bash cut python

3.按照README 文件中的指导来编译Openswan。例如：

$ make programs

$ sudo make install

配置和验证Openswan安装

 

按照本节中的步骤配置每个主机并验证的Openswan安装。

1.编辑/etc/sysctl.conf并修改或添加以下变量：

net.ipv4.conf.default.accept_redirects = 0

net.ipv4.conf.all.accept_redirects = 0

net.ipv4.conf.default.send_redirects = 0

net.ipv4.conf.all.send_redirects = 0

net.ipv4.ip_forward = 1

执行的sysctl -p重新加载该文件。

2.通过执行如下的命令来恢复IPsec的默认SELinux安全上下文：

# restorecon -Rv /etc/ipsec.d

3.如果启用了防火墙，修改它允许IPsec数据包：

•UDP端口500对Internet密钥交换（IKE）协议

•对于UDP IKE NAT穿透端口4500

•对封装安全负载（ESP）IPsec数据包协议50

•（不推荐）协议51认证头（AH）IPsec数据包

以下是IPsec规则为iptables的一个例子：

iptables -A INPUT -p udp --sport 500--dport 500 -j ACCEPT

iptables -A OUTPUT -p udp --sport 500--dport 500 -j ACCEPT

iptables -A INPUT -p udp --sport 4500--dport 4500 -j ACCEPT

iptables -A OUTPUT -p udp --sport 4500--dport 4500 -j ACCEPT

iptables -A INPUT -p 50 -j ACCEPT

iptables -A OUTPUT -p 50 -j ACCEPT

4.编辑/etc/ipsec.conf文件，并进行以下更改：

•将protostack从auto改到 netkey ：

protostack=netkey

•取消注释或添加以下行：

include /etc/ipsec.d/*.conf

这允许您创建和安装的每个主机到主机的隧道单独的配置文件。

5.启动的IPsec的服务命令。

# service start ipsec

6.运行安全检查，检查的IPsec安装。必须安装Python来运行此命令。

$ sudoipsec verify

输出如下所示：

 

Checking ifIPsec got installed and started correctly:

Versioncheck and ipsec on-path [OK]

OpenswanU2.6.43/K2.6.32-504.16.2.el6.x86_64 (netkey)

See `ipsec--copyright' for copyright information.

Checkingfor IPsec support in kernel [OK]

NETKEY:Testing XFRM related proc values

ICMPdefault/send_redirects [OK]

ICMPdefault/accept_redirects [OK]

XFRM larvaldrop [OK]

Hardwarerandom device check [N/A]

Two or moreinterfaces found, checking IP forwarding [OK]

Checkingrp_filter [ENABLED]

/proc/sys/net/ipv4/conf/all/rp_filter[ENABLED]

/proc/sys/net/ipv4/conf/lo/rp_filter[ENABLED]

/proc/sys/net/ipv4/conf/eth0/rp_filter[ENABLED]

/proc/sys/net/ipv4/conf/pan0/rp_filter[ENABLED]

Checkingthat pluto is running [OK]

Plutolistening for IKE on udp 500 [OK]

Plutolistening for IKE on tcp 500 [NOT IMPLEMENTED]

Plutolistening for IKE/NAT-T on udp 4500 [OK]

Plutolistening for IKE/NAT-T on tcp 4500 [NOT IMPLEMENTED]

Plutolistening for IKE on tcp 10000 (cisco) [NOT IMPLEMENTED]

CheckingNAT and MASQUERADEing [TEST INCOMPLETE]

Checking'ip' command [OK]

Checking'iptables' command [OK]

 

注意：检查“IP”命令可结果[IP XFRMBROKEN]，根据iproute的系统上的版本。这可能是造成的iproute版本之间的IP XFRM消息输出变化的误诊。

7.使用如下命令，设置开机启动IPsec：

# chkconfig ipsec on

ConfiguringOpenswan Connections

通过创建一个连接设置在每对集群中的主机之间的IPsec隧道。在每个连接，一台主机被指定为“left”的主机和其他的“right”的主持人。例如，如果你有一个master（MDW），standby master（smdw），三段上的主机（SDW1，sdw2，sdw3），您将需要十个连接，如图所示如下表。

Table 27:IPsec connections for a five-host cluster

Connection Number	Left host	Right host
1	mdw	smdw
2	mdw	sdw1
3	mdw	sdw2
4	mdw	sdw3
5	smdw	sdw1
6	smdw	sdw2
7	smdw	sdw3

 

Connection Number	Left host	Right host
8	sdw1	sdw2
9	sdw1	sdw3
10	sdw2	sdw3

完成这些任务来配置连接：

•                创建主机密钥

•                创建连接配置文件

•                测试IPSec连接

创建主机密钥

要启用隧道RSA公钥认证，每台主机都必须有一个RSA密钥对。在每个主机上使用root用户，输入以下命令来生成认证密钥。

#                 ipsecnewhostkey --output /etc/ipsec.d/ipsec.secrets --bits 4096

密钥保存在/etc/ipsec.d/ipsec.secrets文件及其属性被设置为只允许访问根用户。

要查看主机的公钥，可以使用IPSec的showhostkey命令和--left或  --right选项。

命令输出以适合于粘贴到连接配置文件的格式的公共密钥。

在本实施例中，键被缩短为可读性：

# ipsecshowhostkey --left

  # rsakey AQOW+RwpL

 leftrsasigkey=0sAQOW+RwpLg7CGoyywCnv+vnasGJI7...

# ipsecshowhostkey --right

   # rsakey AQOW+RwpL

 rightrsasigkey=0sAQOW+RwpLg7CGoyywCnv+vnasGJI7...

 

您需要为您配置的每个主机对之间的隧道使用此命令。

创建连接配置文件

IPsec隧道通过在/etc/ipsec.conf文件创建conn section来进行配置。因为我们在/etc/ipsec.conf添加了include/etc/ipsec.d/*.conf，我们可以为每个连接配置一个单独的.conf文件。

请按照以下步骤来配置每对主机的连接。

1.              Log in to thehost that will be on the "left" side of the tunnel.

2.    在/etc/ipsec.d目录下创建一个新的配置文件。配置文件名需要同时包含主机名和.conf的扩展名。下面的配置文件mdw-sdw1.conf, 包含mdw 和 sdw1之间的连接配置:

conn mdw-sdw1

leftid=mdw

left=192.1.2.214

leftrsasigkey=0sAQOW+RwpLg7CGoyywCnv+vnasGJI7... # shortened forreadability

rightid=sdw1

right=192.1.2.215

rightrsasigkey=0sAQNfdDCoDte5bGaGLGkHTKa5GMRl... # shortened forreadability

type=tunnel

authby=rsasig

ike=aes192-sha2;dh20

phase2alg=aes_gcm_c-160-null

auto=start

可用的参数及其缺省值的完整列表，请参阅ipsec.conf文件手册页。例子中的连接名称mdw-sdw1。

对于leftrsasigkey，在"left"主机上运行 ipsecshowhostkey --left。对于rightrsasigkey，在 "right"主机上运行ipsec showhostkey --right命令。

type

Set to tunnel, the default mode.

authby

Set to rsasig. This is more secure than using pre-shared keys (psk).

auto

Set to start so that the tunnel is brought up when IPsec starts up.

ike

The ike parameter is used during phase 1to authenticate the peers and negotiate secure session keys for phase2. The parametervalue is an entry in the format:

cipher-hash;modpgroup,cipher-hash;modpgroup,                                                        ....

•      cipher is anencryption algorithm. AES is more secure than 3DES, which is more secure thanDES. AES has length of 128, 192, or 256 bits. The more bits, the stronger theencryption, but more time is required for computation.

# ipsec auto-status

 

000algorithm IKE hash: id=2, name=OAKLEY_SHA1, hashsize=20

000algorithm IKE hash: id=4, name=OAKLEY_SHA2_256, hashsize=32

000algorithm IKE hash: id=6, name=OAKLEY_SHA2_512, hashsize=64

000algorithm ESP encrypt: id=2, name=ESP_DES, ivlen=8,

keysizemin=64,keysizemax=64

000algorithm ESP encrypt: id=3, name=ESP_3DES, ivlen=8,

keysizemin=192,keysizemax=192

000algorithm ESP encrypt: id=18, name=ESP_AES_GCM_A, ivlen=8,

keysizemin=160,

keysizemax=288000algorithm ESP encrypt: id=19,

name=ESP_AES_GCM_B,ivlen=12, keysizemin=160,

keysizemax=288

000algorithm ESP encrypt: id=20, name=ESP_AES_GCM_C, ivlen=16,

keysizemin=160,

keysizemax=288

 

See theipsec.conf man page for the complete list of available parameters and theirdefault values.

Theconnection name in the example is mdw-sdw1.

For theleftrsasigkey use the output from running ipsec showhostkey --left on the"left" host. For rightrsasigkey use the output from running ipsecshowhostkey --right on the "right" host.

Followingare recommendations for configuring parameters for Greenplum Database IPsecconnections

to obtainthe best security and performance:

type

Set totunnel, the default mode.

authby

Set torsasig. This is more secure than using pre-shared keys (psk).

auto

Set tostart so that the tunnel is brought up when IPsec starts up.

ike

The ikeparameter is used during phase 1 to authenticate the peers and negotiate secure

sessionkeys for phase2. The parameter value is an entry in the format:

cipher-hash;modpgroup,cipher-hash;modpgroup, ....

• cipher is anencryption algorithm. AES is more secure than 3DES, which is more secure thanDES. AES has length of 128, 192, or 256 bits. The more bits, the stronger theencryption, but more time is required for computation.

• hash is thehash algorithm. SHA2 is stronger than SHA1, which is stronger than MD5.

SHA2 isrecommended, but if SHA2 is not supported on the device, use SHA1. SHA2

is a familyof hash functions—SHA-224, SHA-256, SHA-384, SHA-512, SHA-512/224,

SHA-512/256—notall of which are supported by Openswan. To find out which algorithms Openswansupports, run the following command after starting the ipsec service:

# ipsecauto -status

000algorithm IKE hash: id=2, name=OAKLEY_SHA1, hashsize=20

000algorithm IKE hash: id=4, name=OAKLEY_SHA2_256, hashsize=32

000algorithm IKE hash: id=6, name=OAKLEY_SHA2_512, hashsize=64

000algorithm ESP encrypt: id=2, name=ESP_DES, ivlen=8,

keysizemin=64,keysizemax=64

000algorithm ESP encrypt: id=3, name=ESP_3DES, ivlen=8,

keysizemin=192,keysizemax=192

000algorithm ESP encrypt: id=18, name=ESP_AES_GCM_A, ivlen=8,

keysizemin=160,

keysizemax=288000algorithm ESP encrypt: id=19,

name=ESP_AES_GCM_B,ivlen=12, keysizemin=160,

keysizemax=288

000algorithm ESP encrypt: id=20, name=ESP_AES_GCM_C, ivlen=16,

keysizemin=160,

keysizemax=288

See http://en.wikipedia.org/wiki/SHA-2for information about SHA2.

 

• modpgroup isthe Diffie-Hellman group. The peers negotiate a shared secret using theDiffie-Hellman protocol. The Diffie-Hellman group is a set of standardizedparameters the peers agree to use as the basis for their calculations. Thegroups are numbered, and higher numbered groups are more secure (and morecompute-intensive) than lower numbered groups. Avoid the lowest numberedgroups: 1 (modp768), 3(modp1024), and 5 (modp1576), which are not consideredsecure. Choose a higher level

group, suchas dh14, dh15, dh19, dh20, dh21, or dh24.

phase2

Set to esp,the default, to encrypt data. The ah setting creates a connection thatauthenticates, but does not encrypt IP packets.

phase2alg

Thephase2alg parameter specifies algorithms to use for encrypting andauthenticating data. The format and defaults are the same as for the ikeparameter.

The AEScipher and SHA hash algorithm are more secure. For effective use of emerging10-gigabit and 40-gigabit network devices, and to enable high speedcommunication channels, the AES_GCM algorithm is currently the recommended bestoption. To use AES_GCM, verify that the CPU supports the AES_NI instructionset. See AES instruction set for a list of CPUs that support AES_NI.

To see ifthe CPU supports AES-NI, see if the aes flag is set in /proc/cpuinfo:

grep aes/proc/cpuinfo

To see ifAES-N1 has been enabled, search /proc/crypto for the module:

grep module/proc/crypto | sort -u

To see ifthe aesni_intel kernel module is loaded:

/sbin/modinfoaesni_intel

To specifythe AES_GCM algorithm, use the following syntax:

phase2alg=aes_gcm_c-160-null

Openswanrequires adding the salt size (32 bits) to the key size (128, 192, or 256bits). In the example above, "160" is calculated by adding a 128-bitkey size to the 32 bit salt size.

The othervalid values are 224 and 288.

3. Use scpto copy the configuration file to the "right" host. For example:

# scp/etc/ipsec.d/mdw-sdw1.conf sdw1:/etc/ipsec.d/

4. Ensurethat IPsec is started by executing the following command on both the"left" and "right" hosts:

# ipsecservice start

5. Load thetunnel on both left and right hosts with the following command:

# ipsecauto --add mdw-sdw

6. Bring upthe tunnel on both left and right hosts with the following command:

# ipsecauto --up mdw-sdw

 

Test theIPsec Connection

 

To verifyIPsec packets are flowing through a network interface, run the following tcdumpcommand on one

host andthen ping that host from another host.

tcdump -n-i interface_name host hostname

Forexample, run the tcpdump command on sdw1 and then, on mdw, ping sdw2:

# tcpdump-n -i eth0 host mdw

tcpdump:verbose output suppressed, use -v or -vv for full protocol decode

listeningon eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

08:22:10.186743IP 192.168.1.214 > 192.168.1.215: ESP(spi=0xe56f19ea,seq=0x1), length

132

ConfiguringIPsec for Greenplum Database Administrator Guide

201

08:22:10.186808IP 192.168.1.214 > 192.168.1.215: ICMP echo request, id 30987, seq 1,

length 64

08:22:10.186863IP 192.168.1.215 > 192.168.1.214: ESP(spi=0x4e55824c,seq=0x1), length

132

08:22:11.189663IP 192.168.1.214 > 192.168.1.215: ESP(spi=0xe56f19ea,seq=0x2), length

132

08:22:11.189707IP 192.168.1.214 > 192.168.1.215: ICMP echo request, id 30987, seq 2,

length 64

The ESPpackets verify that the IP packets are encrypted and encapsulated.

When youhave connections set up between all of the hosts in the cluster and GreenplumDatabase is

running,you can run the tcpdump command on segment hosts to observe database activityin the IPsec

tunnels.

 

 

•      hash is the hash algorithm. SHA2 is strongerthan SHA1, which is stronger than MD5. SHA2 is recommended, but if SHA2 is notsupported on the device, use SHA1. SHA2 is a family of hash functions-SHA-224,SHA-256, SHA-384, SHA-512, SHA-512/224, SHA-512/256—not all of which aresupported by Openswan. To find out which algorithms Openswan supports, run thefollowing command after starting the ipsec service:

See http://en.wikipedia.org/wiki/SHA-2 for information about SHA2.

• modpgroup is the Diffie-Hellman group.The peers negotiate a shared secret using the Diffie-Hellman protocol. TheDiffie-Hellman group is a set of standardized parameters the peers agree to useas the basis for their calculations. The groups are numbered, and highernumbered groups are more secure (and more compute-intensive) than lowernumbered groups. Avoid the lowest numbered groups: 1 (modp7 68), 3 (modp1024),and 5 (modp1576), which are not considered secure. Choose a higher level group,such as dh14, dh15, dh19, dh20, dh21, or dh24.

phase2

Set to esp,the default, to encrypt data. The ah setting creates a connection thatauthenticates, but does not encrypt IP packets.

phase2alg

The phase2aig parameter specifies algorithms touse for encrypting and authenticating data. The format and defaults are thesame as for the ike parameter.

The AES cipherand SHA hash algorithm are more secure. For effective use of emerging 10-gigabitand 40-gigabit network devices, and to enable high speed communicationchannels, the AES_GCM algorithm is currently the recommended best option. Touse AES_GCM, verify that the CPU supports the AES_NI instruction set. See AES instruction se for a list of CPUs that support AES_NI.

To see if theCPU supports AES-NI, see if the aes flag is set in /proc/cpuinfo:

grep aes/proc/cpuinfo

To see ifAES-N1 has been enab/ed, search /proc/crypto for the module:

grep module/proc/crypto 丨 sort -u

To see if theaesni_intei kernel module is loaded:

/sbin/modinfoaesni_intel

To specify theAES_GCM algorithm, use the following syntax:

phase2alg=aes_gcm_c-160-null

Openswanrequires adding the salt size (32 bits) to the key size (128, 192, or 256bits). In the example above, "160" is calculated by adding a 128-bitkey size to the 32 bit salt size.

The othervalid values are 224 and 288.

3.   Use scp tocopy the configuration file to the "right" host. 例如:

#             scp/etc/ipsec.d/mdw-sdw1.conf sdw1:/etc/ipsec.d/

4.   Ensure thatIPsec is started by executing the following command on both the"left" and "right" hosts:

#             ipsec servicestart

5.   Load thetunnel on both left and right hosts with the following command:

#             ipsec auto --add mdw-sdw

6.   Bring up thetunnel on both left and right hosts with the following command:

#             ipsec auto--up mdw-sdw

Test the IPsec Connection

To verifyIPsec packets are flowing through a network interface, run the following tcdumpcommand on one host and then ping that host from another host.

tcdump -n -iinterface_name host                                                                                                         hostname

例如, run the tcpdump command on sdw1 andthen, on mdw, ping sdw2:

# tcpdump -n-i eth0 host mdw

tcpdump:verbose output suppressed, use -v or -vv for full protocol decode listening onethO, link-type EN10MB (Ethernet), capture size 65535 bytes 08:22:10.186743 IP192.168.1.214 > 192.168.1.215: ESP(spi=oxe56f19ea,seq=ox1), length 132

ICMP echorequest, id 30987, seq 1, ESP(spi=0x4e55824c,seq=0x1), length ESP(spi=oxe56f19ea,seq=ox2), length ICMP echo request, id30987, seq 2,

>192.168.1.215

>192.168.1.214

>192.168.1.215

>192.168.1.215

IP192.168.1.214 IP 192.168.1.215 IP 192.168.1.214 IP 192.168.1.214

08:22:10.186808length 64

08:22:10.186863

132

08:22:11.189663

132

08:22:11.189707length 64

The ESPpackets verify that the IP packets are encrypted and encapsulated.

When you haveconnections set up between all of the hosts in the cluster and GreenplumDatabase is running, you can run the tcpdump command on segment hosts toobserve database activity in the IPsec tunnels.