lvs fullnat+ECMP【4】后端nginx成功解析真实ip

lvs配置：

[root@lvs-fullnat-one keepalived]# cat keepalived.conf 
! Configuration File for keepalived
global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

local_address_group laddr_g1 {
  #172.16.98.11
  172.16.98.10
}

virtual_server_group shanks1 {
  2.2.2.3 80
}

virtual_server 2.2.2.3 80 {
    delay_loop 6
    lb_algo rr
    lb_kind FNAT
    protocol TCP
    syn_proxy
    laddr_group_name laddr_g1    #local address group
    #alpha
    #omega
    #quorum 1
    #hysteresis 0
    #quorum_up " ip addr add 10.255.255.123/32 dev lo;"    #add
    #quorum_down "ip addr del 10.255.255.123/32 dev lo;"    #del vip
    real_server 172.16.97.30 80 {
        weight 100
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

include /etc/keepalived/hosts/*.conf

[root@lvs-fullnat-one keepalived]# 

说明：

global_defs：这个部分不重要，可以不用填写内容，但不能少这个部分，否则可能出现无法启动的情况。
local_address_group：snat的源地址，这里可以写多个，一个IP能转换65536个session。此地方写的地址，要求在网卡中存在。
virtual_server_group：这里需要将vip和vport都声明。
virtual_server：vip和realserver对应关系配置。

网卡多ip配置方法：

[root@lvs-fullnat-one keepalived]# cat /etc/sysconfig/network-scripts/ifcfg-em2
DEVICE=em2
HWADDR=90:B1:1C:5A:37:4E
TYPE=Ethernet
UUID=2b1947f4-1725-4197-abf5-6c8fed750adc
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.99.11
NETMASK=255.255.255.0
IPADDR1=2.2.2.2
NETMASK1=255.255.255.255
[root@lvs-fullnat-one keepalived]# 

或者在ospfd中，通过命令添加（命令类似于思科命令，详细方法问一下百度）

开启keepalived

service keepalived restart
输入一下命令查看输出信息：

[root@lvs-fullnat-one keepalived]# service keepalived restart
Stopping keepalived:                                       [  OK  ]
Starting keepalived:                                       [  OK  ]
[root@lvs-fullnat-one keepalived]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4194304)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  2.2.2.3:80 rr synproxy
  -> 172.16.97.30:80              FullNat 100    0          0         
[root@lvs-fullnat-one keepalived]# ipvsadm -G
VIP:VPORT            TOTAL    SNAT_IP              CONFLICTS  CONNS     
2.2.2.3:80           1        
                              172.16.98.10         0          0         
[root@lvs-fullnat-one keepalived]# 

server端查看nginx的日志文件

做了三次测试，配别是在
1、后端开启toa模块
2、后端没开启toa模块，且lvs配置local_address_group laddr_g1 {172.16.98.11}
3、后端没开启toa模块，且lvs配置local_address_group laddr_g1 {172.16.98.10}
三种情况下的日志。
在开启toa模块的情况下，正确识别真实ip，没开启toa模块的情况下，识别到了snat后的源地址。
以下是截取的日志文件：

172.16.99.4 - - [30/Aug/2018:16:13:24 +0800] "GET /nginx-logo.png HTTP/1.1" 200 368 "http://2.2.2.3/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134" "-"
172.16.98.11 - - [30/Aug/2018:16:15:39 +0800] "GET /poweredby.png HTTP/1.1" 200 2811 "http://2.2.2.3/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134" "-"
172.16.98.10 - - [30/Aug/2018:16:16:58 +0800] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134" "-"