马哥Linux第十九周

Q1、简述lvs四种集群特点及使用场景

• lvs-nat：修改请求报文的目标IP,多目标IP的DNAT

  本质是多目标IP的DNAT，通过将请求报文中的目标地址和目标端口修改为某挑出的RS的RIP和PORT实现转发
  （1）RIP和DIP应在同一个IP网络，且应使用私网地址；RS的网关要指向DIP
  （2）请求报文和响应报文都必须经由Director转发，Director易于成为系统瓶颈
  （3）支持端口映射，可修改请求报文的目标PORT
  （4）VS必须是Linux系统，RS可以是任意OS系统

  此集群类型的优点在于RS可支持任意TCP/IP操作系统，且仅需一个合法的公网Ip即可部署。

• lvs-dr：操纵封装新的MAC地址

  LVS-DR：Direct Routing，直接路由，LVS默认模式,应用最广泛,通过为请求报文重新封装一个MAC首部进行转发，源MAC是DIP所在的接口的MAC，目标MAC是某挑选出的RS的RIP所在接口的MAC地址；源IP/PORT，以及目标IP/PORT均保持不变
  （1） Director和各RS都配置有VIP
  （2） 确保前端路由器将目标IP为VIP的请求报文发往Director

  （3）RS的RIP可以使用私网地址，也可以是公网地址；RIP与DIP在同一IP网络；RIP的网关不能指向DIP，以确保响应报文不会经由Director
  （4）RS和Director要在同一个物理网络
  （5）请求报文要经由Director，但响应报文不经由Director，而由RS直接发往Client
  （6）不支持端口映射（端口不能修败）
  （7）RS可使用大多数OS系统

• lvs-tun：在原请求IP报文之外新加一个IP首部

  转发方式：不修改请求报文的IP首部（源IP为CIP，目标IP为VIP），而在原IP报文之外再封装一个IP首部（源IP是DIP，目标IP是RIP），将报文发往挑选出的目标RS；RS直接响应给客户端（源IP是VIP，目标IP是CIP）
  (1) DIP, VIP, RIP都应该是公网地址
  (2) RS的网关一般不能指向DIP
  (3) 请求报文要经由Director，但响应不经由Director
  (4) 不支持端口映射
  (5) RS的OS须支持隧道功能

• lvs-fullnat：修改请求报文的源和目标IP

  lvs-fullnat：通过同时修改请求报文的源IP地址和目标IP地址进行转发
  CIP --> DIP
  VIP --> RIP
  (1) VIP是公网地址，RIP和DIP是私网地址，且通常不在同一IP网络；因此，RIP的网关一般不会指向DIP
  (2) RS收到的请求报文源地址是DIP，因此，只需响应给DIP；但Director还要将其发往Client
  (3) 请求和响应报文都经由Director
  (4) 支持端口映射
  注意：此类型kernel默认不支持

Q2、描述LVS-DR工作原理，并配置实现。

• LVS-DR工作原理：客户端向目标vip发起请求，lvs接收 ，LVS根据负载均衡算法选择一台活跃的的节点，将此节点的ip所在网卡的mac地址作为目标mac地址,发送到局域网里节点在局域网中收到这个帧,拆开后发现目标IP(VIP)与本地匹配,于是处理这个报文.随后直接返回数据给客户端.此时IP包的目标ip是客户端，源ip是自己的vip地址。

  #环境：5台机器，Client:172.16.0.6    Router:172.16.0.7、192.168.37.7、10.0.0.200(2块网卡，一块NAT，一块仅主机) LVS:192.168.37.17、10.0.0.100    RS1:192.168.37.27、10.0.0.100    RS2:192.168.37.37、10.0.0.100
  
  1、router配置
  [root@router ~]# nmcli connection modify eth0 ipv4.addresses 192.168.37.7/24 ipv4.gateway 192.168.37.2 ipv4.method manual
  [root@router ~]# nmcli connection modify eth0 +ipv4.addresses 10.0.0.200/8
  [root@router ~]# nmcli connection modify eth1 ipv4.addresses 172.16.0.7/24 ipv4.method manual
  [root@router ~]# nmcli connection up eth0 && nmcli connection up eth1
  [root@router ~]# echo "eth0 eth1" | xargs -n1 ip address show
  2: eth0:  mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether 00:0c:29:2d:71:b8 brd ff:ff:ff:ff:ff:ff
      inet 192.168.37.7/24 brd 192.168.37.255 scope global noprefixroute eth0
         valid_lft forever preferred_lft forever
      inet 10.0.0.200/8 brd 10.255.255.255 scope global noprefixroute eth0
         valid_lft forever preferred_lft forever
      inet6 fe80::85ff:45b3:efd3:c838/64 scope link noprefixroute 
         valid_lft forever preferred_lft forever
  3: eth1:  mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether 00:0c:29:2d:71:c2 brd ff:ff:ff:ff:ff:ff
      inet 172.16.0.7/24 brd 172.16.0.255 scope global noprefixroute eth1
         valid_lft forever preferred_lft forever
      inet6 fe80::a5dc:4f4a:e48e:f712/64 scope link noprefixroute 
         valid_lft forever preferred_lft forever
  
  2、LVS
  [root@lvs ~]# yum install ipvsadm -y
  [root@lvs ~]# iptables -t mangle -A PREROUTING -d 10.0.0.100 -p tcp --dport 80 -j MARK --set-mark 1
  [root@lvs ~]# ipvsadm -A -f 1 -s wrr
  [root@lvs ~]# ipvsadm -a -f 1 -r 192.168.37.27 -g -w 1
  [root@lvs ~]# ipvsadm -a -f 1 -r 192.168.37.37 -g -w 1
  [root@lvs ~]# ipvsadm -Ln
  IP Virtual Server version 1.2.1 (size=4096)
  Prot LocalAddress:Port Scheduler Flags
    -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
  FWM  1 wrr
    -> 192.168.37.27:0              Route   1      0          0         
    -> 192.168.37.37:0              Route   1      0          0
  [root@lvs ~]# ip address add 10.0.0.100/24 dev eth0 label eth0:0
  [root@lvs ~]# vim /etc/sysctl.conf 
  net.ipv4.ip_forward = 1
  [root@lvs ~]# sysctl -p
  [root@lvs ~]# route del default
  [root@lvs ~]# route add default gw 192.168.37.7
  
  3、配置RS1、RS2（相同部分）
  [root@rs1 ~]# yum install -y  httpd
  [root@rs1 ~]# ip address add 10.0.0.100/32 dev lo label lo:0
  [root@rs1 ~]# vim /etc/sysctl.conf
  net.ipv4.conf.all.arp_ignore = 1
  net.ipv4.conf.all.arp_announce = 2
  net.ipv4.conf.lo.arp_ignore = 1
  net.ipv4.conf.lo.arp_announce = 2
  [root@rs1 ~]# sysctl -p
  [root@rs1 ~]# systemctl start httpd
  [root@rs1 ~]# route del default
  [root@rs1 ~]# route add default gw 192.168.37.7
  [root@rs1 ~]# echo RS1 > /var/www/html/index.html
  
  #rs2
  [root@rs2 ~]# echo RS2 > /var/www/html/index.html
  
  4、client
  [root@centos6 ~]$ vim /etc/sysconfig/network-scripts/ifcfg-eth0
  DEVICE=eth0
  TYPE=Ethernet
  ONBOOT=yes
  BOOTPROTO=none
  NAME="eth0"
  IPADDR=172.16.0.6
  PREFIX=24
  GATEWAY=172.16.0.7
  [root@centos6 ~]$ service network restart
  root@centos6 ~]$ while :;do curl 10.0.0.100;sleep 0.5;done
  RS1
  RS2
  RS1
  RS2
  RS1
  RS2
  RS1
  RS2
  

Q3、实现LVS+Keepalived高可用。

#环境：6台机器，Client:172.16.0.6  Router:172.16.0.7、192.168.37.7、10.0.0.200(2块网卡，一块NAT，一块仅主机) LVS1:192.168.37.17、10.0.0.100   RS1:192.168.37.27、10.0.0.100    RS2:192.168.37.37、10.0.0.100    LVs2:192.168.37.47

1、LVS配置
#LVS1
[root@lvs ~]# ipvsadm -C
[root@lvs ~]# ssh-keygen
[root@lvs ~]# ssh-copy-id 192.168.37.47
[root@lvs ~]# yum install keepalived httpd -y
[root@lvs ~]# echo "192.168.37.17 lvs" >> /etc/hosts
[root@lvs ~]# echo "192.168.37.47 lvs2" >> /etc/hosts
[root@lvs ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from lvs@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id lvs
   vrrp_garp_interval 0 
   vrrp_gna_interval 0    
   vrrp_iptables
   vrrp_mcast_group4 224.0.0.100
}

vrrp_instance VI_1 {
    state MASTER 
    interface eth0
    virtual_router_id 10
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        10.0.0.100/32 dev eth0 label eth0:0
    }
    track_interface {
        eth0   
    }   
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 10.0.0.100 80 {      
    delay_loop 5                        
    lb_algo wrr                         
    lb_kind DR                          
    persistence_timeout 50              
    protocol TCP        
    sorry_server 127.0.0.1 80
    
    real_server 192.168.37.27 80 {
        weight 1
        HTTP_GET {
            url {
              path /                    
              status_code 200           
            }
            connect_timeout 1           
            nb_get_retry 3              
            delay_before_retry 3
        }
    }

    real_server 192.168.37.37 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 1
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@lvs ~]# vim /etc/keepalived/notify.sh 
#!/bin/bash
contact='root@localhost'
notify() {
    mailsubject="$(hostname) to be $1, vip floating"
    mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
    echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
    master)
        notify master
    ;;
    backup)
        notify backup
    ;;
    fault)
        notify fault
    ;;
    *)
        echo "Usage: $(basename $0) {master|backup|fault}"
        exit 1
    ;;
esac
[root@lvs ~]# scp /etc/keepalived/notify.sh lvs2:/etc/keepalived/  
[root@lvs ~]# scp /etc/keepalived/keepalived.conf lvs2:/etc/keepalived/
[root@lvs ~]# echo "Sorry Server 1" > /var/www/html/index.html

#LVS2
[root@lvs2 ~]# route del default
[root@lvs2 ~]# route add default gw 192.168.37.7
[root@lvs2 ~]# ssh-keygen
[root@lvs2 ~]# ssh-copy-id 192.168.37.37
[root@lvs2 ~]# yum install keepalived httpd -y
[root@lvs2 ~]# echo "192.168.37.17 lvs" >> /etc/hosts
[root@lvs2 ~]# echo "192.168.37.47 lvs2" >> /etc/hosts
[root@lvs2 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
    ...
    router_id lvs2
    ...
}
vrrp_instance VI_1 {
    state BACKUP
    priority 80
    ...
}
[root@lvs ~]# echo "Sorry Server 2" > /var/www/html/index.html

#LV1、LV2启动keepalived
systemctl start keepalived httpd

2、配置RS1、RS2（相同部分）
[root@rs1 ~]# yum install -y  httpd
[root@rs1 ~]# ip address add 10.0.0.100/32 dev lo label lo:0
[root@rs1 ~]# vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
[root@rs1 ~]# sysctl -p
[root@rs1 ~]# systemctl start httpd
[root@rs1 ~]# route del default
[root@rs1 ~]# route add default gw 192.168.37.7
[root@rs1 ~]# echo RS1 > /var/www/html/index.html

#rs2
[root@rs2 ~]# echo RS2 > /var/www/html/index.html

3、client测试
[root@centos6 ~]$ while :;do curl 10.0.0.100;sleep 0.5;done

#RS1、RS2分别停止服务
systemctl stop httpd

#lvs停止服务查看sorry情况
systemctl stop keepalived