lvs+ldirectord

原网页

http://www.noblenet.org/evergreenwiki/index.php/Ldirectord_setup

Ldirectord setup

Jump to: navigation, search

Following these directions here:
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster

System          Name            IP
Virtual IP	evergreen	172.29.120.84
Load node 1:	load1		172.29.120.79
Apache node1:	Apache01	172.29.120.83
Apache node2:	Apache02	172.29.120.68
Apache node3:	Apache03	172.29.120.71
Apache node4:	Apache04	172.29.120.74

echo ip_vs_dh >> /etc/modules
echo ip_vs_ftp >> /etc/modules
echo ip_vs >> /etc/modules
echo ip_vs_lblc >> /etc/modules
echo ip_vs_lblcr >> /etc/modules
echo ip_vs_lc >> /etc/modules
echo ip_vs_nq >> /etc/modules
echo ip_vs_rr >> /etc/modules
echo ip_vs_sed >> /etc/modules
echo ip_vs_sh >> /etc/modules
echo ip_vs_wlc >> /etc/modules
echo ip_vs_wrr >> /etc/modules

modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr

2. Install ipvsadm and ldirectord. Get perl-doc also in order to read the ldirectord help pages.

apt-get install ipvsadm
update-rc.d: warning: ipvsadm start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
ipvsadm is not configured to run. Please run dpkg-reconfigure ipvsadm.

apt-get install ldirectord
Setting up ldirectord (1:1.0.3-3.1) ...
No configuration file found, doing nothing. ... (warning).

apt-get install perl-doc

3. Enable packet forwarding on the load balancer in /etc/sysctl.conf. Then load the new setting.

# Enables packet forwarding
net.ipv4.ip_forward = 1

sysctl -p

4. Configure ipvsadm. Running dpkg-reconfigure will create the two files ipvsadm.rules and ipvsadm.

dpkg-reconfigure ipvsadm
Load on boot:  No
Daemon method: none

/etc/ipvsadm.rules
# emtpy rules file for ipvsadm

/etc/default/ipvsadm
AUTO="false"
DAEMON="none"

5. Configure ldirectord.

/etc/ha.d/ldirectord.cf
checktimeout=10
checkinterval=2
autoreload=no
logfile="/var/log/ldirectord.log"
quiescent=yes
virtual=172.29.120.84:80
    real=172.29.120.83:80 gate
    real=172.29.120.68:80 gate
    real=172.29.120.71:80 gate
    real=172.29.120.74:80 gate
    #fallback=127.0.0.1:80 gate
    service=http
    request="ldirector.html"
    receive="Test Page"
    scheduler=wlc
    protocol=tcp
    checktype=negotiate

virtual=172.29.120.84:443
    real=172.29.120.83:443 gate
    real=172.29.120.68:443 gate
    real=172.29.120.71:443 gate
    real=172.29.120.74:443 gate
    #fallback=127.0.0.1:443 gate
    service=http
    request="ldirector.html"
    receive="Test Page"
    scheduler=wlc  #weighted least connection; rr round robin
    protocol=tcp
    checktype=negotiate

Point to this file in /etc/default/ldirectord
CONFIG_FILE=/etc/ha.d/ldirectord.cf

Status of ldirector:

ldirectord ldirectord.cf status

\1 better written as $1 at /usr/sbin/ldirectord line 1252.
\1 better written as $1 at /usr/sbin/ldirectord line 1252.
ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 2629

6. Configure heartbeat. Not using heartbeat. ldirectord typically is started from heartbeat but can also be run from the command line.

7. Start ldirectord.

/etc/init.d/ldirectord start

ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.29.120.84:80 wlc
  -> 172.29.120.68:80             Route   1      0          0
  -> 172.29.120.71:80             Route   1      0          0
  -> 172.29.120.74:80             Route   1      0          0
  -> 172.29.120.83:80             Route   1      0          0
TCP  172.29.120.84:443 wlc
  -> 172.29.120.68:443            Route   0      0          0
  -> 172.29.120.71:443            Route   0      0          0
  -> 172.29.120.74:443            Route   0      0          0
  -> 172.29.120.83:443            Route   0      0          0

The load balancer should list the virtual IP:

ip addr sh eth0

2: eth0:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether 78:2b:cb:11:93:9c brd ff:ff:ff:ff:ff:ff
    inet 172.29.120.79/26 brd 172.29.120.127 scope global eth0

8. Configure the apache nodes to accept requests on the virtual IP 172.29.120.84.

/etc/sysctl.conf
# NOBLE load balancing settings
# Enable configuration of arp_ignore option
net.ipv4.conf.all.arp_ignore = 1

# When an arp request is received on eth0, only respond if that address is
# configured on eth0. In particular, do not respond if the address is
# configured on lo
net.ipv4.conf.eth0.arp_ignore = 1

# Ditto for eth1, add for all ARPing interfaces
#net.ipv4.conf.eth1.arp_ignore = 1


# Enable configuration of arp_announce option
net.ipv4.conf.all.arp_announce = 2

# When making an ARP request sent through eth0 Always use an address that
# is configured on eth0 as the source address of the ARP request.  If this
# is not set, and packets are being sent out eth0 for an address that is on
# lo, and an arp request is required, then the address on lo will be used.
# As the source IP address of arp requests is entered into the ARP cache on
# the destination, it has the effect of announcing this address.  This is
# not desirable in this case as adresses on lo on the real-servers should
# be announced only by the linux-director.
net.ipv4.conf.eth0.arp_announce = 2

# Ditto for eth1, add for all ARPing interfaces
#net.ipv4.conf.eth1.arp_announce = 2

Load the new settings:
sysctl -p

9. Configure /etc/ha.d/haresources (to start ldirectord from heartbeat). Is this necessary?

load1   \
        ldirectord::ldirectord.cf \
        IPaddr2::172.29.120.84/26/eth0 \

10. Configure network interface for the virtual IP on the apache servers.

/etc/network/interfaces
# Load balancing virtual interface
auto lo:0
iface lo:0 inet static
  address 172.29.120.84
  netmask 255.255.255.255
  pre-up sysctl -p > /dev/null

enable the interface:
ifup lo:0

Run ifconfig to verify interface is up:
lo:0      Link encap:Local Loopback
          inet addr:172.29.120.84  Mask:255.255.255.255
          UP LOOPBACK RUNNING  MTU:16436  Metric:1

11. Create /openils/var/web/ldirector.html on the apache servers.

Test Page

12. Test.

Browse to 172.29.120.84 (virtual IP)