lsyncd+rsync同步文件

 

一、概述

一般rsync软件是通过crond这支后台进行（计划任务）来实现自动同步数据，如今已有更好的开源软件来代替使用crond了，那就是lsyncd（ Live Syncing (Mirror) Daemon），它的工作原理：监视本地（rsync client）的目录，当源数据有文件或目录更新时，更新本地文件或目录到远端机器（rsync server），保持实时文件同步，但是它更新数据时需要远端rsync server运行rsync demon

配置环境如下
（1）需要配置rsyncd.conf文件的一端，称为rsync server
（2）不需要配置rsyncd.conf文件的一端，称为rsync client
（3）本次环境如下
rsync服务器：192.168.90.27（安装rsync）
rsync客户端：192.168.90.26（安装rsync+lsyncd）

lsyncd的功能：
（1）开源软件lsyncd采用inotify原理监听某一目录，如果目录内发生增、删、改、利用rsync协议自动同步到多个服务器
（2）inotify，从kernel 2.6.13开始正式并入内核，RHEL5支持
（3）可以本地多点目录的监控并实现到远程目录的同步
（4）在rsync client上通过lsyncd监控并推送数据给rsync server的rsync daemon，rsync server接收lsyncd推送过来的数据，并写入本地磁盘
（5）官方介绍：http://code.google.com/p/lsyncd
注意：而对于那种实时都在变化的数据（例如：数据库），那么这种数据的同步，DRBD技术是一个更好的选择

 

二、软件的安装配置

1、服务器端192.168.90.27

下载安装rsync

 

wget http://rsync.samba.org/ftp/rsync/rsync-3.1.0.tar.gz
tar -zxf rsync-3.1.0.tar.gz
cd rsync-3.1.0
./configure --disable-ipv6 && make && make install

 配置rsync

vi /etc/rsyncd.conf

 

log file=/var/log/rsyncd.log 
pid file=/var/run/rsyncd.pid 
lock file=/var/run/rsync.lock
max connections=8
port=1873
use chroot=no
[web-upload]
uid=bookres
gid=bookres
read only=false
hosts allow=192.168.90.26
path=/usr/local/project/web/filedata/upload/save
read only=false
ignore errors
ignore nonreadable

 

设置开机启动

vi /etc/rc.d/init.d/web-rsync

#! /bin/bash
# chkconfig: - 85 15
# description: Starts and Stops the web-rsync daemon.
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the web-rsync web server

# source function library.
. /etc/rc.d/init.d/functions

status1=$(ps -ef | egrep "rsync --daemon.*--config=/etc/rsyncd.conf" | grep -v 'grep')
pidfile="/var/run/rsyncd.pid"
start_rsync="rsync --daemon --config=/etc/rsyncd.conf"

function start() {
    if [ "${status1}X" == "X" ];then
        rm -f $pidfile
        ${start_rsync}
        status2=$(ps -ef | egrep "rsync --daemon.*--config=/etc/rsyncd.conf" | grep -v 'grep')
        if [  "${status2}X" != "X"  ];then
            echo "rsync service start.......OK"
        fi
    else
        echo "rsync service is running !"
    fi
}

function stop() {
    if [ "${status1}X" != "X" ];then
        kill -9 $(cat $pidfile)
        status2=$(ps -ef | egrep "rsync --daemon.*--config=/etc/rsyncd.conf" | grep -v 'grep')
        if [ "${statusw2}X" == "X" ];then
            echo "rsync service stop.......OK"
        fi
    else
        echo "rsync service is not running !"
    fi
}


function status() {
    if [ "${status1}X" != "X" ];then
        echo "rsync service is running !"
    else
         echo "rsync service is not running !"
    fi
}

function restart() {
    if [ "${status1}X" == "X" ];then
        echo "rsync service is not running..."
        start
    else
        stop
        start
    fi
}

case $1 in
        "start")
               start
                ;;
        "stop")
               stop
                ;;
        "status")
               status
               ;;
        "restart")
               restart
               ;;
        *)
          echo
                echo  "Usage: $0 start|stop|restart|status"
          echo
esac

exit 0

 加到开机启动

chkconfig --add web-rsync
chkconfig web-rsync on

 

启动

service web-rsync start

 

 

2、客户端192.168.90.26

下载安装rsync，不需要配置文件

 

wget http://rsync.samba.org/ftp/rsync/rsync-3.1.0.tar.gz
tar -zxf rsync-3.1.0.tar.gz
cd rsync-3.1.0
./configure --disable-ipv6 && make && make install

 下载安装lsyncd

 

wget https://lsyncd.googlecode.com/files/lsyncd-2.1.5.tar.gz
tar -zxf lsyncd-2.1.5.tar.gz
cd lsyncd-2.1.5
./configure && make && make install

如果编译安装失败，lsyncd依赖lua组件，lua比较难编译安装， 建议使用 yum install lsyncd 安装

 

 配置lsyncd

vi /etc/lsyncd.conf

settings {
    logfile = "/var/log/lsyncd.log",
    statusFile = "/var/log/lsyncd-status.log",
    pidfile = "/var/run/lsyncd.pid",
    statusInterval = 1,
    maxProcesses = 8,
}

sync{
    default.rsync,
    source = "/usr/local/project/web/filedata/upload/save",
    target = "192.168.90.27::web-upload",
	rsync = {
        binary   = "/usr/local/bin/rsync",
        archive  = true,
        compress = true,
        checksum = false,
        ipv6 = false,
        _extra = {"--port=1873"}
    }
}

 设置开机启动

vi /etc/rc.d/init.d/lsyncd

#!/bin/bash
#
# lsyncd: Starts the lsync Daemon
#
# chkconfig: 345 80 30
# description: Lsyncd uses rsync to synchronize local directories with a remote
# machine running rsyncd. Lsyncd watches multiple directories
# trees through inotify. The first step after adding the watches
# is to, rsync all directories with the remote host, and then sync
# single file buy collecting the inotify events.
# processname: lsyncd
# config: /etc/lsyncd.lua
# pidfile: /var/run/lsyncd.pid

# Source function library.
. /etc/init.d/functions

RETVAL=0
PIDFILE="/var/run/lsyncd.pid"
LOCKFILE="/var/lock/subsys/lsyncd"
LSYNCD="/usr/bin/lsyncd"
CONFIG="/etc/lsyncd.conf"
PROG="lsyncd"

start() {
	echo -n "Starting $PROG: "

	if [ -f $PIDFILE ]; then
		PID=`cat $PIDFILE`
		echo $PROG already running: $PID
		exit 1;
	else
		daemon --pidfile=$PIDFILE $LSYNCD -pidfile $PIDFILE $CONFIG
		RETVAL=$?
		echo
		[ $RETVAL -eq 0 ] && touch $LOCKFILE
		return $RETVAL
	fi

}

stop() {
        echo -n "Stopping $PROG: "

        killproc lsyncd
        echo 
        rm -f $LOCKFILE
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status lsyncd
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "Usage:  {start|stop|status|restart}"
        exit 1
        ;;
esac
exit $?

 

修改inotify相关限制配置,因为lsyncd的核心为inotify： 

vi /etc/sysctl.conf

fs.inotify.max_user_instances = 128000
fs.inotify.max_user_watches = 8192000
fs.inotify.max_queued_events = 1638400

保存，使它生效，执行命令

sysctl -p

其中： 
fs.inotify.max_user_instances ：每个用户可创建的 inotify 实例最大上限数量 
fs.inotify.max_user_watches ：每个 inotify 实例可监听的最大上限数量 

fs.inotify.max_queued_events ：对应的 inotify 实例队列可容纳的最大上限事件数量  

 

加到开机启动

chkconfig --add lsyncd
chkconfig lsyncd on

启动

service lsyncd start