目录
一、 sersync项目简介与框架设计... 1
1.项目简介... 1
2.基本架构... 2
3.设计简析... 3
4.inotify事件实例... 4
二、sersync安装配置333. 5
1.sersync同步需求逻辑图... 5
2.安装环境准备... 6
2.1系统资源列表... 6
2.2检查系统环境... 6
3.配置同步服务器... 6
3.1slave上部署rsync服务... 6
3.2master上配置rsync客户端... 9
3.3master上开始部署sersync服务... 10
3.4troubleshooting. 30
三、sersync配置文件说明... 31
1.XML配置文件说明... 31
1.1初始化的配置文件... 31
2.filter文件过滤功能... 33
3.debug开启... 34
4.XFS文件系统开关... 34
5.inotify监控参数设定(优化)... 34
6.文件监控与远程同步设置... 34
7.rsync参数配置... 34
8.失败日志脚本配置... 35
9.crontab定期整体同步功能... 35
四、sersync插件基本配置和使用... 35
1.插件相关XML. 35
2.command插件... 36
3.refreshCDN. 36
4.socket插件... 36
5.http插件... 36
项目简介与框架设计
2. 基本架构
3. 设计简析
inotify -- Linux 2.6 内核中的文件系统变化通知机制:http://www.ibm.com/developerworks/cn/linux/l-inotifynew/
安装配置
同步需求逻辑图
当前版本的sersync依赖于rsync进行数据同步,如下图,在同步主服务器master上开启sersync,sersync服务监控配置路径中的文件系统时间变化,然后调用rsync命令把更新的文件同步到目标服务器slave上,因此需要在主服务器上配置sersync,在同步目标服务器slave上配置rsync server(注意,是rsync服务)。从下图也可以看出通过sersync可以实现数据分发,从一点到多点的数据同步。
用户会实时往同步服务器master上写入或更新文件数据,此时需要在同步主服务器master上配置sersync服务,在同步目标服务器s1和s2上配置rsync守护进程,此时在主服务器master上产生的写入或更新的文件会被sersync服务实时同步到多个目标服务器s1和s2等。注意,实际上sersync就是监控本地数据写入或更新事件,然后调用客户端命令,将写入或更新事件对应的文件通过rsync推送到目标服务器如s1和s2等。
2. 安装环境准备
实际实验中未按照上面配置参数进行。
[root@StudyServer[2] ~]# cat /etc/redhat-release #查看系统的版本号
[root@StudyServer[2] ~]# uname -r #查看操作系统内核版本号
[root@StudyServer[2] ~]# uname -m #查看系统是32bit还是64bit
3. 配置同步服务器
①检查rsync版本:rsync --version
[root@StudyServer[2] ~]# rsync --version
rsync version 3.0.6 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
[root@StudyServer[2] ~]#
#rsync server
#create by mangguo2015.03.02
rsyncd.conf start##
uid = root
gid = root
#定义本地的path路径拥有的权限,本地path路径为path = /data0/www/www/,如果没有拥有该权限,则sync无法对该目录进行同步
use chroot = no
max connections = 2000
timeout = 600
pid file =/var/run/rsyncd.pid
lock file =/var/run/rsyncd.lock
log file =/var/log/rsyncd.log
#日志文件
ignore errors
read only = false
#定义成false则客户端可以上传文件,即为客户端可以推送数据,服务端可以拉取数据。如果设置成true,则意味着sync客户端只能被sync服务端拉取,sync客户端无法推送数据到sync服务端。
list = false
hosts allow =192.168.1.0/24
hosts deny =0.0.0.0/24
auth users =rsync_backup
secrets file =/etc/rsync.password
##########################
[www]
comment = wwwby mangguo 2015.03.02
path =/data0/www/www/
##########################
[bbs]
comment = wwwby mangguo 2015.03.02
path = /data0/www/bbs/
##########################
[blog]
comment = www by mangguo 2015.03.02
path = /data0/www/blog/
#定义三个模块即为三个路径。rsync同步模块名为[www][bbs][blog],将同步过来的文件分别放入对应path指定目录path= /data0/www/www/,path = /data0/www/bbs/,path = /data0/www/blog/下。如果有多台目标服务器slave,则每一台都将需要进行类似的rsync配置,配置文件需要更换uid和gid。
上述命令写入到/etc/rsyncd.conf中
[root@StudyServer[2] ~]# echo "rsync_backup:mangguo">/etc/rsync.password
[root@StudyServer[2] ~]# chmod 600 /etc/rsync.password
[root@StudyServer[2] ~]# cat /etc/rsync.password
rsync_backup:mangguo
[root@StudyServer[2] ~]# ll /etc/rsync.password
-rw-------. 1 root root 21 Feb 28 11:53 /etc/rsync.password
[root@StudyServer[2] ~]# rsync --daemon
[root@StudyServer[2] ~]# ps -ef | grep rsync
root 2754 1 0 12:04 ? 00:00:00 rsync --daemon
root 2756 2603 0 12:04 pts/0 00:00:00 grep rsync
[root@StudyServer[2] ~]# netstat -lnt |grep 873
tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN
tcp 0 0 :::873 :::* LISTEN
#检查rsync的默认端口号873是否开启来检测rsync服务是否正常开启
[root@StudyServer[2] ~]# echo "/usr/local/bin/rsync --daemon" >>/etc/rc.local
[root@StudyServer[2] ~]# grep daemon /etc/rc.local
/usr/local/bin/rsync --daemon
[root@StudyServer[2] ~]# pkill rsync
[root@StudyServer[2] ~]# rsync --daemon
[root@StudyServer[2] ~]# ps -ef | grep rsync
root 2780 1 0 12:10 ? 00:00:00 rsync --daemon
root 2782 2603 0 12:10 pts/0 00:00:00 grep rsync
echo "mangguo" >/etc/rsync.password
#设置的密码为mangguo,与3.1.3密码设置中设置的密码相同
chmod 600 /etc/rsync.password
#for check
cat /etc/rsync.password
ll /etc/rsync.password
此步骤非常关键,若测试不成功,则后面的sersync配置好了也不会同步数据
①分别创建待同步数据
mkdir -p /data0/www/www/ /data0/www/bbs/ /data0/www/blog/
touch /data0/www/bbs/bbs.log /data0/www/www/www.log /data0/www/blog/blog.log
tree /data0
②执行同步命令
在手工同步之前先关闭防火墙,否则会导致同步失败。
rsync -avzP /data0/www/www/ [email protected]::www/ --password-file=/etc/rsync.password
rsync-avzP /data0/www/www/ [email protected]::www/ --password-file=/etc/rsync.password
rsync-avzP /data0/www/bbs/ [email protected]::bbs/ --password-file=/etc/rsync.password
rsync -avzP /data0/www/bbs/ [email protected]::bbs/ --password-file=/etc/rsync.password
rsync-avzP /data0/www/blog/ [email protected]::blog/ --password-file=/etc/rsync.password
rsync -avzP /data0/www/blog/ [email protected]::blog/ --password-file=/etc/rsync.password
在后面进行部署sersync之间,sersync主服务器必须要确保手工可以把文件推送到s1和s2上,这样后续sersync才能调用这些命令来自动推送。若目标服务器s1和s2中出现了www.log,bbs.log,blog.log则说明手工推送成功。
执行成功后会提示如下:
[root@StudyServer[1] ~]# rsync -avzP /data0/www/www/ [email protected]::www/ --password-file=/etc/rsync.password
sending incremental file list
./
www.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 81 bytes received 30 bytes 74.00 bytes/sec
total size is 0 speedup is 0.00
[root@StudyServer[1] ~]# rsync -avzP /data0/www/bbs/ [email protected]::bbs/ --password-file=/etc/rsync.password
sending incremental file list
bbs.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 78 bytes received 27 bytes 210.00 bytes/sec
total size is 0 speedup is 0.00
[root@StudyServer[1] ~]# rsync -avzP /data0/www/blog/ [email protected]::blog/ --password-file=/etc/rsync.password
sending incremental file list
sent 39 bytes received 8 bytes 94.00 bytes/sec
total size is 0 speedup is 0.00
在Google code中下载sersync的可执行文件版本,里面含有配置文件与可执行文件,本次试验用sersync2.5进行说明。
mkdir -p/home/mangguo/tools
cd/home/mangguo/tools
wget http://sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz
可通过手工下载:https://code.google.com/p/sersync/downloads
tar zxvfsersync2.5.4_64bit_binary_stable_final.tar.gz -C /usr/local/
cd /usr/local
mv GNU-Linux-x86sersync
#将解压后的tar文件中的GNU-Linux-x86文件重命名为sersync
tree sersync/
cd sersync/
mkdir conf bin logs
mv confxml.xmlconf
mv sersync2bin.sersync
tree .
/bincpconf/confxml.xml conf/confxml.xml.mangguo.$(date+%F)
#备份配置文件
①初始化配置文件
[root@StudyServer[1] sersync]# cat -n conf/confxml.xml
1<?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7<exclude expression="(.*)\.svn"></exclude>
8<exclude expression="(.*)\.gz"></exclude>
9<exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24<localpath watch="/opt/tongbu">
25 <remote ip="127.0.0.1" name="tongbu1"/>
26 <!--<remote ip="192.168.8.39" name="tongbu"/>-->
27 <!--<remote ip="192.168.8.40" name="tongbu"/>-->
28 </localpath>
29 <rsync>
30 <commonParams params="-artuz"/>
31 <auth start="false" users="root" passwordfile="/etc/rsync.pas"/>
32 <userDefinedPort start="false" port="874"/><!-- port=874 -->
33 <timeout start="false" time="100"/><!-- timeout=100 -->
34 <ssh start="false"/>
35 </rsync>
36<failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
37<crontab start="false" schedule="600"><!--600mins-->
38 <crontabfilter start="false">
39<exclude expression="*.php"></exclude>
40<exclude expression="info/*"></exclude>
41 </crontabfilter>
42 </crontab>
43<plugin start="false" name="command"/>
44 </sersync>
45
46 <plugin name="command">
47<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
48 <filter start="false">
49 <include expression="(.*)\.php"/>
50 <include expression="(.*)\.sh"/>
51 </filter>
52 </plugin>
53
54 <plugin name="socket">
55 <localpath watch="/opt/tongbu">
56 <deshost ip="192.168.138.20" port="8009"/>
57 </localpath>
58 </plugin>
59 <plugin name="refreshCDN">
60<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
61 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
62 <sendurl base="http://pic.xoyo.com/cms"/>
63 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
64 </localpath>
65 </plugin>
66 </head>
②更改优化sersync配置
※修改24-28行,原内容如下:
24<localpath watch="/opt/tongbu">
25 <remote ip="127.0.0.1" name="tongbu1"/>
26 <!--<remote ip="192.168.8.39" name="tongbu"/>-->
#定义同步的目的主机,即为slave服务器(rsync服务端),remote ip=“192.168.8.39”表示slave服务器的IP地址,name="tongbu"表示同步到salve服务器的某个模块
27 <!--<remote ip="192.168.8.40" name="tongbu"/>-->
28</localpath>
#这段代码用于定义本地需要同步的目录
修改后的内容如下:
<localpath watch="/data0/www/www">
<remote ip="192.168.1.103" name="www"/>
<remote ip="192.168.1.105" name="www"/>
</localpath>
<!--######################################-->
<localpath watch="/data0/www/bbs">
<remote ip="192.168.1.103" name="bbs"/>
<remote ip="192.168.1.105" name="bbs"/>
</localpath>
<!--######################################-->
<localpath watch="/data0/www/blog">
<remote ip="192.168.1.103" name="blog"/>
<remote ip="192.168.1.105" name="blog"/>
</localpath>
<!--######################################-->
此时watch="/data0/www/www"表示定义服务端待同步的目录,和目标服务器的模块name="www"
※修改原内容的31-34行,认证部分
29 <rsync>
30 <commonParams params="-artuz"/>
#表示同步的参数
31 <auth start="false" users="root" passwordfile="/etc/rsync.pas"/>
#auth start="false"表示不启用认证,配置中auth start="true"表示启用认证,下文的配置中,users="rsync_backup",passwordfile="/etc/rsync.password"表示密码文件位置
32 <userDefinedPort start="false" port="874"/><!-- port=874 -->
33 <timeout start="false" time="100"/><!-- timeout=100 -->
34 <ssh start="false"/>
35 </rsync>
修改后的内容如下:
29 <rsync>
30 <commonParams params="-artuz"/>
31 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
32 <userDefinedPort start="false" port="874"/><!-- port=874 -->
33 <timeout start="true" time="100"/><!-- timeout=100 -->
34 <ssh start="false"/>
35 </rsync>
#该部分内容用于拼接:
rsync -avzP /data0/www/www/ [email protected]::www/ --password-file=/etc/rsync.password这个命令中的参数。
※修改36-37行,原内容为
36<failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
修改后的内容为:
36<failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
当同步失败后,日志记录到/usr/local/sersync/logs/rsync_fail_log.sh,并且没60分钟对失败的logs进行重新同步
③修改后的完整配置文件为:
[root@StudyServer[1] sersync]# cat -n conf/confxml.xml
1<?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7<exclude expression="(.*)\.svn"></exclude>
8<exclude expression="(.*)\.gz"></exclude>
9<exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24 <localpath watch="/data0/www/www">
25 <remote ip="192.168.1.103" name="www"/>
26 <remote ip="192.168.1.105" name="www"/>
27 </localpath>
28 <!--######################################-->
29 <localpath watch="/data0/www/bbs">
30 <remote ip="192.168.1.103" name="bbs"/>
31 <remote ip="192.168.1.105" name="bbs"/>
32 </localpath>
33 <!--######################################-->
34 <localpath watch="/data0/www/blog">
35 <remote ip="192.168.1.103" name="blog"/>
36 <remote ip="192.168.1.105" name="blog"/>
37 </localpath>
38 <!--######################################-->
39 <rsync>
40 <commonParams params="-artuz"/>
41 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
42 <userDefinedPort start="false" port="874"/><!-- port=874 -->
43 <timeout start="true" time="100"/><!-- timeout=100 -->
44 <ssh start="false"/>
45 </rsync>
46 <failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
47<crontab start="false" schedule="600"><!--600mins-->
48 <crontabfilter start="false">
49<exclude expression="*.php"></exclude>
50 <exclude expression="info/*"></exclude>
51 </crontabfilter>
52 </crontab>
53<plugin start="false" name="command"/>
54 </sersync>
55
56 <plugin name="command">
57<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
58 <filter start="false">
59 <include expression="(.*)\.php"/>
60 <include expression="(.*)\.sh"/>
61 </filter>
62 </plugin>
63
64 <plugin name="socket">
65 <localpath watch="/opt/tongbu">
66 <deshost ip="192.168.138.20" port="8009"/>
67 </localpath>
68 </plugin>
69 <plugin name="refreshCDN">
70<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
71 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
72 <sendurl base="http://pic.xoyo.com/cms"/>
73 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
74 </localpath>
75 </plugin>
76 </head>
①配置sersync环境变量
[root@StudyServer[1] sersync]# echo 'export PATH=$PATH:/usr/local/sersync/bin'>>/etc/profile
[root@StudyServer[1] sersync]# tail -1 /etc/profile
export PATH=$PATH:/usr/local/sersync/bin
[root@StudyServer[1] sersync]# source /etc/profile
[root@StudyServer[1] sersync]# which sersync
/usr/local/sersync/bin/sersync
[root@StudyServer[1] sersync]#
②启动命令
[root@StudyServer[1] ~]# sersync -r -d -o /usr/local/sersync/conf/confxml.xml
#以下是启动的工程和结果
set the system param
execute:echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute:echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -r rsync all the local files to the remote servers before the sersync work
option: -d run as a daemon
option: -o config xml name: /usr/local/sersync/conf/confxml.xml
daemon thread num: 10
#默认启动10个线程
parse xml config file
host ip : localhost host port: 8008
#host的IP地址和端口号
daemon start,sersync run behind the console
use rsync password-file :
user is rsync_backup
passwordfile is /etc/rsync.password
#密码文件的用户和对应的密码
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12 = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
#工作的线程数
Max threads numbers is: 32 = 12(Thread pool nums) + 20(Sub threads)
please according your cpu ,use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /data0/www/www && rsync -artuz -R --delete ./ --timeout=100 [email protected]::www--password-file=/etc/rsync.password >/dev/null 2>&1
#启动的命令
run the sersync:
watch path is: /data0/www/www
#问题来了,在最后一行中发现仅能第一个模块的路径可以同步,其他下面的路径不能同步
③同步检查
第一步:在master上的对应目录下创建文件
[root@StudyServer[1] www]# for n in `ls`;do echo 111 >$n/$n.txt;done
[root@StudyServer[1] www]# ls -R
.:
bbs blog www
./bbs:
bbs.log bbs.txt
./blog:
blog.log blog.txt
./www:
www.log www.txt
[root@StudyServer[1] www]#
#可以看到master服务器中的每个目录下都已经创建了对应的*.txt文件
第二步:在slave上检查文件是否被同步到对应的目录下
[root@StudyServer[2] www]# ls -R
.:
bbs blog www
./bbs:
bbs.log
./blog:
./www:
www.log www.txt
[root@StudyServer[2] www]#
[root@StudyServer[3] www]# ls -R
.:
bbs blog www
./bbs:
bbs.log
./blog:
blog.log
./www:
www.log www.txt
[root@StudyServer[3] www]#
#可以看到只有www这个目录同步了在master上创建的文件
针对只能同步一个目录的解决方案:
思路是既然www能够同步成功而且www在/usr/local/sersync/conf/confxml.xml中的同步命令处于第一行,那么可以在/usr/local/sersync/conf/中分别为bbs、www、blog创建不同的confxml.xml文件。具体情况如下:
[root@StudyServer[1] conf]# ll
total 16
-rwxr-xr-x. 1 root root 2298 Mar 4 16:34 bbs_confxml.xml
-rwxr-xr-x. 1 root root 2304 Mar 4 16:35 blog_confxml.xml
-rwxr-xr-x. 1 root root 2300 Mar 4 16:35 www_confxml.xml
#在/usr/local/sersync/conf/下创建了三个不同confxml.xml文件
bbs_confxml.xml内容:
[root@StudyServer[1] conf]# cat -n bbs_confxml.xml
1<?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7<exclude expression="(.*)\.svn"></exclude>
8 <exclude expression="(.*)\.gz"></exclude>
9<exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24 <!--######################################-->
25 <localpath watch="/data0/www/bbs">
26 <remote ip="192.168.1.103" name="bbs"/>
27 <remote ip="192.168.1.105" name="bbs"/>
28 </localpath>
29 <!--######################################-->
30 <rsync>
#删除了www、blog的同步命令部分,留下与bbs相关的同步命令部分
31 <commonParams params="-artuz"/>
32 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
33 <userDefinedPort start="false" port="874"/><!-- port=874 -->
34 <timeout start="true" time="100"/><!-- timeout=100 -->
35 <ssh start="false"/>
36 </rsync>
37<failLog path="/usr/local/sersync/logs/bbs_rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
#相比confixml.xml的不同之处是将rsync_fail_log.sh修改成bbs_rsync_fail_log.sh
38<crontab start="false" schedule="600"><!--600mins-->
39 <crontabfilter start="false">
40<exclude expression="*.php"></exclude>
41<exclude expression="info/*"></exclude>
42 </crontabfilter>
43 </crontab>
44<plugin start="false" name="command"/>
45 </sersync>
46
47 <plugin name="command">
48 <param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
49 <filter start="false">
50 <include expression="(.*)\.php"/>
51 <include expression="(.*)\.sh"/>
52 </filter>
53 </plugin>
54
55 <plugin name="socket">
56 <localpath watch="/opt/tongbu">
57 <deshost ip="192.168.138.20" port="8009"/>
58 </localpath>
59 </plugin>
60 <plugin name="refreshCDN">
61<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
62 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
63 <sendurl base="http://pic.xoyo.com/cms"/>
64 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
65 </localpath>
66 </plugin>
67 </head>
blog_confxml.xml内容:
[root@StudyServer[1] conf]# cat -n blog_confxml.xml
1<?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7<exclude expression="(.*)\.svn"></exclude>
8 <exclude expression="(.*)\.gz"></exclude>
9<exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24 <!--######################################-->
25 <localpath watch="/data0/www/blog">
26 <remote ip="192.168.1.103" name="blog"/>
27 <remote ip="192.168.1.105" name="blog"/>
28 </localpath>
29 <!--######################################-->
30 <rsync>
31 <commonParams params="-artuz"/>
32 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
33 <userDefinedPort start="false" port="874"/><!-- port=874 -->
34 <timeout start="true" time="100"/><!-- timeout=100 -->
35 <ssh start="false"/>
36 </rsync>
37<failLog path="/usr/local/sersync/logs/blog_rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
38<crontab start="false" schedule="600"><!--600mins-->
39 <crontabfilter start="false">
40<exclude expression="*.php"></exclude>
41<exclude expression="info/*"></exclude>
42 </crontabfilter>
43 </crontab>
44<plugin start="false" name="command"/>
45 </sersync>
46
47 <plugin name="command">
48<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
49 <filter start="false">
50 <include expression="(.*)\.php"/>
51 <include expression="(.*)\.sh"/>
52 </filter>
53 </plugin>
54
55 <plugin name="socket">
56 <localpath watch="/opt/tongbu">
57 <deshost ip="192.168.138.20" port="8009"/>
58 </localpath>
59 </plugin>
60 <plugin name="refreshCDN">
61<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
62 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
63 <sendurl base="http://pic.xoyo.com/cms"/>
64 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
65 </localpath>
66 </plugin>
67 </head>
www_confxml.xml内容:
[root@StudyServer[1] conf]# cat -n www_confxml.xml
1 <?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7 <exclude expression="(.*)\.svn"></exclude>
8<exclude expression="(.*)\.gz"></exclude>
9<exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24 <!--######################################-->
25 <localpath watch="/data0/www/www">
26 <remote ip="192.168.1.103" name="www"/>
27 <remote ip="192.168.1.105" name="www"/>
28 </localpath>
29 <!--######################################-->
30 <rsync>
31 <commonParams params="-artuz"/>
32 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
33 <userDefinedPort start="false" port="874"/><!-- port=874 -->
34 <timeout start="true" time="100"/><!-- timeout=100 -->
35 <ssh start="false"/>
36 </rsync>
37<failLog path="/usr/local/sersync/logs/www_rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
38<crontab start="false" schedule="600"><!--600mins-->
39 <crontabfilter start="false">
40<exclude expression="*.php"></exclude>
41<exclude expression="info/*"></exclude>
42 </crontabfilter>
43 </crontab>
44<plugin start="false" name="command"/>
45 </sersync>
46
47 <plugin name="command">
48<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
49 <filter start="false">
50 <include expression="(.*)\.php"/>
51 <include expression="(.*)\.sh"/>
52 </filter>
53 </plugin>
54
55 <plugin name="socket">
56 <localpath watch="/opt/tongbu">
57 <deshost ip="192.168.138.20" port="8009"/>
58 </localpath>
59 </plugin>
60 <plugin name="refreshCDN">
61<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
62 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
63 <sendurl base="http://pic.xoyo.com/cms"/>
64 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
65 </localpath>
66 </plugin>
67 </head>
[root@StudyServer[1] conf]#
④多实例初始化同步命令
※查看启动的sersync进程并将启动的sersync进程关闭
[root@StudyServer[1] conf]# ps -ef | grep sersync
root 7144 1 0 Mar03 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/confxml.xml
root 7169 1 0 Mar03 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/confxml.xml
root 7197 1 0 Mar03 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/confxml.xml
root 7230 1 0 Mar03 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/confxml.xml
root 7309 1 0 Mar03 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/confxml.xml
root 11973 1056 0 18:40 pts/0 00:00:00 grep sersync
[root@StudyServer[1] conf]# pkill sersync
[root@StudyServer[1] conf]# ps -ef | grep sersync
root 11979 1056 0 18:41 pts/0 00:00:00 grep sersync
[root@StudyServer[1] conf]#
※同步命令
sersync -r-d -o /usr/local/sersync/conf/www_confxml.xml
sersync -r -d -o/usr/local/sersync/conf/bbs_confxml.xml
sersync -r -d -o /usr/local/sersync/conf/blog_confxml.xml
#执行结果如下:
[root@StudyServer[1] conf]# sersync -r -d -o /usr/local/sersync/conf/www_confxml.xml
set the system param
execute:echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute:echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -r rsync all the local files to the remote servers before the sersync work
option: -d run as a daemon
option: -o config xml name: /usr/local/sersync/conf/www_confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost host port: 8008
daemon start,sersync run behind the console
use rsync password-file :
user is rsync_backup
passwordfile is /etc/rsync.password
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12 = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
Max threads numbers is: 32 = 12(Thread pool nums) + 20(Sub threads)
please according your cpu ,use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /data0/www/www && rsync -artuz -R --delete ./ --timeout=100 [email protected]::www --password-file=/etc/rsync.password >/dev/null 2>&1
run the sersync:
watch path is: /data0/www/www
[root@StudyServer[1] conf]# sersync -r -d -o /usr/local/sersync/conf/bbs_confxml.xml
set the system param
execute:echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute:echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -r rsync all the local files to the remote servers before the sersync work
option: -d run as a daemon
option: -o config xml name: /usr/local/sersync/conf/bbs_confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost host port: 8008
daemon start,sersync run behind the console
use rsync password-file :
user is rsync_backup
passwordfile is /etc/rsync.password
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12 = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
Max threads numbers is: 32 = 12(Thread pool nums) + 20(Sub threads)
please according your cpu ,use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /data0/www/bbs && rsync -artuz -R --delete ./ --timeout=100 [email protected]::bbs --password-file=/etc/rsync.password >/dev/null 2>&1
run the sersync:
watch path is: /data0/www/bbs
[root@StudyServer[1] conf]# sersync -r -d -o /usr/local/sersync/conf/blog_confxml.xml
set the system param
execute:echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute:echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -r rsync all the local files to the remote servers before the sersync work
option: -d run as a daemon
option: -o config xml name: /usr/local/sersync/conf/blog_confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost host port: 8008
daemon start,sersync run behind the console
use rsync password-file :
user is rsync_backup
passwordfile is /etc/rsync.password
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12 = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
Max threads numbers is: 32 = 12(Thread pool nums) + 20(Sub threads)
please according your cpu ,use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /data0/www/blog && rsync -artuz -R --delete ./ --timeout=100 [email protected]::blog --password-file=/etc/rsync.password >/dev/null 2>&1
run the sersync:
watch path is: /data0/www/blog
[root@StudyServer[1] conf]#
#命令执行成功后查看进程情况:启动了三个进程,每个进程对应一个需要同步的目录
[root@StudyServer[1] conf]# ps -ef | grep sersync
root 12026 1 0 18:50 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/www_confxml.xml
root 12046 1 0 18:50 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/bbs_confxml.xml
root 12069 1 0 18:51 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/blog_confxml.xml
root 12090 1056 0 18:52 pts/0 00:00:00 grep sersync
[root@StudyServer[1] conf]#
#再次同步测试
[root@StudyServer[1] www]# for n in `ls`;do echo 222 >$n/$n$n.txt;done
[root@StudyServer[1] www]# ls -R
.:
bbs blog www
./bbs:
bbsbbs.txt bbs.log bbs.txt
./blog:
blogblog.txt blog.log blog.txt
./www:
www.log www.txt wwwwww.txt
#客户端创建文件
[root@StudyServer[2] www]# ls -R
.:
bbs blog www
./bbs:
bbsbbs.txt bbs.log bbs.txt
./blog:
blogblog.txt blog.log blog.txt
./www:
www.log www.txt wwwwww.txt
#s1查看到了master中同步过来的文件
[root@StudyServer[3] www]# ls -R
.:
bbs blog www
./bbs:
bbsbbs.txt bbs.log bbs.txt
./blog:
blogblog.txt blog.log blog.txt
./www:
www.log www.txt wwwwww.txt
#s2中查看到了master中同步过来的文件
⑤将配置文件放入/etc/rc.local文件中防止重启后配置文件无法启动
/bin/cp/etc/rc.local /etc/rc.local_$(date+%F)
cat>>/etc/rc.local<<EOF
#sync data to 182183
sersync -d -o/usr/local/sersync/conf/www_confxml.xml
sersync-d -o /usr/local/sersync/conf/bbs_confxml.xml
sersync-d -o /usr/local/sersync/conf/blog_confxml.xml
EOF
备注:如果在上述命令的最后一行未加入EOF,则在执行上述除EOF命令后退出时命令执行失效。
[root@StudyServer[1] www]# cat /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
#sync data to 182 183
sersync -d -o /usr/local/sersync/conf/www_confxml.xml
sersync -d -o /usr/local/sersync/conf/bbs_confxml.xml
sersync -d -o /usr/local/sersync/conf/blog_confxml.xml
#查看到了在/etc/rc.local文件中的配置文件
⑥批量同步测试
[root@StudyServer[1] www]# for n in `seq 10000`;do echo 22222 >www/$n.txt;done
#往www这个目录下写10000个文件
[root@StudyServer[1] www]# ps-ef
ot 24950 12026 0 19:58 ? 00:00:00 sh -c cd /data0/www/www && rsync -artuz -R --timeout=100 "./2017.txt" rsync_backup@192
root 24951 12026 0 19:58 ? 00:00:00 sh -c cd /data0/www/www && rsync -artuz -R --timeout=100 "./2017.txt" rsync_backup@192
root 24952 24950 0 19:58 ? 00:00:00 rsync -artuz -R --timeout=100 ./2017.txt [email protected]::www --password-fil
root 24953 24951 0 19:58 ? 00:00:00 rsync -artuz -R --timeout=100 ./2017.txt [email protected]::www --password-fil
#查看master服务器上调用的sersync的进程,master服务器上写的越多,master并发的进程越多,同步到slave的速度受硬件瓶颈、网络瓶颈的因素限制,同步实时性受到影响。本地已经完成了10000个文件的写入,但是master服务器的同步的线程依然在同步。
故障描述:
[root@StudyServer[1]~]# rsync -avzP /data0/www/www/ rsync_backup@#192.168.1.103::www/--password-file=/etc/rsync.password
rsync:getaddrinfo: #192.168.1.103 873: Name or service not known
rsync error: errorin socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
故障原因:
rsync 客户端与sync服务端连接失败,原因是rsync 服务端未启动rsync --daemon
解决方法:
[root@StudyServer[2] ~]# ps -ef | grep rsync
root 1147 1058 0 18:43 pts/0 00:00:00 grep rsync
[root@StudyServer[2] ~]# rsync --daemon #开启rsync --daemon
[root@StudyServer[2] ~]# ps -ef | grep rsync
root 1149 1 0 18:43 ? 00:00:00 rsync --daemon
root 1151 1058 0 18:43 pts/0 00:00:00 grep rsync
[root@StudyServer[2] ~]# netstat -lnt |grep 873
tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN
tcp 0 0 :::873 :::* LISTEN
在rsync 服务端启动rsync daemon解决
故障描述:
[root@StudyServer[1] ~]# rsync -avzP /data0/www/www/ rsync_backup@#192.168.1.103::www/ --password-file=/etc/rsync.password
rsync: getaddrinfo: #192.168.1.103 873: Name or service not known
rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
故障原因:
命令中多了一个#,即为命令书写错误
解决方法:
删除#号即可
配置文件说明
配置文件说明
sersync可选功能是通过xml配置文件来实现,基本配置文件如下
[root@StudyServer[1] sersync]# cat -n conf/confxml.xml
1<?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7<exclude expression="(.*)\.svn"></exclude>
8<exclude expression="(.*)\.gz"></exclude>
9 <exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24 <localpath watch="/data0/www/www">
25 <remote ip="192.168.1.103" name="www"/>
26 <remote ip="192.168.1.105" name="www"/>
27 </localpath>
28 <!--######################################-->
29 <localpath watch="/data0/www/bbs">
30 <remote ip="192.168.1.103" name="bbs"/>
31 <remote ip="192.168.1.105" name="bbs"/>
32 </localpath>
33 <!--######################################-->
34 <localpath watch="/data0/www/blog">
35 <remote ip="192.168.1.103" name="blog"/>
36 <remote ip="192.168.1.105" name="blog"/>
37 </localpath>
38 <!--######################################-->
39 <rsync>
40 <commonParams params="-artuz"/>
41 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
42 <userDefinedPort start="false" port="874"/><!-- port=874 -->
43 <timeout start="true" time="100"/><!-- timeout=100 -->
44 <ssh start="false"/>
45 </rsync>
46 <failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
47<crontab start="false" schedule="600"><!--600mins-->
48 <crontabfilter start="false">
49<exclude expression="*.php"></exclude>
50 <exclude expression="info/*"></exclude>
51 </crontabfilter>
52 </crontab>
53<plugin start="false" name="command"/>
54 </sersync>
55
56 <plugin name="command">
57<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
58 <filter start="false">
59 <include expression="(.*)\.php"/>
60 <include expression="(.*)\.sh"/>
61 </filter>
62 </plugin>
63
64 <plugin name="socket">
65 <localpath watch="/opt/tongbu">
66 <deshost ip="192.168.138.20" port="8009"/>
67 </localpath>
68 </plugin>
69 <plugin name="refreshCDN">
70<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
71 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
72 <sendurl base="http://pic.xoyo.com/cms"/>
73 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
74 </localpath>
75 </plugin>
76 </head>
备注:xml配置文件注释不是"#",而是<!--中间是注释内容-->
3 <host hostip="localhost" port="8008"></host>
hostIP与port是针对插件的保留字段,对于同步功能没有任何作用,保留默认即可。
2. filter文件过滤功能
对于sersync监控的文件,会默认过来系统的临时文件(以"."开头,以"~"结尾),除了这些文件外,可以自定义需要被过滤的文件。
6 <filter start="false">
7<exclude expression="(.*)\.svn"></exclude>
8<exclude expression="(.*)\.gz"></exclude>
9<exclude expression="^info/*"></exclude>
10<exclude expression="^static/*"></exclude>
11 </filter>
自定义需要被过滤的文件的方法:将start设置成true后开启过滤功能,在exclude标签中填写正则表达式,默认给出两个例子分别是过滤以".gz"结尾的文件与过滤监控目录下的info路径(监控路径/info/*),可以根据需要添加,但开启的时候,自己测试是否书写正确,正则表达式出现错误时控制台会提示错误,相比较使用rsync的exclude 功能,被过滤的路径不会加入监控,大大减少rsync同步的通讯量。
3. debug开启
4 <debug start="false"/>
设置成true,开启debug模式,会在sersync正在运行的控制台,打印inotify事件与rsync同步命令。在生产环境中不开启,采用默认的false即可,只有在测试环境中才开启。
4. XFS文件系统开关
5 <fileSystem xfs="false"/>
对于xfs文件系统的用户需要将该选项开启,才能使sersync正常工作
5. inotify监控参数设定(优化)
对于inotify监控参数可以进行设置,根据项目的特点优化sersync
12 <inotify>
13<delete start="true"/>
14 <createFolder start="true"/>
15<createFile start="false"/>
16<closeWrite start="true"/>
17<moveFrom start="true"/>
18<moveTo start="true"/>
19<attrib start="false"/>
20<modify start="false"/>
21 </inotify>
对于大多数应用,把createfile(监控文件事件选项)设置成false来提高性能,减少rsync通讯。因为拷贝文件到监控目录会产生create事件与close_write事件,关闭create事件,只监控文件拷贝结束时的事件close_write,同样可以实现文件完成同步。
将createfolder保持为true。将createfolder设为false,则不会对产生的目录进行监控,该目录下的子文件与子目录同样不会被监控,所以除非特殊需求,一般都开启。默认情况下对创建文件(目录)事件与删除文件或目录都进行监控,如果项目中不需要删除远程目标服务器的文件或目录,可以将delete参数设置为false,则不会对删除事件进行监控。
6. 文件监控与远程同步设置
23 <sersync>
24 <localpath watch="/data0/www/www">
25 <remote ip="192.168.1.103" name="www"/>
26 <remote ip="192.168.1.105" name="www"/>
27 </localpath>
/data0/www/www目录为sersync主服务器本地待同步的目录,ip="192.168.1.103"为从服务器的IP地址,如果有多个从服务器,依次列出来即可。name="www"中的www是rsyncd.conf中的模块名字,即括号中的名称。即为设定需要被同步的目录、同步的目的地、同步的模块。
7. rsync参数配置
39<rsync>
40 <commonParams params="-artuz"/>
41 <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
42 <userDefinedPort start="false" port="874"/><!-- port=874 -->
43 <timeout start="true" time="100"/><!-- timeout=100 -->
44 <ssh start="false"/>
45 </rsync>
在commonparams项,可以自定义rsync的同步参数,默认是-artuz,auth start="true"设置成true时,使用rsync认证模式传送,需要配置user与passwordfile(-password-file=/etc/rsync.pass)来使用。userDefinedPort 当远程同步目标服务器的rsync端口不是默认端口时使用(- port=874 )。timeout设置rsync的timeout时间。 <ssh start="false"/>如果开启,表示ssh使用rsync -e ssh的方式进行传输。
8. 失败日志脚本配置
46<failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
若文件同步失败,会重新传送,再次失败就会写入rsync_fail_log.sh,然后每隔一段时间(timeToExecute进行设置)执行该脚本再次重新发送,然后清空该脚本,可以通过path来设置日志路径。
9. crontab定期整体同步功能
47<crontab start="false" schedule="600"><!--600mins-->
48 <crontabfilter start="false">
49<exclude expression="*.php"></exclude>
50<exclude expression="info/*"></exclude>
51 </crontabfilter>
52 </crontab>
crontab可以对监控路径与远程目标主机每隔一段时间进行一次整体同步,可能由于某些原因两次失败重传都失败了,此时若开启了crontab功能,还可以进一步保证各个服务器文件一致,如果文件量较大,crontab的时间间隔需要设置大一些,否则可能增加通讯开销。schedule参数是设置crontab的时间间隔,默认是600分钟。如果开启filter文件过滤功能,那么crontab整体同步也需要设置过滤,否则实时同步的时候文件被过滤了。但crontab整体同步时,如果不单独设置crontabfilter,还会将过滤的文件同步到远程从服务器,crontab的过滤正则表达式与filter过滤不同,特给出了两个实例分别对应于过滤文件与目录。总是如果同时开启了filter与crontab,则要开启crontab的crontabfilter,并按照示例设置使得与filter的过滤--对应。
10. 插件设置
53<plugin start="false" name="command"/>
当start="false"中的false换成true,将文件同步到远程服务器后会调用name参数设定的插件。可以查看插件设置了解详细信息。
插件基本配置和使用
56 <plugin name="command">
57<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
58 <filter start="false">
59 <include expression="(.*)\.php"/>
60 <include expression="(.*)\.sh"/>
61 </filter>
62 </plugin>
63
64 <plugin name="socket">
65 <localpath watch="/opt/tongbu">
66 <deshost ip="192.168.138.20" port="8009"/>
67 </localpath>
68 </plugin>
69 <plugin name="refreshCDN">
70<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
71 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
72 <sendurl base="http://pic.xoyo.com/cms"/>
73 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
74 </localpath>
75 </plugin>
如上xml所示,<plugin start="false"name="command"/>中false换成true时,在同步文件或路径到远程服务器后,才会调用插件。通过name参数指定需要执行的插件。目前支持的插件有command refreshCDN socket http 四种插件。其中http由于兼容性原因已经去除,以后重新加入。command refreshCDN socket http这些插件可以单独使用(发生文件改变不同步,只调用插件),只需在命令行下使用 -m 参数即可。若需要作为插件与同步程序一起使用,见同步程序说明的插件设置。
2. command插件
56 <plugin name="command">
57<param prefix="/bin/sh" suffix="" ignoreError="true"/><!--prefix /opt/tongbu/mmm.sh suffix-->
58 <filter start="false">
59 <include expression="(.*)\.php"/>
60 <include expression="(.*)\.sh"/>
61 </filter>
62 </plugin>
当文件同步完成后,会调用command插件,如同步文件是test.php,则test.php文件在改动之后,调用rsync同步到远程服务器后,调用command插件,执行:
/bin/shtest.php suffix>/dev/null2>&1
如果suffix设置了,则会放在inotify事件test.php之后,如果ignoreerror为true,则会添加>/dev/null2>&1,当然还可以设置command的filter,当filter为true时,include可以只对正则表达式匹配到的文件调用command
3. refreshCDN
refreshCDN,就在同步过程中将文件发送到目的服务器后刷新CDN接口。如果不想使用,则将start设置为false即可。如果需要使用其他插件,则查看其他Plugin变迁,将插件名称改为xml中其他插件的名称,该模块根据ChinaCDN的协议进行设计,当文件产生时就向CDN接口发送需要刷新的路径位置,刷新模块需要配置的xml文件如下:
69 <plugin name="refreshCDN">
70<localpath watch="/data0/htdocs/cms.xoyo.com/site/">
71 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
72 <sendurl base="http://pic.xoyo.com/cms"/>
73 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
74 </localpath>
内容解析:
4. socket插件
5. http插件