sersync服务
首先准备三台机器
M-server
S1-server
S2-server
1、首先在在S1-server 与 S2-server上部署rsync 服务
配置文件如下:
[root@S1-Server ~]# cat /etc/rsyncd.conf
#rsync_config_______________start
##rsyncd.conf start##
uid = root
gid = root
use chroot = no
max connections = 200
timeout = 600
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 192.168.2.0/24
hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
########################################
[www]
comment = www by zhangjie 2013-06-13
path = /data0/www/www
########################################
[bbs]
comment = www by zhangjie 2013-06-13
path = /data0/www/bbs
########################################
[blog]
comment = www by zhangjie 2013-06-13
path = /data0/www/blog
#rsync_config_______________end
S2-server同上面的配置一样
2、建立rsync同步目录
mkdir /data0/www/{bbs,www,blog}/ -p
检查已建立好的目录
[root@S1-Server ~]# mkdir /data0/www/{bbs,www,blog}/ -p
[root@S1-Server ~]# tree /data0/www/
/data0/www/
|-- bbs
|-- blog
`-- www
3 directories, 0 files
3、配置相关权限认证
[root@S1-Server ~]# cat >/etc/rsync.password<<EOF
> rsync_backup:zhangjie
> EOF
[root@S1-Server ~]# cat /etc/rsync.password
rsync_backup:zhangjie
[root@S1-Server ~]# chmod 600 /etc/rsync.password
[root@S1-Server ~]# ll /etc/rsync.password
-rw------- 1 root root 22 Jun 13 00:19 /etc/rsync.password
4、开启rsync守护进程
[root@S1-Server ~]# rsync --daemon
[root@S1-Server ~]# ps -ef |grep rsync
root 5408 1 0 00:21 ? 00:00:00 rsync --daemon
root 5416 4004 0 00:21 pts/0 00:00:00 grep rsync
[root@S1-Server ~]# lsof -i :873
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
rsync 5408 root 3u IPv6 24598 TCP *:rsync (LISTEN)
rsync 5408 root 5u IPv4 24599 TCP *:rsync (LISTEN)
[root@S1-Server ~]# netstat -lnt |grep 873
tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN
tcp 0 0 :::873 :::* LISTEN
5、设置开机自启动
[root@S1-Server ~]# echo "/usr/bin/rsync --daemon" >/etc/rc.local
[root@S1-Server ~]# tail -1 /etc/rc.local
/usr/bin/rsync --daemon
6、重启的命令
pkill rsync
rsync --daemon
ps -ef |grep rsync
7、在M-server上配置rsync客户端
[root@M-Server ~]# echo "zhangjie" >/etc/rsync.password
[root@M-Server ~]# cat /etc/rsync.password
zhangjie
[root@M-Server ~]# chmod 600 /etc/rsync.password
[root@M-Server ~]# ll /etc/rsync.password
-rw------- 1 root root 9 Jun 13 21:55 /etc/rsync.password
8、在M-server上手动测试rsync同步情况
1)分别创建待同步数据
[root@M-Server ~]# mkdir -p /data0/www/{www,bbs,blog}/
[root@M-Server ~]# touch /data0/www/www/www.log /data0/www/bbs/bbs.log /data0/www/blog/blog.log
[root@M-Server ~]# tree /data0/
/data0/
`-- www
|-- bbs
| `-- bbs.log
|-- blog
| `-- blog.log
`-- www
`-- www.log
4 directories, 3 files
2)执行同步命令
[root@M-Server ~]#
rsync -avzP /data0/www/www/ [email protected]::www/ --password-file=/etc/rsync.password
sending incremental file list
./
www.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 77 bytes received 30 bytes 214.00 bytes/sec
total size is 0 speedup is 0.00
[root@M-Server ~]#
rsync -avzP /data0/www/www/ [email protected]::www/ --password-file=/etc/rsync.password
sending incremental file list
./
www.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 77 bytes received 30 bytes 214.00 bytes/sec
total size is 0 speedup is 0.00
[root@M-Server ~]#
rsync -avzP /data0/www/bbs/ [email protected]::bbs/ --password-file=/etc/rsync.password
sending incremental file list
./
bbs.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 77 bytes received 30 bytes 214.00 bytes/sec
total size is 0 speedup is 0.00
[root@M-Server ~]#
rsync -avzP /data0/www/bbs/ [email protected]::bbs/ --password-file=/etc/rsync.password
sending incremental file list
./
bbs.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 77 bytes received 30 bytes 214.00 bytes/sec
total size is 0 speedup is 0.00
[root@M-Server ~]#
rsync -avzP /data0/www/blog/ [email protected]::blog/ --password-file=/etc/rsync.password
sending incremental file list
./
blog.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 78 bytes received 30 bytes 216.00 bytes/sec
total size is 0 speedup is 0.00
[root@M-Server ~]#
rsync -avzP /data0/www/blog/ [email protected]::blog/ --password-file=/etc/rsync.password
sending incremental file list
./
blog.log
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/2)
sent 78 bytes received 30 bytes 216.00 bytes/sec
total size is 0 speedup is 0.00
3)上S1-server 与S2-server上验证是否推送成功
[root@S1-Server ~]# tree /data0/www/
/data0/www/
|-- bbs
| `-- bbs.log
|-- blog
| `-- blog.log
`-- www
`-- www.log
3 directories, 3 files
成功
[root@S2-Server ~]# tree /data0/www/
/data0/www/
|-- bbs
| `-- bbs.log
|-- blog
| `-- blog.log
`-- www
`-- www.log
3 directories, 3 files
成功
注:要说明一下,要在M-server上安装sersync服务首先要保证在M-server上手动测试推送是否能推送成功,这个很重要,如果没成功,即使你在M-server安装了sersync服务, 也是没有用的
9、开始在M-server上部署sersync服务
1)下载sersync软件(上google下载)
wget
http://sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz
[root@M-Server ~]# wget http://sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz
--2013-06-13 22:34:58-- http://sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz
Resolving sersync.googlecode.com... 173.194.72.82, 2404:6800:4008:c00::52
Connecting to sersync.googlecode.com|173.194.72.82|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://180.184.129.18:80/1Q2W3E4R5T6Y7U8I9O0P1Z2X3C4V5B/sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz [following]
--2013-06-13 22:34:58-- http://180.184.129.18/1Q2W3E4R5T6Y7U8I9O0P1Z2X3C4V5B/sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz
Connecting to 180.184.129.18:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 727290 (710K) [application/octet-stream]
Saving to: `sersync2.5.4_64bit_binary_stable_final.tar.gz'
100%[==============================================================================================================================>] 727,290 89.3K/s in 10s
2013-06-13 22:35:08 (71.3 KB/s) - `sersync2.5.4_64bit_binary_stable_final.tar.gz' saved [727290/727290]
10、安装sersync (解压就能用)
[root@M-Server ~]# tar -zxvf sersync2.5.4_64bit_binary_stable_final.tar.gz -C /usr/local/
GNU-Linux-x86/
GNU-Linux-x86/sersync2
GNU-Linux-x86/confxml.xml
[root@S2-Server ~]# cd /usr/local/
[root@S2-Server local]# mv GNU-Linux-x86 sersync
[root@S2-Server local]# tree sersync/
sersync/
|-- confxml.xml
`-- sersync2
0 directories, 2 files
11、规范sersync目录结构
[root@M-Server local]# cd sersync/
[root@M-Server sersync]# mkdir conf bin logs
[root@M-Server sersync]# mv confxml.xml conf
[root@M-Server sersync]# mv sersync2 bin/sersync
12、配置 sersync
备份
[root@M-Server sersync]# /bin/cp conf/confxml.xml conf/confxml.xml.zhangjie.$(date +%F)
初始的配置文件
[root@M-Server sersync]# cat -n conf/confxml.xml
1 <?xml version="1.0" encoding="ISO-8859-1"?>
2 <head version="2.5">
3 <host hostip="localhost" port="8008"></host>
4 <debug start="false"/>
5 <fileSystem xfs="false"/>
6 <filter start="false">
7 <exclude expression="(.*)\.svn"></exclude>
8 <exclude expression="(.*)\.gz"></exclude>
9 <exclude expression="^info/*"></exclude>
10 <exclude expression="^static/*"></exclude>
11 </filter>
12 <inotify>
13 <delete start="true"/>
14 <createFolder start="true"/>
15 <createFile start="false"/>
16 <closeWrite start="true"/>
17 <moveFrom start="true"/>
18 <moveTo start="true"/>
19 <attrib start="false"/>
20 <modify start="false"/>
21 </inotify>
22
23 <sersync>
24 <localpath watch="/opt/tongbu">
25 <remote ip="127.0.0.1" name="tongbu1"/>
26 <!--<remote ip="192.168.8.39" name="tongbu"/>-->
27 <!--<remote ip="192.168.8.40" name="tongbu"/>-->
28 </localpath>
29 <rsync>
30 <commonParams params="-artuz"/>
31 <auth start="false" users="root" passwordfile="/etc/rsync.pas"/>
32 <userDefinedPort start="false" port="874"/><!-- port=874 -->
33 <timeout start="false" time="100"/><!-- timeout=100 -->
34 <ssh start="false"/>
35 </rsync>
36 <failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
37 <crontab start="false" schedule="600"><!--600mins-->
38 <crontabfilter start="false">
39 <exclude expression="*.php"></exclude>
40 <exclude expression="info/*"></exclude>
41 </crontabfilter>
42 </crontab>
43 <plugin start="false" name="command"/>
44 </sersync>
45
46 <plugin name="command">
47 <param prefix="/bin/sh" suffix="" ignoreError="true"/> <!--prefix /opt/tongbu/mmm.sh suffix-->
48 <filter start="false">
49 <include expression="(.*)\.php"/>
50 <include expression="(.*)\.sh"/>
51 </filter>
52 </plugin>
53
54 <plugin name="socket">
55 <localpath watch="/opt/tongbu">
56 <deshost ip="192.168.138.20" port="8009"/>
57 </localpath>
58 </plugin>
59 <plugin name="refreshCDN">
60 <localpath watch="/data0/htdocs/cms.xoyo.com/site/">
61 <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/>
62 <sendurl base="http://pic.xoyo.com/cms"/>
63 <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/>
64 </localpath>
65 </plugin>
66 </head>
更改优化sersync配置
1)修改24-28行的部分,原内容是
24 <localpath watch="/opt/tongbu">
25 <remote ip="127.0.0.1" name="tongbu1"/>
26 <!--<remote ip="192.168.8.39" name="tongbu"/>-->
27 <!--<remote ip="192.168.8.40" name="tongbu"/>-->
28 </localpath>
修改后内容为:
<localpath watch="/data0/www/www"> #### 同步本地服务器的目录
<remote ip="192.168.2.106" name="www"/> ####同步到远端服务器的目录,www是rsync服务端模块名称
<remote ip="192.168.2.107" name="www"/>
</localpath>
<!--######################################-->
<localpath watch="/data0/www/bbs">
<remote ip="192.168.2.106" name="bbs"/>
<remote ip="192.168.2.107" name="bbs"/>
</localpath>
<!--######################################-->
<localpath watch="/data0/www/blog">
<remote ip="192.168.2.106" name="blog"/>
<remote ip="192.168.2.107" name="blog"/>
</localpath>
<!--######################################-->
2)修改认证部分
修改前内容:
<rsync>
<commonParams params="-artuz"/>
<auth start="false" users="root" passwordfile="/etc/rsync.pas"/>
<userDefinedPort start="false" port="874"/><!-- port=874 -->
<timeout start="false" time="100"/><!-- timeout=100 -->
<ssh start="false"/>
</rsync>
修改后的内容
<rsync>
<commonParams params="-artuz"/> #### -artuz是rsync客户端命令的参数
<auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/> ## true 表示使用认证后面是帐号跟密码
<userDefinedPort start="false" port="874"/><!-- port=874 --> #### false表示假 意思是还是使用默认873端口
<timeout start="true" time="100"/><!-- timeout=100 --> #### true表示真 超时时间为100秒
<ssh start="false"/> ### 表示假 不使用ssh通道
</rsync>
3)继续修改failLog
修改前
<failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
修改后
<failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->
说明:失败之后60分钟会同步一次,然后把失败的队列写入到哪,这里就写入到我们刚才创建的logs目录下
13、开启sersync守护进程同步数据
配置sersync环境变量
[root@M-Server sersync]# echo 'export PATH=$PATH:/usr/local/sersync/bin'>/etc/profile
[root@M-Server sersync]# tail -1 /etc/profile
export PATH=$PATH:/usr/local/sersync/bin
[root@M-Server sersync]# source /etc/profile
[root@M-Server sersync]# which sersync
/usr/local/sersync/bin/sersync
启动命令
[root@M-Server bbs]# sersync -r -d -o /usr/local/sersync/conf/confxml.xml
set the system param
execute锛�echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute锛�echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -r rsync all the local files to the remote servers before the sersync work
option: -d run as a daemon
option: -o config xml name锛? /usr/local/sersync/conf/confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost host port: 8008
daemon start锛�sersync run behind the console
use rsync password-file :
user is rsync_backup
passwordfile is /etc/rsync.password
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12 = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
Max threads numbers is: 32 = 12(Thread pool nums) + 20(Sub threads)
please according your cpu 锛�use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /data0/www/www && rsync -artuz -R --delete ./ --timeout=100
[email protected]::www --password-file=/etc/rsync.password >/dev/null 2>&1
run the sersync:
watch path is: /data0/www/www
注:-r 是先做一次同步,-d是在后台启动 -o 是指定二进制文件的路径
结果发现: 发现问题 仅能一个模块的路径可以同步,其它路径下的不能同步
多实例
复制原来的配置文件confxml.xml为三份,改名为如下三个
[root@M-Server conf]# ls *confxml.xml
bbs_confxml.xml blog_confxml.xml www_confxml.xml
修改内容为,以bbs为例,只保留bbs的其它的删掉,另外两个也一样
<localpath watch="/data0/www/bbs">
<remote ip="192.168.2.106" name="bbs"/>
<remote ip="192.168.2.107" name="bbs"/>
</localpath>
修改为bbs的failLog存放文件名
<failLog path="/usr/local/sersync/logs/
bbs_rsync_fail_log.sh
" timeToExecute="60"/><!--default every 60mins execute once-->
启动多实例
sersync -r -d -o /usr/local/sersync/conf/www_confxml.xml
sersync -r -d -o /usr/local/sersync/conf/bbs_confxml.xml
sersync -r -d -o /usr/local/sersync/conf/blog_confxml.xml
查看启动进程,可以看到启动了三个进程
[root@M-Server conf]# ps -ef|grep sersync
root 4347 1 0 00:48 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/www_confxml.xml
root 4367 1 0 00:49 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/bbs_confxml.xml
root 4387 1 0 00:49 ? 00:00:00 sersync -r -d -o /usr/local/sersync/conf/blog_confxml.xml
root 4406 3392 0 00:49 pts/0 00:00:00 grep sersync
放到rc.loal开机自启动
[root@M-Server conf]# cat >>/etc/rc.local<EOF
-bash: EOF: No such file or directory
[root@M-Server conf]# cat >>/etc/rc.local<<EOF
> sersync -d -o /usr/local/sersync/conf/www_confxml.xml
> sersync -d -o /usr/local/sersync/conf/bbs_confxml.xml
> sersync -d -o /usr/local/sersync/conf/blog_confxml.xml
> EOF
[root@M-Server conf]# tail -3 /etc/rc.local
sersync -d -o /usr/local/sersync/conf/www_confxml.xml
sersync -d -o /usr/local/sersync/conf/bbs_confxml.xml
sersync -d -o /usr/local/sersync/conf/blog_confxml.xml
注意:放到rc.local里面不加-r 因为加-r 是表示重新同步,如果文件很多是没有必要再重新同步一次的
测试
在M-server上测试一次写入10000个文件
for n in `seq 10000` ;do echo dddd>www/$n.txt;done
我们发现本地都写完了,但是还在同步中。。。。说明了sersync还是有瓶颈的,并不是特别实时的