作业-第08周--课堂-Day32-搭建rsync服务

Day32 课堂作业

1. rsync介绍与实践

1、什么是rsync?

rsyncRsync英文全称为Remote synchronization 缩写 rsync
是开源、高速的、可实现本地以及远程,全量以及增量的数据复制(拷贝)工具。

官方链接资料:http://www.samba.org/ftp/rsync/rsync.html

2、全量和增量区别

全量复制:

[root@nfs01 ~]# cp -a /etc/ /opt/
[root@nfs01 ~]# \cp -a /etc/ /opt
[root@nfs01 ~]# touch /etc/oldboy.txt
[root@nfs01 ~]# \cp -a /etc/ /opt/

增量复制:只复制 oldboy.txt

3、rsync的作用

工作中需要定时/实时数据备份。本地服务器目录、不同机器、
不同机房之间的数据备份。都可以用 rsync 完成。

4、rsync功能特性

  • 支持拷贝普通文件与特殊文件,如链接文件,设备文件等。
  • 支持排除指定文件或目录同步的功能,相当于打包命令 tar 的排除功能。
  • 可以做到保持原文件或目录的权限、时间、软硬链接、属主、组等所有属性均不改变。
  • 可实现增量复制,既只复制发生变化的数据,因此数据传输效率很高。
  • 可以使用 rcp , rsh ,ssh 等方式来配合进行隧道加密传输文件( rsync 本身不对数据加密)。
  • 可以通过 socket (进程方式)传输文件和数据(服务端和客户端)。
  • 支持匿名或认证(无需系统用户)进程模式传输,安全的进行数据备份及镜像

5、增量复制原理

使用 quick check 算法,只对增量的部分复制,根据大小属性的变化进行复制。
版本2.x 比对差异后复制,版本 3.x 一边比对一边复制。
参考地址:https://coolshell.cn/articles/7425.html

6、rsync三种工作模式介绍

1)本地( local
rsync 就是一个命令

语法:
rsync [OPTION...] SRC... [DEST]
rsync 参数选项 [源目录或文件] [目的目录或文件]

命令操作:
rsync命令
a. 把数据从一个地方复制到另一个地方(仅在一台机器增量),相当于 cp
b. 通过加参数实现删除文件和清空文件内容的功能,相当于 rm 命令。
c. 查看属性信息功能,相当于 ls

a. 拷贝实践
不保持属性:

[root@backup~]# ls /opt
[root@backup ~]# rsync /etc/hosts /opt
[root@backup ~]# ls /opt
hosts
[root@backup ~]# \cp /etc/hosts /opt

保持属性:

[root@backup ~]# rsync -zrtopg /etc/hosts /opt/
[root@backup ~]# ls -lhi /etc/hosts /opt/hosts
16829878 -rw-r--r--. 1 root root 332 4月  12 11:24 /etc/hosts
   71373 -rw-r--r--  1 root root 332 4月  12 11:24 /opt/hosts

b. 删除实践

删除文件内容:

--------------------------------------------------------------
==>创建一个空文件/null.txt<==
--------------------------------------------------------------
[root@backup ~]# touch /null.txt 
[root@backup ~]# cat /opt/hosts 
127.0.0.1    localhost localhost.localdomain localhost4 localhost4.localdomain4
::1          localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.1.5 lb01
172.16.1.6 lb02
172.16.1.7 web01
172.16.1.8 web02
172.16.1.9 web03
172.16.1.31 nfs01
172.16.1.41 backup
172.16.1.51 db01 db01.etiantian.org
172.16.1.61 m01

--------------------------------------------------------------
==>让后面hosts和一样前面null.txt<==
--------------------------------------------------------------
[root@backup ~]# rsync -r --delete /null.txt /opt/hosts
[root@backup ~]# cat /opt/hosts 

删除目录下所有文件:

--------------------------------------------------------------
==>创建一个目录/null<==
--------------------------------------------------------------
[root@backup ~]# mkdir /null

--------------------------------------------------------------
==>让后面/opt和前面/null目录内容保持一致<==
--------------------------------------------------------------
[root@backup ~]# rsync -r --delete /null/ /opt/      
[root@backup ~]# ls /opt/

c. 查看属性

[root@backup ~]# rsync /etc/hosts
-rw-r--r--            332 2019/04/12 11:24:41 hosts

2)远程Shell模式

借助类似 ssh 隧道传输数据,适合不同的机器之间复制。
异地拷贝,相当于 scp,区别:scp 是远程全量拷贝

有两种情况
第一种: pull :(拉) 从远端拉取到本地。

语法:

rsync [OPTION...] [USER@]HOST:SRC... [DEST]
rsync [参数选项] [认证用户]@[主机地址]:[源路径] [本地路径]

第二种: push :(推)从本地推到远端。

语法:

rsync [OPTION...] SRC... [USER@]HOST:[DEST]
rsync [参数选项] [本地路径] [认证用户]@[主机地址]:[目标路径]

pull 的实践:

--------------------------------------------------------------
==>两条命令在目前是等价的<==
--------------------------------------------------------------
rsync -avz [email protected]:/opt/hosts /opt
rsync -avz -e "ssh -p 22" [email protected]:/opt/hosts /opt

--------------------------------------------------------------
==>输入1.41的root密码即可<==
--------------------------------------------------------------
[root@nfs01 /opt]# rsync -avz -e "ssh -p 22" [email protected]:/opt/hosts /opt
[email protected]'s password: 
receiving incremental file list

sent 20 bytes  received 44 bytes  11.64 bytes/sec
total size is 332  speedup is 5.19

--------------------------------------------------------------
==>在本地检查结果<==
--------------------------------------------------------------
[root@nfs01 ~]# cd /opt/
[root@nfs01 /opt]# ls
hosts
[root@nfs01 /opt]# 
rsync -avz /etc/hosts [email protected]:/opt/
rsync -avz /etc/hosts -e "ssh -p 22" [email protected]:/opt/

上述命令是等价的。-e 指定通道 ssh ssh 服务连接客户端 -p 22指定 22 端口。

push 实践:

--------------------------------------------------------------
==>加密传输。<==
--------------------------------------------------------------
[root@nfs01 ~]# rsync -avz /etc/hosts [email protected]:/opt/ 
The authenticity of host '172.16.1.41 (172.16.1.41)' can't be established.
ECDSA key fingerprint is SHA256:qZSBkrmOv7xO/63qOU1uLXkPyNVHdkqvrNAcAmXqNEk.
ECDSA key fingerprint is MD5:23:d0:cb:a9:f4:7c:0b:eb:2d:07:00:e1:a3:12:d8:33.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.1.41' (ECDSA) to the list of known hosts.
[email protected]'s password: 
sending incremental file list
hosts

sent 219 bytes  received 35 bytes  14.51 bytes/sec
total size is 332  speedup is 1.31

--------------------------------------------------------------
==>在backup里检查<==
--------------------------------------------------------------
[root@backup ~]# cd /opt/
[root@backup /opt]# ls
hosts

--------------------------------------------------------------
==>命令也可以这样写<==
--------------------------------------------------------------
[root@nfs01 ~]# rsync -avz /etc/hosts -e "ssh -p 22" [email protected]:/opt/
[email protected]'s password: 
sending incremental file list

sent 44 bytes  received 12 bytes  22.40 bytes/sec
total size is 332  speedup is 5.93

小结

  • a. 参照物,执行命令的机器
  • b. [email protected] 使用的用户和主机,就用主机和用户的密码。

强调一个重点:适合rsync

nullnull/ 区别,

  • null 是目录和目录下的内容,
  • null/ 只是目录下的内容,不含本身。

3)rsync守护进程模式

首先要搭建rsync服务端(要有守护进程),然后才能在客户端实现推拉数据。
该模式是企业运维的重要模式(见后续)
rsync 服务模式:开启后台进程,接受别人访问。

7、rsync命令参数

表1:rsync 命令参数:

参数 说明
-v, --verbose 显示输出过程
-z, --compress 压缩
-a, --archive 多参数集合(-rtopgDl)
-r, --recursive 递归
-t, --times 保持修改时间属性
-o, --owne r 保持属主不变
-p, --perms 保持权限不变
-g, --group 保持用户组不变
-l, --links 保持拷贝软连接
-q, --quie t 安静的拷贝
--bwlimit=KBPS 限制I/O带宽,KBytes per second。
--delete 删除
--exclude 排除
--exclude-from 从文件中排除

提示:企业常用参数组合:-avz 或者 -vzrtopg

2. 部署rsync实战

架构前的准备已在前面介绍了,在此,不在重复了

rsync 守护进程模式应用实践

(1)以下操作都是在backup服务器

1)安装(前期已经安装过了)

--------------------------------------------------------------
==>查看版本<==
--------------------------------------------------------------
[root@backup ~]# rsync --version
rsync  version 3.1.2  protocol version 31

--------------------------------------------------------------
==>若没有安装,下面是安装的命令<==
--------------------------------------------------------------
yum install rsync

2)配置配置文件/etc/rsyncd.conf

--------------------------------------------------------------
==>操作前先备份是个好习惯<==
--------------------------------------------------------------
cp /etc/rsyncd.conf{,.ori}

--------------------------------------------------------------
==>执行以下命令<==
--------------------------------------------------------------
cat>/etc/rsyncd.conf<配置文件说明<==
--------------------------------------------------------------
cat>/etc/rsyncd.conf<注释
#created by oldboy ----------------------->注释
#site: http://www.oldboyedu.com ------->注释
uid = rsync ------------------------------->管理备份目录的用户
gid = rsync ------------------------------>管理备份目录的用户组
use chroot = no ----------------------->安全功能,数据是否锁定到备份目录
fake super = yes ---------------------->使用时,是否不需要切换root用户
max connections = 200 -------------->并发连接,同时多少客户端访问
timeout = 600 ----------------------->超时时间
pid file = /var/run/rsyncd.pid ------->进程号所在文件
lock file = /var/run/rsync.lock ------->锁文件
log file = /var/log/rsyncd.log --------->日志文件,查看报错等
ignore errors ---------------------------->忽略错误
read only = false ------------------------>可写
list = false -------------------------------->不允许列表
hosts allow = 172.16.1.0/24 -------------->哪些主机可以访问(和下面二选一)
hosts deny = 0.0.0.0/32 ------------------->哪些主机不可以访问(和上面二选一)
auth users = rsync_backup ---------------->远程虚拟连接用户
secrets file = /etc/rsync.password --------->存放密码的文件,格式:用户名:密码 ;权限:必须600
[backup] --------------------------------------->【模块名】远程访问使用模块名访问
comment = welcome to oldboyedu backup!-->注释
path = /backup/ --------------------------------->服务端用户备份的目录,用户和组为:rsync rsync
EOF

--------------------------------------------------------------
==>演示如下<==
--------------------------------------------------------------
[root@backup ~]# cat>/etc/rsyncd.conf< #rsync_config_______________start
> #created by oldboy
> #site: http://www.oldboyedu.com
> uid = rsync
> gid = rsync
> use chroot = no
> fake super = yes
> max connections = 200
> timeout = 600 
> pid file = /var/run/rsyncd.pid
> lock file = /var/run/rsync.lock
> log file = /var/log/rsyncd.log
> ignore errors
> read only = false
> list = false
> hosts allow = 172.16.1.0/24
> hosts deny = 0.0.0.0/32
> auth users = rsync_backup
> secrets file = /etc/rsync.password
> [backup]
> comment = welcome to oldboyedu backup!
> path = /backup/
> EOF

--------------------------------------------------------------
==>查看内容如下<==
--------------------------------------------------------------
[root@backup ~]# cat /etc/rsyncd.conf
#rsync_config_______________start
#created by oldboy
#site: http://www.oldboyedu.com
uid = rsync
gid = rsync
use chroot = no
fake super = yes
max connections = 200
timeout = 600 
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
[backup]
comment = welcome to oldboyedu backup!
path = /backup/

参考资料
man rsync 查命令的参数
man rsyncd.conf 查配置参数
https://www.samba.org/ftp/rsync/rsync.html

3)创建用户和备份目录

--------------------------------------------------------------
==>命令如下<==
--------------------------------------------------------------
useradd rsync
id rsync
mkdir -p /backup
chown -R rsync.rsync /backup/
ls -ld /backup/
 
--------------------------------------------------------------
==>演示如下<==
--------------------------------------------------------------
[root@backup ~]# useradd rsync
[root@backup ~]# id rsync
uid=1001(rsync) gid=1001(rsync) 组=1001(rsync)
[root@backup ~]# mkdir -p /backup
[root@backup ~]# chown -R rsync.rsync /backup/
[root@backup ~]# ls -ld /backup/
drwxr-xr-x 2 rsync rsync 6 4月  15 12:12 /backup/

4)启动和检查

--------------------------------------------------------------
==>CentOS 6及以前启动方法<==
--------------------------------------------------------------
rsync --daemon

--------------------------------------------------------------
==>CentOS7 启动方法<==
--------------------------------------------------------------
systemctl start rsyncd

--------------------------------------------------------------
==>设置开机自启动<==
--------------------------------------------------------------
systemctl enable rsyncd

--------------------------------------------------------------
==>查看状态<==
--------------------------------------------------------------
systemctl status rsyncd

--------------------------------------------------------------
==>演示如下<==
--------------------------------------------------------------
[root@backup ~]# systemctl start rsyncd
[root@backup ~]# systemctl enable rsyncd
[root@backup ~]# systemctl status rsyncd
● rsyncd.service - fast remote file copy program daemon
   Loaded: loaded (/usr/lib/systemd/system/rsyncd.service; enabled; vendor preset: disabled)
   Active: active (running) since 一 2019-04-15 12:21:29 CST; 3h 15min ago
 Main PID: 22814 (rsync)
   CGroup: /system.slice/rsyncd.service
           └─22814 /usr/bin/rsync --daemon --no-detach

4月 15 12:21:29 backup systemd[1]: Started fast remote file copy program daemon.

--------------------------------------------------------------
==>检查进程<==
--------------------------------------------------------------
[root@backup ~]# ps -ef|grep sync|grep -v grep  
root       7521      1  0 11:39 ?        00:00:00 /usr/bin/rsync --daemon --no-detach

--------------------------------------------------------------
==>检查端口<==
--------------------------------------------------------------
[root@backup ~]# netstat -lntup|grep 873 
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      7521/rsync          
tcp6       0      0 :::873                  :::*                    LISTEN      7521/rsync 

--------------------------------------------------------------
==>检查端口<==
--------------------------------------------------------------
[root@backup ~]# lsof -i :873 
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rsync   7521 root    3u  IPv4  41439      0t0  TCP *:rsync (LISTEN)
rsync   7521 root    5u  IPv6  41440      0t0  TCP *:rsync (LISTEN)

面试题:如何查看某端口对应什么服务?
答:netstat -lntup|grep 端口号
或者 lsof -i :端口号

5)配置密码文件

--------------------------------------------------------------
==>命令如下<==
--------------------------------------------------------------
echo "rsync_backup:oldboy" > /etc/rsync.password
chmod 600 /etc/rsync.password
cat /etc/rsync.password
ls -l /etc/rsync.password
 
--------------------------------------------------------------
==>演示如下<==
--------------------------------------------------------------
[root@backup ~]# echo "rsync_backup:oldboy" > /etc/rsync.password
[root@backup ~]# chmod 600 /etc/rsync.password
[root@backup ~]# cat /etc/rsync.password
rsync_backup:oldboy
[root@backup ~]# ls -l /etc/rsync.password
-rw------- 1 root root 20 4月  15 11:51 /etc/rsync.password

至此,rsync 服务端配置完成。

(2)以下操作都是在客户端服务器

以下方法2选1(推荐第二种)

方法1:认证密码文件(后面还需要输入参数,比较长)

--------------------------------------------------------------
==>命令如下<==
--------------------------------------------------------------
echo "oldboy" > /etc/rsync.password
chmod 600 /etc/rsync.password
cat /etc/rsync.password
ls -l /etc/rsync.password

--------------------------------------------------------------
==>演示如下<==
--------------------------------------------------------------
[root@nfs01 ~]# echo "oldboy" > /etc/rsync.password
[root@nfs01 ~]# chmod 600 /etc/rsync.password
[root@nfs01 ~]# cat /etc/rsync.password
oldboy
[root@nfs01 ~]# ls -l /etc/rsync.password
-rw------- 1 root root 7 4月  15 11:55 /etc/rsync.password

方法2:(后面不需要输入密码)

--------------------------------------------------------------
==>命令如下<==
--------------------------------------------------------------
echo ' export RSYNC_PASSWORD=oldboy' >>/etc/bashrc 
tail -1 /etc/bashrc 
 . /etc/bashrc 
echo $RSYNC_PASSWORD

--------------------------------------------------------------
==>演示如下<==
--------------------------------------------------------------
[root@nfs01 ~]# echo ' export RSYNC_PASSWORD=oldboy' >>/etc/bashrc 
[root@nfs01 ~]# tail -1 /etc/bashrc 
 export RSYNC_PASSWORD=oldboy
[root@nfs01 ~]# . /etc/bashrc 
[root@nfs01 ~]# echo $RSYNC_PASSWORD
oldboy

至此, nfs01rsync 客户端配置完成

(3)守护进程模式,客户端rsync的命令语法:

配置服务器端守护进程,实现数据传输

1、服务器端守护进程。
2、客户端执行命令。

1)、pull :(拉)从远端拉取到本地。

语法1(常用):(注意:与之前的语法有所变化)

rsync [OPTION...] [USER@]HOST::SRC... [DEST]
rsync [参数选项] [虚拟用户]@[主机地址]::[模块名] [本地路径]

语法2:
rsync [OPTION...] rsync://[USER@]HOST::SRC... [DEST]
rsync [参数选项] rsync://[虚拟用户]@[主机地址]/[模块名] [本地路径]

2)、push :(推)从本地推到远端。

语法1(常用):

rsync [OPTION...] [DEST] [USER@]HOST::SRC...
rsync [参数选项] [本地路径] [虚拟用户]@[主机地址]::[模块名]

语法2:

rsync [OPTION...] [DEST] rsync://[USER@]HOST::SRC...
rsync [参数选项] [本地路径] rsync://[虚拟用户]@[主机地址]/[模块名]

(4)测试成果

以下在实际操作过程报错

错误1:

[root@nfs01 ~]# rsync -avz /etc/hosts [email protected]::backup --password-file=/etc/rsync.password
@ERROR: invalid uid rsync
rsync error: error starting client-server protocol (code 5) at main.c(1648) [sender=3.1.2]

解答:原因是在配置服务端的时候未创造 rsync 用户导致的

[root@backup ~]# useradd rsync
[root@backup ~]# id rsync
uid=1001(rsync) gid=1001(rsync) 组=1001(rsync)
[root@backup ~]# mkdir -p /backup
[root@backup ~]# chown -R rsync.rsync /backup/
[root@backup ~]# ls -ld /backup/
drwxr-xr-x 2 rsync rsync 6 4月  15 12:12 /backup/

错误2:

[root@nfs01 ~]# rsync -avz /etc/hosts [email protected]::backup --password-file=/etc/rsync.password
sending incremental file list
hosts
rsync: chgrp ".hosts.YDuTjO" (in backup) failed: Operation not permitted (1)

sent 223 bytes  received 124 bytes  694.00 bytes/sec
total size is 332  speedup is 0.96
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]

解答:增加如下参数到 /etc/rsyncd.conf

fake  super  = yes #不用root用户也可以存储文件的完整属性。
This allows the full attributes of a file to  be  stored  without having to have the daemon actually running as root.
[root@backup ~]# grep fake /etc/rsyncd.conf
fake super = yes

改完配置,要重启服务:

[root@backup ~]# systemctl restart rsyncd

然后再测试:成功

--------------------------------------------------------------
==>客户端操作以下命令<==
--------------------------------------------------------------
[root@nfs01 ~]# rsync -avz /etc/hosts [email protected]::backup --password-file=/etc/rsync.password
sending incremental file list
hosts

sent 89 bytes  received 49 bytes  276.00 bytes/sec
total size is 332  speedup is 2.41
[root@nfs01 ~]# rsync -avz /etc [email protected]::backup --password-file=/etc/rsync.password

--------------------------------------------------------------
==>服务端检查<==
--------------------------------------------------------------
[root@backup ~]# ls /backup/
etc  hosts

注意: 如果客户端按照环境变量的方式配置,则可以忽略 --password-file=/etc/rsync.password 参数。(前面也提到了,就是这种情况)

--------------------------------------------------------------
==>不需要参数和密码也可以<==
--------------------------------------------------------------
[root@nfs01 ~]# rsync -avz /etc [email protected]::backup
sending incremental file list

sent 52,071 bytes  received 644 bytes  105,430.00 bytes/sec
total size is 31,244,350  speedup is 592.70

测试增量

[root@nfs01 ~]# touch /etc/oldboy.txt
[root@nfs01 ~]# rsync -avz /etc [email protected]::backup
sending incremental file list
etc/oldboy.txt

sent 52,098 bytes  received 655 bytes  105,506.00 bytes/sec
total size is 31,244,350  speedup is 592.28

第二种语法测试

[root@nfs01 ~]# rsync -avz /etc rsync://[email protected]/backup
sending incremental file list

sent 52,070 bytes  received 644 bytes  105,428.00 bytes/sec
total size is 31,244,350  speedup is 592.71

你可能感兴趣的:(作业-第08周--课堂-Day32-搭建rsync服务)