企业实际应用之同步远程yum源到本地
最近在弄平台化工作，目前公网dns已经弄完，这样解析工作就很方便了，同时作为公网dns，就不需要每台主机添加nameserver来执行ip，下一步需要弄本地yum源，如果不弄的话，如果机房服务器过多，多个同时yum安装软件的话，很浪费带宽，而且yum源还不容易控制，比如有的软件安装需要epel源，有的需要rpmforge源，所以为了方便管理与控制，在机房内网搭建了一个本地yum源，同步远程的base、epel与rpmforge源（主要是centos 5与6系统的32位于64位的源），内网的主机都通过dns解析后的域名来进行访问。
一、同步远程yum源到本地
下面是我的同步脚本
#!/bin/bash
#Script name:rsync_yumrepo.sh
RsyncBin="/usr/local/rsync-3.1.0/bin/rsync"
RsyncPerm='-avrt --delete --no-iconv --bwlimit=1000'
Centos_5_base='/data/yum_repo/Centos-5/Base/'
Centos_5_epel='/data/yum_repo/Centos-5/Epel/'
Centos_5_rpmforge='/data/yum_repo/Centos-5/Rpmforge/'
Centos_6_base='/data/yum_repo/Centos-6/Base/'
Centos_6_epel='/data/yum_repo/Centos-6/Epel/'
Centos_6_rpmforge='/data/yum_repo/Centos-6/Rpmforge/'
LogFile='/data/yum_repo/rsync_yum_log'
Date=`date +%Y-%m-%d`
#check
function check {
if [ $? -eq 0 ];then
echo -e "\033[1;32mRsync is success!\033[0m" >>$LogFile/$Date.log
else
echo -e "\033[1;31mRsync is fail!\033[0m" >>$LogFile/$Date.log
fi
}
if [ ! -d "$LogFile" ];then
mkdir $LogFile
fi
#rsync centos 5 base
echo 'Now start to rsync centos 5 base!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm rsync://mirrors.yun-idc.com/centos/5/os/ $Centos_5_base >>$LogFile/$Date.log
check
#rsync centos 5 epel
echo 'Now start to rsync centos 5 epel!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm --exclude=SRPMS/ --exclude=ppc/ rsync://mirrors.yun-idc.com/epel/5/ $Centos_5_epel >>$LogFile/$Date.log
check
#rsync centos 5 rpmforge
echo 'Now start to rsync centos 5 i386 rpmforge!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm rsync://ftp-stud.fht-esslingen.de/dag/redhat/el5/en/i386/rpmforge/ $Centos_5_rpmforge/i386/ >>$LogFile/$Date.log
check
echo 'Now start to rsync centos 5 x86_64 rpmforge!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm rsync://ftp-stud.fht-esslingen.de/dag/redhat/el5/en/x86_64/rpmforge/ $Centos_5_rpmforge/x86_64/ >>$LogFile/$Date.log
check
#rsync centos 6 base
echo 'Now start to rsync centos 6 base!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm rsync://mirrors.yun-idc.com/centos/6/os/ $Centos_6_base >>$LogFile/$Date.log
check
#rsync centos 6 epel
echo 'Now start to rsync centos 6 epel!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm --exclude=SRPMS/ --exclude=ppc64/ rsync://mirrors.yun-idc.com/epel/6/ $Centos_6_epel >>$LogFile/$Date.log
check
#rsync centos 6 rpmforge
echo 'Now start to rsync centos 6 i386 rpmforge!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm rsync://ftp-stud.fht-esslingen.de/dag/redhat/el6/en/i386/rpmforge/ $Centos_6_rpmforge/i386/ >>$LogFile/$Date.log
check
echo 'Now start to rsync centos 6 x86_64 rpmforge!' >>$LogFile/$Date.log
$RsyncBin $RsyncPerm rsync://ftp-stud.fht-esslingen.de/dag/redhat/el6/en/x86_64/rpmforge/ $Centos_6_rpmforge/x86_64/ >>$LogFile/$Date.log
check
主要是使用rsync（最新的3.1.0版本），然后然后tcp模式来进行连接，同步mirrors.yun-idc.com的base与epel源到本地，同步ftp-stud.fht-esslingen.de的rpmforge源到本地。然后文件目录与rsync参数自己根据需求修改。
把这个脚本放到crontab里，让它每天凌晨同步
00 02 * * * /bin/bash /data/yum_repo/rsync_yum_shell/rsync_yumrepo.sh
同步后会有一个日子记录，下面是我的日志
Now start to rsync centos 5 base! ***************************************************** Welcome to CDS(yun-idc.com) Mirror Site Rsync Server! ***************************************************** receiving incremental file list sent 42 bytes received 305,322 bytes 55,520.73 bytes/sec total size is 10,349,249,542 speedup is 33,891.52 Rsync is success! Now start to rsync centos 5 epel! ***************************************************** Welcome to CDS(yun-idc.com) Mirror Site Rsync Server! ***************************************************** receiving incremental file list sent 64 bytes received 1,284,809 bytes 856,582.00 bytes/sec total size is 17,669,250,363 speedup is 13,751.75 Rsync is success! Now start to rsync centos 5 i386 rpmforge! Welcome to the ftp-stud.hs-esslingen.de archives. If have any unusual problems, please report them via e-mail to [email protected]. All transfers are logged. If you don't like this policy, then disconnect now. This server does not support --checksum (-c) This server does not support --compress (-z) receiving incremental file list ./ RPMS/ sent 33 bytes received 548,502 bytes 84,390.00 bytes/sec total size is 6,891,180,123 speedup is 12,562.88 Rsync is success! Now start to rsync centos 5 x86_64 rpmforge! Welcome to the ftp-stud.hs-esslingen.de archives. If have any unusual problems, please report them via e-mail to [email protected]. All transfers are logged. If you don't like this policy, then disconnect now. This server does not support --checksum (-c) This server does not support --compress (-z) receiving incremental file list ./ RPMS/ sent 33 bytes received 551,172 bytes 100,219.09 bytes/sec total size is 6,500,184,928 speedup is 11,792.68 Rsync is success! Now start to rsync centos 6 base! ***************************************************** Welcome to CDS(yun-idc.com) Mirror Site Rsync Server! ***************************************************** receiving incremental file list sent 42 bytes received 546,204 bytes 364,164.00 bytes/sec total size is 11,109,838,945 speedup is 20,338.53 Rsync is success! Now start to rsync centos 6 epel! ***************************************************** Welcome to CDS(yun-idc.com) Mirror Site Rsync Server! ***************************************************** receiving incremental file list sent 66 bytes received 1,749,197 bytes 233,235.07 bytes/sec total size is 30,814,156,298 speedup is 17,615.51 Rsync is success! Now start to rsync centos 6 i386 rpmforge! Welcome to the ftp-stud.hs-esslingen.de archives. If have any unusual problems, please report them via e-mail to [email protected]. All transfers are logged. If you don't like this policy, then disconnect now. This server does not support --checksum (-c) This server does not support --compress (-z) receiving incremental file list ./ RPMS/ sent 33 bytes received 225,253 bytes 50,063.56 bytes/sec total size is 2,342,484,019 speedup is 10,397.82 Rsync is success! Now start to rsync centos 6 x86_64 rpmforge! Welcome to the ftp-stud.hs-esslingen.de archives. If have any unusual problems, please report them via e-mail to [email protected]. All transfers are logged. If you don't like this policy, then disconnect now. This server does not support --checksum (-c) This server does not support --compress (-z) receiving incremental file list ./ RPMS/ sent 33 bytes received 226,542 bytes 50,350.00 bytes/sec total size is 2,180,556,413 speedup is 9,623.99 Rsync is success!
上面日志都是没有新的同步情况，是因为我之前已经同步完成，远程源没有变化所以我本地也没有变更，下面是我同步的网卡流量监控情况
可以看到传输的最大速度是8.65Mbs
二、客户端的repo文件
1、配置web服务器，我选择nginx
我的vhost配置为
server {
listen 80;
server_name yum-server.****.net;
index index.html index.htm index.php;
root /data/yum_repo/;
error_page 502 = /502.html;
location ~ .*\.(php|php5)?$ {
#fastcgi_pass unix:/tmp/php-cgi.sock;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi.conf;
}
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|mp3)$ {
expires 30d;
}
location ~ .*\.(js|css)?$ {
expires 12h;
}
}
log_format access '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" $http_x_forwarded_for';
access_log /data/logs/access.log access;
这个servername是你在dns里设置的名称，为了保密我没有写我真实的dns，大家可以根据自己的需求修改。
2、repo文件
Centos-Base-5.repo
[Base] name=CentOS-Base5 $basearch baseurl=http://yum-server.****.net/Centos-5/Base/$basearch enable=1 gpgcheck=1 gpgkey=http://yum-server.****.net/Centos-5/Base/$basearch/RPM-GPG-KEY-CentOS-5 [Epel] name=CentOS-Base5 $basearch baseurl=http://yum-server.****.net/Centos-5/Epel/$basearch enable=1 gpgcheck=0 [Rpmforge] name=CentOS-Base5 $basearch baseurl=http://yum-server.****.net/Centos-5/Rpmforge/$basearch enable=1 gpgcheck=0
Centos-Base-6.repo
[Base] name=CentOS-Base6 $basearch baseurl=http://yum-server.****.net/Centos-6/Base/$basearch enable=1 gpgcheck=1 gpgkey=http://yum-server.****.net/Centos-6/Base/$basearch/RPM-GPG-KEY-CentOS-6 [Epel] name=CentOS-Base6 $basearch baseurl=http://yum-server.****.net/Centos-6/Epel/$basearch enable=1 gpgcheck=0 [Rpmforge] name=CentOS-Base6 $basearch baseurl=http://yum-server.****.net/Centos-6/Rpmforge/$basearch enable=1 gpgcheck=0
然后进行下面操作：
1、按照不同的系统选择不同的repo文件，放到服务器的/etc/yum.repos.d里；
2、删除之前/etc/yum.repos.d里的repo；
3、清理缓存yum clean all；
4、将服务器上的软件包信息现在本地缓存,以提高搜索安装软件的速度 yum makecache.
[root@ip-10-10-13-8 bin]# cd /etc/yum.repos.d [root@ip-10-10-13-8 yum.repos.d]# ll 总用量 4 -rw-r--r-- 1 root root 453 12月 16 10:04 Centos-Base-6.repo [root@ip-10-10-13-8 yum.repos.d]# yum clean all Loaded plugins: aliases, changelog, downloadonly, fastestmirror, kabi, presto, security, tmprepo, verify, versionlock Loading support for CentOS kernel ABI Cleaning repos: Base Epel Rpmforge Cleaning up Everything Cleaning up list of fastest mirrors 0 delta-package files removed, by presto [root@ip-10-10-13-8 yum.repos.d]# yum makecache Loaded plugins: aliases, changelog, downloadonly, fastestmirror, kabi, presto, security, tmprepo, verify, versionlock Loading support for CentOS kernel ABI Determining fastest mirrors Base | 3.7 kB 00:00 Base/group_gz | 220 kB 00:00 Base/filelists_db | 5.9 MB 00:00 Base/primary_db | 4.4 MB 00:00 Base/other_db | 2.8 MB 00:00 Epel | 4.2 kB 00:00 Epel/group_gz | 237 kB 00:00 Epel/filelists_db | 7.9 MB 00:00 Epel/primary_db | 5.7 MB 00:00 Epel/other_db | 3.3 MB 00:00 Epel/updateinfo | 678 kB 00:00 Rpmforge | 1.9 kB 00:00 Rpmforge/filelists_db | 2.3 MB 00:00 Rpmforge/primary_db | 2.7 MB 00:00 Rpmforge/other_db | 564 kB 00:00 Metadata Cache Created [root@ip-10-10-13-8 yum.repos.d]# yum repolist Loaded plugins: aliases, changelog, downloadonly, fastestmirror, kabi, presto, security, tmprepo, verify, versionlock Loading support for CentOS kernel ABI Loading mirror speeds from cached hostfile repo id repo name status Base CentOS-Base6 x86_64 6,367 Epel CentOS-Base6 x86_64 10,138 Rpmforge CentOS-Base6 x86_64 4,650 repolist: 21,155 [root@ip-10-10-13-8 yum.repos.d]# yum list|grep echoping echoping.x86_64 5.2.0-1.2.el6.rf @Rpmforge
现在本地yum源就已经完成，可以控制软件源，并且由于走内网所有不浪费机房公网网络流量。