日均百万PV架构第一弹(基本架构搭建)
规划图:
以上架构可在生产环境中可以做出多样性的拆分与组合,文中四台主机限于虚拟机资源受限情况
规划如下: (结合图片四台主机的区分应该不难)
slave1.king.com
172.16.43.1
DNS轮询 -> slave1.king.com , slave2.king.com
haproxy七层代理流量(文件类型)分离 -> imgs , text , dynamic
keepalived 为haproxy HA
slave2.king.com
172.16.43.2
haproxy七层代理流量(文件类型)分离 -> imgs , text , dynamic
keepalived 为haproxy HA
slave3.king.com
172.16.43.3
nginx虚拟主机组 -> imgs1.king.com , imgs2.king.com
text1.king.com , text2.king.com
dynamic1.king.com
php-fpm模块
mysql数据库
nfs文件共享 /nfsshared
slave4.king.com
172.16.43.4
nginx虚拟主机 -> dynamic2.king.com
php-fpm模块
mysql-proxy(mmm)
mysql数据库
过程如下:
1. 一台haproxy(lvs)分离数据, dns解析双haproxy (slave1)
2. 四个虚拟主机分离imgs,text 静态测试 (slave3)
3. nginx , php-fpm , mysql 构建,测试动态代理 (slave3 , slave4)
4. mysql-proxy 实现读写分离 (slave4)
5. 实现双主keepalived HA的haproxy (slave2 , slave1)
1. 一台haproxy(lvs)分离数据, dns解析双haproxy (slave1.king.com)
i) dns解析
yum -y install named
.
vim /etc/named.rfc1912.zones
zone "king.com" IN {
type master;
file "king.com.zone";
};
.
vim /var/named/king.com.zone
$TTL 600
@ IN SOA dns.king.com. adminmail.king.com. (
2014050401
1H
5M
3D
12H )
IN NS dns
dns IN A 172.16.43.1
www IN A 172.16.43.88
www IN A 172.16.43.188
.
# 启动named
service named start
ii) haproxy配置 #安装 yum -y install haproxy # vim /etc/haproxy/haproxy.cfg global log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon # defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 30000 # listen stats mode http bind 0.0.0.0:8080 stats enable stats hide-version stats uri /haproxyadmin?stats stats realm Haproxy\ Statistics stats auth admin:admin stats admin if TRUE # frontend http-in bind *:80 mode http log global option httpclose option logasap option dontlognull capture request header Host len 20 capture request header Referer len 60 acl img_static path_beg -i /images /imgs acl img_static path_end -i .jpg .jpeg .gif .png acl text_static path_beg -i / /static /js /css acl text_static path_end -i .html .shtml .js .css # use_backend img_servers if img_static use_backend text_servers if text_static default_backend dynamic_servers # backend img_servers balance roundrobin server imgsrv1 imgs1.king.com:80 check maxconn 4000 server imgsrv2 imgs2.king.com:80 check maxconn 4000 # backend text_servers balance roundrobin server textsrv1 text1.king.com:80 check maxconn 4000 server textsrv2 text2.king.com:80 check maxconn 4000 # backend dynamic_servers balance roundrobin server websrv1 dynamic1.king.com:80 check maxconn 1000 server websrv2 dynamic2.king.com:80 check maxconn 1000
iii) hosts配置 #vim /etc/hosts 172.16.43.1 slave1.king.com 172.16.43.2 slave2.king.com 172.16.43.3 slave3.king.com 172.16.43.4 slave4.king.com # 172.16.43.3 imgs1.king.com 172.16.43.3 imgs2.king.com 172.16.43.3 text1.king.com 172.16.43.3 text2.king.com 172.16.43.3 dynamic1.king.com 172.16.43.4 dynamic2.king.com # # # 启动haproxy service haproxy start
2. 四个虚拟主机分离imgs,text 静态测试 (slave3.king.com)
i) nginx 安装
参见 http://apprentice.blog.51cto.com/2214645/1403422#nginx_install
ii) 配置 hosts 文件实现解析与 slave1.king.com hosts文件一致
iii) 配置nginx
#vim /etc/nginx/nginx.conf
#
#user nobody;
worker_processes 1;
#
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#
#pid logs/nginx.pid;
#
events {
worker_connections 1024;
}
#
http {
include mime.types;
default_type application/octet-stream;
#
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$st,,atus $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#
#access_log logs/access.log main;
#
sendfile on;
#tcp_nopush on;
#
#keepalive_timeout 0;
keepalive_timeout 5;
#
gzip on;
#
open_file_cache max=10000 inactive=60s;
open_file_cache_valid 60s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
#
proxy_cache_path /var/log/cache levels=1:2 keys_zone=web:100m max_size=1g inactive=12h;
#
server {
listen 80;
server_name dynamic1.king.com;
#
access_log /var/log/nginx/dynamic1.access.log;
#
location ~ \.php$ {
root /nfsshared/html;
fastcgi_pass 172.16.43.3:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
#
location / {
root /nfsshared/html;
proxy_cache web;
index index.php index.html index.htm;
}
}
#
server {
listen 80;
server_name imgs1.king.com;
#
access_log /var/log/nginx/imgs1.access.log;
#
location ~* \.(jpg|png|gif|jpeg)$ {
root /nfsshared/html/imgs;
}
#
location ~* \.(jpg|png|gif|jpeg)$ {
root /nfsshared/html/images;
}
#
error_page 404 /404.html;
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
#
server {
listen 80;
server_name imgs2.king.com;
#
access_log /var/log/nginx/imgs2.access.log;
#
location ~* \.(jpg|png|gif|jpeg)$ {
root /nfsshared/html/imgs;
}
#
location ~* \.(jpg|png|gif|jpeg)$ {
root /nfsshared/html/images;
}
#
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
#
server {
listen 80;
server_name text1.king.com;
#
access_log /var/log/nginx/text1.access.log;
#
location ~* \.html$ {
root /nfsshared/html/static;
}
#
location ~* \.css$ {
root /nfsshared/html/css;
}
#
location ~* \.js$ {
root /nfsshared/html/js;
}
#
location / {
root /nfsshared/html/static;
index index.html index.htm;
}
#
error_page 404 /404.html;
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
#
server {
listen 80;
server_name text2.king.com;
#
access_log /var/log/nginx/text2.access.log;
#
location ~* \.html$ {
root /nfsshared/html/static;
}
#
location ~* \.css$ {
root /nfsshared/html/css;
}
#
location ~* \.js$ {
root /nfsshared/html/js;
}
#
location / {
root /nfsshared/html/static;
index index.html index.htm;
}
#
error_page 404 /404.html;
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
iv) 测试文件
vim /nfsshared/static/index.html
<html>
<head>
<link rel="stylesheet" href="../css/test.css">
<script src="../js/test.js"></script>
</head>
<body>
<div id="testid" >测试文字</div>
<img src="../imgs/2.jpg" height=200 width=200 />
<img src="../images/1.jpg" height=200 width=200 />
</body>
</html>
#
vim /nfsshared/css/test.css
#testid {
font-size: 40px;
border: 10px red dashed;
}
#
vim /nfsshared/js/test.js
window.onload = function() {
alert("加载页面...使用js..");
}
v) 测试
访问 imgs2.king.com/2.jpg text1.king.com/test.css 均无问题
3. nginx , php-fpm , mysql 构建 (slave3.king.com , slave4.king.com)
#i) nginx安装及配置
#
vim /etc/nginx/nginx.conf
server {
listen 80;
server_name dynamic2.king.com;
#
access_log /var/log/nginx/dynamic2.access.log;
open_file_cache max=10000 inactive=60s;
open_file_cache_valid 60s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
#
proxy_cache_path /var/log/cache levels=1:2 keys_zone=web:100m max_size=1g inactive=12h;
location ~ \.php$ {
root /nfsshared/html;
fastcgi_pass 172.16.43.4:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
#
location / {
root /nfsshared/html;
proxy_cache web;
index index.php index.html index.htm;
}
}
ii) 解决php安装依赖 yum -y groupinstall "Desktop Platform Development" yum -y install libmcrypt-devel yum -y install bzip2-devel # ii) 安装php with fpm tar xf php-5.4.19.tar.bz2 cd php-5.4.19 ./configure --prefix=/usr/local/php --with-mysql=mysqlnd --with-pdo-mysql=mysqlnd --with-mysqli=mysqlnd --with-openssl --enable-mbstring --with-freetype-dir --with-jpeg-dir --with-png-dir --with-zlib --with-libxml-dir=/usr --enable-xml --enable-sockets --enable-fpm --with-mcrypt --with-config-file-path=/etc --with-config-file-scan-dir=/etc/php.d --with-bz2 make && make install # ii) 为php提供配置文件： cp php.ini-production /etc/php.ini # ii) 配置php-fpm # 为php-fpm提供SysV init脚本，并将其添加至服务列表：sapi在源码包下 cp sapi/fpm/init.d.php-fpm /etc/rc.d/init.d/php-fpm chmod +x /etc/rc.d/init.d/php-fpm chkconfig --add php-fpm chkconfig php-fpm on # # 为php-fpm提供配置文件： cp /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf # 编辑php-fpm的配置文件： vim /usr/local/php/etc/php-fpm.conf # 配置fpm的相关选项为你所需要的值，并启用pid文件： pid = /usr/local/php/var/run/php-fpm.pid listen = 172.16.43.2:9000
iii) MariaDB安装与配置(创建mysql的数据目录) mkdir /data groupadd -r mysql useradd -g mysql -r -s /sbin/nologin -M -d /data mysql chown -R mysql:mysql /data # iii) 安装二进制mysql tar xf mysql-5.5.33-linux2.6-x86_64.tar.gz -C /usr/local cd /usr/local ln -sv mysql-5.5.33-linux2.6-x86_64 mysql cd mysql chown -R mysql:mysql . mysql/scripts/mysql_install_db --user=mysql --datadir=/data chown -R root . # 提供mysql的配置文件 cp support-files/my-large.cnf /etc/my.cnf # 需要添加如下行指定mysql数据文件的存放位置： datadir = /data # iii) 为mysql提供sysv服务脚本： cd /usr/local/mysql cp support-files/mysql.server /etc/rc.d/init.d/mysqld chmod +x /etc/rc.d/init.d/mysqld 添加至服务列表： chkconfig --add mysqld chkconfig mysqld on echo "export PATH=/usr/local/mysql/bin:$PATH" > /etc/profile.d/mysql.sh . /etc/profile.d/mysql.sh # iii) 启动服务并授权php服务器账号访问 service mysqld restart mysql grant all on *.* to 'root'@'172.16.%.%' identified by '123456'; flush privileges;
slave4.king.com也是如上的配置 nginx , php-fpm , mysql
其他配置参见 http://apprentice.blog.51cto.com/2214645/1403422#php_install
4. mysql-proxy(mmm) 实现读写分离 (slave4.king.com)
其他配置参见 http://apprentice.blog.51cto.com/2214645/1399141#mysql_mmm
5. 实现双主keepalived HA的haproxy (slave2.king.com, slave1.king.com)
其他配置参见 http://apprentice.blog.51cto.com/2214645/1404853#keepalivedHA
可用性测试,在slave2.king.com瘫痪后,vip转移至slave1.king.com一个节点上
