【汇总压力测试】linux+nginx+php-fpm+mysql优化
====================== 压测前：
netstat -ntlp
ulimit -SHn 65535 （加入开机启动，nginx启动前）
检查优化：
nginx.conf
php-fpm.conf
my.cnf
优化/etc/sysctl.conf
增加：
kernel.shmmax = 68719476736
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
#syncookie
net.ipv4.tcp_max_syn_backlog = 65536
net.core.netdev_max_backlog = 32768
net.core.somaxconn = 32768
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2
#time_wait
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.ip_local_port_range = 1024 65535
查看mysql连接
#netstat -anp | grep mysql | wc -l
netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
查看TIME_WAIT连接数
netstat -ae|grep "TIME_WAIT" |wc -l
netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
查找较多time_wait连接
netstat -n|grep TIME_WAIT|awk '{print $5}'|sort|uniq -c|sort -rn|head -n20
===================================实例测试
1 ） 系统说明
服务器说明：
两台 CentOS release 5.4 (Final) 32 位， 1 颗 CPU 8 核
a. 服务器 1
服务： Nginx + php (fpm)
nginx 开启的进程数 8
FastCGI 进程数 128
b. 服务器 2
服务 :Memcache+mysql
memcache 替代 session ，开启两个端口，分别分配 1024M;
2) 模拟场景 : 首页，登录，退出
并发量： 900
=============== 系统说明
#cat /etc/issue
CentOS release 5.4 (Final)
Kernel \r on an \m
# cat /proc/cpuinfo | grep physical | uniq -c
8 physical id : 1
# ulimit -a
core file size (blocks, -c) 0 需要修改
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 143360
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024 需要修改 (因为是tcp协议 要打开套接字，要打开文件句柄，而单进程的最大打开文件句柄操作系统是有限制的，默认是1024)
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 143360
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
#getconf LONG_BIT
32
两台服务器：
Nginx + php (fpm) 113.106.90.80
Memcache+mysql 113.106.90.78
nginx 开启的进程数 8
FastCGI 进程数 128
worker_connections 65535;
每个工作进程允许最大的同时连接数 （ Maxclient = work_processes * worker_connections ）
――― memcache 替代 session
Mysql 服务器：
/usr/local/bin/memcached -d -m 10 -u root -l 192.168.12.203 -p 13001 -c 1024 -P /tmp/memcached.pid
/usr/local/bin/memcached -d -m 10 -u root -l 192.168.12.203 -p 13002 -c 1024 -P /tmp/memcached2.pid
session.save_handler = memcache
session.save_path = "tcp://192.168.12.203:13001,tcp://192.168.12.203:13002"
＝＝＝＝＝＝＝＝＝＝＝优化处
―――――― 1 linux 默认 core file size 为 0
何谓 core 文件，当一个程序崩溃时，在进程当前工作目录的 core 文件中复制了该进程的存储图像。 core 文件仅仅是一个内存映象（同时加上调试信息），主要是用来调试的。 core 文件是个二进制文件，需要用相应的工具来分析程序崩溃时的内存映像。
系统默认 core 文件的大小为 0 ，所以没有创建。可以用 ulimit 命令查看和修改 core 文件的大小。
$ulimit -c
0
$ ulimit -c 1000
$ ulimit -c
1000
-c 指定修改 core 文件的大小， 1000 指定了 core 文件大小。也可以对 core 文件的大小不做限制，如：
# ulimit -c unlimited
#ulimit -c
Unlimited
如果想让修改永久生效，则需要修改配置文件
Vim /etc/profile 中的：
ulimit -S -c 0 > /dev/null 2>&1
修改为
ulimit -S -c 1000 > /dev/null 2>&1
#source /etc/profile 重启生效
参考资料： http://hi.baidu.com/jrckkyy/blog/item/2562320a5bdbc534b1351d95.html
―――――― 2 linux 默认值 open files 和 max user processes 为 1024
#ulimit -n
1024
#ulimit –u
1024
问题描述： 说明 server 只允许同时打开 1024 个文件，处理 1024 个用户进程
使用ulimit -a 可以查看当前系统的所有限制值，使用ulimit -n 可以查看当前的最大打开文件数。
新装的linux 默认只有1024 ，当作负载较大的服务器时，很容易遇到error: too many open files 。因此，需要将其改大。
解决方法：
使用 ulimit –n 65535 可即时修改，但重启后就无效了。（注ulimit -SHn 65535 等效 ulimit -n 65535 ，-S 指soft ，-H 指hard)
有如下三种修改方式：
在/etc/rc.local 中增加一行 ulimit -SHn 65535
2. 在/etc/profile 中增加一行 ulimit -SHn 65535
3. 在/etc/security/limits.conf 最后增加：
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535
具体使用哪种，在 CentOS 中使用第1 种方式无效果，使用第3 种方式有效果，而在Debian 中使用第2 种有效果
# ulimit -n
65535
# ulimit -u
65535
备注：ulimit 命令本身就有分软硬设置，加-H 就是硬，加-S 就是软默认显示的是软限制
soft 限制指的是当前系统生效的设置值。 hard 限制值可以被普通用户降低。但是不能增加。 soft 限制不能设置的比 hard 限制更高。 只有 root 用户才能够增加 hard 限制值。
―――――― 3 发现存在大量 TIME_WAIT 状态的连接
问题描述： 发现大量 time_wait 状态的连接 , 有时甚至达到 7000 以上
netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
查找较多 time_wait 连接
netstat -n|grep TIME_WAIT|awk '{print $5}'|sort|uniq -c|sort -rn|head -n20
发现是 DB 服务器的 mysql 和 memcache 的未释放导致。
建议：在 PHP 程序开发时，建议大家尽量 mysql_close ，减少 msyql 消耗。
解决方法： 对于大量使用 tcp 连接的应用来说，也需要对 /etc/sysctl.conf 中的参数进行相应优化 :
vim /etc/sysctl.conf
编辑文件，加入以下内容：
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30
然后执行 /sbin/sysctl -p 让参数生效。
优化后：
发现大量的 TIME_WAIT 已不存在， mysql 进程的占用率很快就降下来的。
参考资料： http://space.itpub.net/9653957/viewspace-626059
―――――― 4 nginx.conf 优化
－－－－－－－5 php优化
open_basedir = /data/www/ #将用户可操作的文件限制在某目录
safe_mode = on #打开php的安全模式，开启重要文件的权限控制
safe_mode_exec_dir = /data/www/ #执行某些程序目录设置。不要执行任何程序，那么就可以指向我们网页目录
disable_functions = chdir,getcwd,scandir,chgrp,chmod,chown #禁用函数
expose_php = Off #关闭PHP版本信息在http头中的泄漏
display_errors = Off #禁止错误
关闭error_reporting
log_errors = On
post_max_size = 8M
upload_max_filesize = 10M
apc.enabled = 1
apc.cache_by_default = on
apc.shm_segments = 1
apc.shm_size = 32
apc.ttl = 600
apc.user_ttl = 600
apc.num_files_hint = 0
apc.write_lock = On
――――――php-fpm.conf
<?xml version="1.0" ?>
<configuration>
All relative paths in this config are relative to php's install prefix
<section name="global_options">
Pid file
<value name="pid_file">/usr/local/services/php/logs/php-fpm.pid</value>
Error log file
<value name="error_log">/data/logs/php/php-fpm.log</value>
Log level
<value name="log_level">notice</value>
When this amount of php processes exited with SIGSEGV or SIGBUS ...
<value name="emergency_restart_threshold">10</value>
... in a less than this interval of time, a graceful restart will be initiated.
Useful to work around accidental curruptions in accelerator's shared memory.
<value name="emergency_restart_interval">1m</value>
Time limit on waiting child's reaction on signals from master
<value name="process_control_timeout">5s</value>
Set to 'no' to debug fpm
<value name="daemonize">yes</value>
</section>
<workers>
<section name="pool">
Name of pool. Used in logs and stats.
<value name="name">default</value>
Address to accept fastcgi requests on.
Valid syntax is 'ip.ad.re.ss:port' or just 'port' or '/path/to/unix/socket'
<value name="listen_address">127.0.0.1:9000</value>
<value name="listen_options">
Set listen(2) backlog
<value name="backlog">-1</value>
Set permissions for unix socket, if one used.
In Linux read/write permissions must be set in order to allow connections from web server.
Many BSD-derrived systems allow connections regardless of permissions.
<value name="owner"></value>
<value name="group"></value>
<value name="mode">0666</value>
</value>
Additional php.ini defines, specific to this pool of workers.
<value name="php_defines">
<value name="sendmail_path">/usr/sbin/sendmail -t -i</value>
<value name="display_errors">1</value>
</value>
Unix user of processes
<value name="user">www</value>
Unix group of processes
<value name="group">www</value>
Process manager settings
<value name="pm">
Sets style of controling worker process count.
Valid values are 'static' and 'apache-like'
<value name="style">static</value>
Sets the limit on the number of simultaneous requests that will be served.
Equivalent to Apache MaxClients directive.
Equivalent to PHP_FCGI_CHILDREN environment in original php.fcgi
Used with any pm_style.
<value name="max_children">128</value>
Settings group for 'apache-like' pm style
<value name="apache_like">
Sets the number of server processes created on startup.
Used only when 'apache-like' pm_style is selected
<value name="StartServers">20</value>
Sets the desired minimum number of idle server processes.
Used only when 'apache-like' pm_style is selected
<value name="MinSpareServers">5</value>
Sets the desired maximum number of idle server processes.
Used only when 'apache-like' pm_style is selected
<value name="MaxSpareServers">35</value>
</value>
</value>
The timeout (in seconds) for serving a single request after which the worker process will be terminated
Should be used when 'max_execution_time' ini option does not stop script execution for some reason
'0s' means 'off'
<value name="request_terminate_timeout">0s</value>
The timeout (in seconds) for serving of single request after which a php backtrace will be dumped to slow.log file
'0s' means 'off'
<value name="request_slowlog_timeout">0s</value>
The log file for slow requests
<value name="slowlog">/data/logs/php/php-fpm-slow.log</value>
Set open file desc rlimit
<value name="rlimit_files">65535</value>
Set max core size rlimit
<value name="rlimit_core">0</value>
Chroot to this directory at the start, absolute path
<value name="chroot"></value>
Chdir to this directory at the start, absolute path
<value name="chdir"></value>
Redirect workers' stdout and stderr into main error log.
If not set, they will be redirected to /dev/null, according to FastCGI specs
<value name="catch_workers_output">yes</value>
How much requests each process should execute before respawn.
Useful to work around memory leaks in 3rd party libraries.
For endless request processing please specify 0
Equivalent to PHP_FCGI_MAX_REQUESTS
<value name="max_requests">102400</value>
Comma separated list of ipv4 addresses of FastCGI clients that allowed to connect.
Equivalent to FCGI_WEB_SERVER_ADDRS environment in original php.fcgi (5.2.2+)
Makes sense only with AF_INET listening socket.
<value name="allowed_clients">127.0.0.1</value>
Pass environment variables like LD_LIBRARY_PATH
All $VARIABLEs are taken from current environment
<value name="environment">
<value name="HOSTNAME">$HOSTNAME</value>
<value name="PATH">/usr/local/bin:/usr/bin:/bin</value>
<value name="TMP">/tmp</value>
<value name="TMPDIR">/tmp</value>
<value name="TEMP">/tmp</value>
<value name="OSTYPE">$OSTYPE</value>
<value name="MACHTYPE">$MACHTYPE</value>
<value name="MALLOC_CHECK_">2</value>
</value>
</section>
</workers>
</configuration>
－－－－－－－－－－－－－－－－6 mysql优化
[client]
#password = your_password
default-character-set = utf8
port = 33306
socket = /data/mysql/mysql.sock
# Here follows entries for some specific programs
# The MySQL server
[mysqld]
port = 33306
socket = /data/mysql/mysql.sock
skip-external-locking
key_buffer_size = 16M
max_allowed_packet = 32M
table_open_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
open_files_limit = 10240
max_connections = 3000
max_connect_errors = 6000
table_cache = 1024
back_log = 600
max_allowed_packet = 32M
sort_buffer_size = 2M
join_buffer_size = 2M
thread_cache_size = 64
thread_concurrency = 16
query_cache_size = 32M
query_cache_limit = 2M
query_cache_min_res_unit = 2k
#default-storage-engine = MyISAM
#default_table_type = MyISAM
thread_stack = 192K
transaction_isolation = READ-COMMITTED
tmp_table_size = 256M
max_heap_table_size = 256M
binlog_cache_size = 8M
max_binlog_cache_size = 8M
max_binlog_size = 512M
expire_logs_days = 7
key_buffer_size = 256M
read_buffer_size = 1M
read_rnd_buffer_size = 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover
skip-name-resolve
#master-connect-retry = 10
#slave-skip-errors = 1032,1062,126,1114,1146,1048,1396
innodb_additional_mem_pool_size = 16M
innodb_buffer_pool_size = 1024M
innodb_file_io_threads = 4
innodb_thread_concurrency = 16
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 16M
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 120
innodb_file_per_table = 0
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates
[myisamchk]
key_buffer_size = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout