====================== 压测前:
netstat -ntlp
ulimit -SHn 65535 (加入开机启动,nginx启动前)
检查优化:
nginx.conf
php-fpm.conf
my.cnf
优化/etc/sysctl.conf
增加:
kernel.shmmax = 68719476736
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
#syncookie
net.ipv4.tcp_max_syn_backlog = 65536
net.core.netdev_max_backlog = 32768
net.core.somaxconn = 32768
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2
#time_wait
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.ip_local_port_range = 1024 65535
查看mysql连接
#netstat -anp | grep mysql | wc -l
netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
查看TIME_WAIT连接数
netstat -ae|grep "TIME_WAIT" |wc -l
netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
查找较多time_wait连接
netstat -n|grep TIME_WAIT|awk '{print $5}'|sort|uniq -c|sort -rn|head -n20
===================================实例测试
1 ) 系统说明
服务器说明:
两台 CentOS release 5.4 (Final) 32 位, 1 颗 CPU 8 核
a. 服务器 1
服务: Nginx + php (fpm)
nginx 开启的进程数 8
FastCGI 进程数 128
b. 服务器 2
服务 :Memcache+mysql
memcache 替代 session ,开启两个端口,分别分配 1024M;
2) 模拟场景 : 首页,登录,退出
并发量: 900
=============== 系统说明
#cat /etc/issue
CentOS release 5.4 (Final)
Kernel \r on an \m
# cat /proc/cpuinfo | grep physical | uniq -c
8 physical id : 1
# ulimit -a
core file size (blocks, -c) 0 需要修改
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 143360
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024 需要修改 (因为是tcp协议 要打开套接字,要打开文件句柄,而单进程的最大打开文件句柄操作系统是有限制的,默认是1024)
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 143360
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
#getconf LONG_BIT
32
两台服务器:
Nginx + php (fpm) 113.106.90.80
Memcache+mysql 113.106.90.78
nginx 开启的进程数 8
FastCGI 进程数 128
worker_connections 65535;
每个工作进程允许最大的同时连接数 ( Maxclient = work_processes * worker_connections )
――― memcache 替代 session
Mysql 服务器:
/usr/local/bin/memcached -d -m 10 -u root -l 192.168.12.203 -p 13001 -c 1024 -P /tmp/memcached.pid
/usr/local/bin/memcached -d -m 10 -u root -l 192.168.12.203 -p 13002 -c 1024 -P /tmp/memcached2.pid
session.save_handler = memcache
session.save_path = "tcp://192.168.12.203:13001,tcp://192.168.12.203:13002"
===========优化处
―――――― 1 linux 默认 core file size 为 0
何谓 core 文件,当一个程序崩溃时,在进程当前工作目录的 core 文件中复制了该进程的存储图像。 core 文件仅仅是一个内存映象(同时加上调试信息),主要是用来调试的。 core 文件是个二进制文件,需要用相应的工具来分析程序崩溃时的内存映像。
系统默认 core 文件的大小为 0 ,所以没有创建。可以用 ulimit 命令查看和修改 core 文件的大小。
$ulimit -c
0
$ ulimit -c 1000
$ ulimit -c
1000
-c 指定修改 core 文件的大小, 1000 指定了 core 文件大小。也可以对 core 文件的大小不做限制,如:
# ulimit -c unlimited
#ulimit -c
Unlimited
如果想让修改永久生效,则需要修改配置文件
Vim /etc/profile 中的:
ulimit -S -c 0 > /dev/null 2>&1
修改为
ulimit -S -c 1000 > /dev/null 2>&1
#source /etc/profile 重启生效
参考资料: http://hi.baidu.com/jrckkyy/blog/item/2562320a5bdbc534b1351d95.html
―――――― 2 linux 默认值 open files 和 max user processes 为 1024
#ulimit -n
1024
#ulimit –u
1024
问题描述: 说明 server 只允许同时打开 1024 个文件,处理 1024 个用户进程
使用ulimit -a 可以查看当前系统的所有限制值,使用ulimit -n 可以查看当前的最大打开文件数。
新装的linux 默认只有1024 ,当作负载较大的服务器时,很容易遇到error: too many open files 。因此,需要将其改大。
解决方法:
使用 ulimit –n 65535 可即时修改,但重启后就无效了。(注ulimit -SHn 65535 等效 ulimit -n 65535 ,-S 指soft ,-H 指hard)
有如下三种修改方式:
在/etc/rc.local 中增加一行 ulimit -SHn 65535
2. 在/etc/profile 中增加一行 ulimit -SHn 65535
3. 在/etc/security/limits.conf 最后增加:
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535
具体使用哪种,在 CentOS 中使用第1 种方式无效果,使用第3 种方式有效果,而在Debian 中使用第2 种有效果
# ulimit -n
65535
# ulimit -u
65535
备注:ulimit 命令本身就有分软硬设置,加-H 就是硬,加-S 就是软默认显示的是软限制
soft 限制指的是当前系统生效的设置值。 hard 限制值可以被普通用户降低。但是不能增加。 soft 限制不能设置的比 hard 限制更高。 只有 root 用户才能够增加 hard 限制值。
―――――― 3 发现存在大量 TIME_WAIT 状态的连接
问题描述: 发现大量 time_wait 状态的连接 , 有时甚至达到 7000 以上
netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'
查找较多 time_wait 连接
netstat -n|grep TIME_WAIT|awk '{print $5}'|sort|uniq -c|sort -rn|head -n20
发现是 DB 服务器的 mysql 和 memcache 的未释放导致。
建议:在 PHP 程序开发时,建议大家尽量 mysql_close ,减少 msyql 消耗。
解决方法: 对于大量使用 tcp 连接的应用来说,也需要对 /etc/sysctl.conf 中的参数进行相应优化 :
vim /etc/sysctl.conf
编辑文件,加入以下内容:
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30
然后执行 /sbin/sysctl -p 让参数生效。
优化后:
发现大量的 TIME_WAIT 已不存在, mysql 进程的占用率很快就降下来的。
参考资料: http://space.itpub.net/9653957/viewspace-626059
―――――― 4 nginx.conf 优化
-------5 php优化
open_basedir = /data/www/ #将用户可操作的文件限制在某目录
safe_mode = on #打开php的安全模式,开启重要文件的权限控制
safe_mode_exec_dir = /data/www/ #执行某些程序目录设置。不要执行任何程序,那么就可以指向我们网页目录
disable_functions = chdir,getcwd,scandir,chgrp,chmod,chown #禁用函数
expose_php = Off #关闭PHP版本信息在http头中的泄漏
display_errors = Off #禁止错误
关闭error_reporting
log_errors = On
post_max_size = 8M
upload_max_filesize = 10M
apc.enabled = 1
apc.cache_by_default = on
apc.shm_segments = 1
apc.shm_size = 32
apc.ttl = 600
apc.user_ttl = 600
apc.num_files_hint = 0
apc.write_lock = On
――――――php-fpm.conf
All relative paths in this config are relative to php's install prefix
Pid file
Error log file
Log level
When this amount of php processes exited with SIGSEGV or SIGBUS ...
... in a less than this interval of time, a graceful restart will be initiated.
Useful to work around accidental curruptions in accelerator's shared memory.
Time limit on waiting child's reaction on signals from master
Set to 'no' to debug fpm
Name of pool. Used in logs and stats.
Address to accept fastcgi requests on.
Valid syntax is 'ip.ad.re.ss:port' or just 'port' or '/path/to/unix/socket'
Set listen(2) backlog
Set permissions for unix socket, if one used.
In Linux read/write permissions must be set in order to allow connections from web server.
Many BSD-derrived systems allow connections regardless of permissions.
Additional php.ini defines, specific to this pool of workers.
Unix user of processes
Unix group of processes
Process manager settings
Sets style of controling worker process count.
Valid values are 'static' and 'apache-like'
Sets the limit on the number of simultaneous requests that will be served.
Equivalent to Apache MaxClients directive.
Equivalent to PHP_FCGI_CHILDREN environment in original php.fcgi
Used with any pm_style.
Settings group for 'apache-like' pm style
Sets the number of server processes created on startup.
Used only when 'apache-like' pm_style is selected
Sets the desired minimum number of idle server processes.
Used only when 'apache-like' pm_style is selected
Sets the desired maximum number of idle server processes.
Used only when 'apache-like' pm_style is selected
The timeout (in seconds) for serving a single request after which the worker process will be terminated
Should be used when 'max_execution_time' ini option does not stop script execution for some reason
'0s' means 'off'
The timeout (in seconds) for serving of single request after which a php backtrace will be dumped to slow.log file
'0s' means 'off'
The log file for slow requests
Set open file desc rlimit
Set max core size rlimit
Chroot to this directory at the start, absolute path
Chdir to this directory at the start, absolute path
Redirect workers' stdout and stderr into main error log.
If not set, they will be redirected to /dev/null, according to FastCGI specs
How much requests each process should execute before respawn.
Useful to work around memory leaks in 3rd party libraries.
For endless request processing please specify 0
Equivalent to PHP_FCGI_MAX_REQUESTS
Comma separated list of ipv4 addresses of FastCGI clients that allowed to connect.
Equivalent to FCGI_WEB_SERVER_ADDRS environment in original php.fcgi (5.2.2+)
Makes sense only with AF_INET listening socket.
Pass environment variables like LD_LIBRARY_PATH
All $VARIABLEs are taken from current environment
----------------6 mysql优化
[client]
#password = your_password
default-character-set = utf8
port = 33306
socket = /data/mysql/mysql.sock
# Here follows entries for some specific programs
# The MySQL server
[mysqld]
port = 33306
socket = /data/mysql/mysql.sock
skip-external-locking
key_buffer_size = 16M
max_allowed_packet = 32M
table_open_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
open_files_limit = 10240
max_connections = 3000
max_connect_errors = 6000
table_cache = 1024
back_log = 600
max_allowed_packet = 32M
sort_buffer_size = 2M
join_buffer_size = 2M
thread_cache_size = 64
thread_concurrency = 16
query_cache_size = 32M
query_cache_limit = 2M
query_cache_min_res_unit = 2k
#default-storage-engine = MyISAM
#default_table_type = MyISAM
thread_stack = 192K
transaction_isolation = READ-COMMITTED
tmp_table_size = 256M
max_heap_table_size = 256M
binlog_cache_size = 8M
max_binlog_cache_size = 8M
max_binlog_size = 512M
expire_logs_days = 7
key_buffer_size = 256M
read_buffer_size = 1M
read_rnd_buffer_size = 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover
skip-name-resolve
#master-connect-retry = 10
#slave-skip-errors = 1032,1062,126,1114,1146,1048,1396
innodb_additional_mem_pool_size = 16M
innodb_buffer_pool_size = 1024M
innodb_file_io_threads = 4
innodb_thread_concurrency = 16
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 16M
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 120
innodb_file_per_table = 0
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates
[myisamchk]
key_buffer_size = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout