====================== 压测前:

netstat -ntlp
ulimit -SHn 65535  (加入开机启动,nginx启动前)

检查优化:
nginx.conf
php-fpm.conf
my.cnf

 


优化/etc/sysctl.conf

增加:
 kernel.shmmax = 68719476736
 # Controls the maximum number of shared memory segments, in pages
 kernel.shmall = 4294967296
    
        #syncookie
        net.ipv4.tcp_max_syn_backlog = 65536
        net.core.netdev_max_backlog =  32768
        net.core.somaxconn = 32768

        net.core.wmem_default = 8388608
        net.core.rmem_default = 8388608
        net.core.rmem_max = 16777216
        net.core.wmem_max = 16777216

        net.ipv4.tcp_timestamps = 0 
        net.ipv4.tcp_synack_retries = 2 
        net.ipv4.tcp_syn_retries = 2 
        #time_wait
        net.ipv4.tcp_syncookies = 1 
        net.ipv4.tcp_tw_recycle = 1 
        net.ipv4.tcp_fin_timeout = 5 
        net.ipv4.tcp_tw_reuse = 1 

        net.ipv4.tcp_mem = 94500000 915000000 927000000
        net.ipv4.tcp_max_orphans = 3276800
        net.ipv4.ip_local_port_range = 1024  65535

 

 

 

 

 

查看mysql连接

#netstat -anp | grep mysql | wc -l

netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'

查看TIME_WAIT连接数
netstat -ae|grep "TIME_WAIT" |wc -l

netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'

查找较多time_wait连接
netstat -n|grep TIME_WAIT|awk '{print $5}'|sort|uniq -c|sort -rn|head -n20

 

 

 

 

 

===================================实例测试

1 ) 系统说明

服务器说明:

两台 CentOS release 5.4 (Final) 32 位, 1 颗 CPU 8 核

 

a.       服务器 1

服务: Nginx + php (fpm)

nginx 开启的进程数 8

FastCGI 进程数 128

 

b.       服务器 2

服务 :Memcache+mysql

memcache 替代 session ,开启两个端口,分别分配 1024M;

 

 

2) 模拟场景 : 首页,登录,退出

并发量: 900

 

 

 

=============== 系统说明

 

#cat /etc/issue

CentOS release 5.4 (Final)

Kernel \r on an \m

 

 

# cat /proc/cpuinfo | grep physical | uniq -c

      8 physical id     : 1

 

# ulimit -a

core file size          (blocks, -c) 0  需要修改

data seg size            (kbytes, -d) unlimited

scheduling priority             (-e) 0

file size               (blocks, -f) unlimited

pending signals                 (-i) 143360

max locked memory       (kbytes, -l) 32

max memory size         (kbytes, -m) unlimited

open files                       (-n) 1024   需要修改 (因为是tcp协议  要打开套接字,要打开文件句柄,而单进程的最大打开文件句柄操作系统是有限制的,默认是1024)

pipe size            (512 bytes, -p) 8

POSIX message queues     (bytes, -q) 819200

real-time priority              (-r) 0

stack size              (kbytes, -s) 10240

cpu time               (seconds, -t) unlimited

max user processes              (-u) 143360

virtual memory          (kbytes, -v) unlimited

file locks                      (-x) unlimited

 

 

 

#getconf LONG_BIT

32

 

 

两台服务器:

Nginx + php (fpm)   113.106.90.80

Memcache+mysql   113.106.90.78

 

 

 

 

nginx 开启的进程数 8

FastCGI 进程数 128

worker_connections 65535; 
每个工作进程允许最大的同时连接数   ( Maxclient = work_processes *   worker_connections )

 

 

――― memcache 替代 session

Mysql 服务器:

/usr/local/bin/memcached -d -m 10 -u root -l 192.168.12.203 -p 13001 -c 1024 -P /tmp/memcached.pid

/usr/local/bin/memcached -d -m 10 -u root -l 192.168.12.203 -p 13002 -c 1024 -P /tmp/memcached2.pid

 

 

session.save_handler = memcache
session.save_path = "tcp://192.168.12.203:13001,tcp://192.168.12.203:13002"

 

 

 

===========优化处

 

―――――― 1 linux 默认 core file size 为 0

何谓 core 文件,当一个程序崩溃时,在进程当前工作目录的 core 文件中复制了该进程的存储图像。 core 文件仅仅是一个内存映象(同时加上调试信息),主要是用来调试的。 core 文件是个二进制文件,需要用相应的工具来分析程序崩溃时的内存映像。

系统默认 core 文件的大小为 0 ,所以没有创建。可以用 ulimit 命令查看和修改 core 文件的大小。

 

$ulimit -c

0

$ ulimit -c 1000

$ ulimit -c

1000

-c 指定修改 core 文件的大小, 1000 指定了 core 文件大小。也可以对 core 文件的大小不做限制,如:

# ulimit -c unlimited

#ulimit -c

Unlimited

 

如果想让修改永久生效,则需要修改配置文件

 

Vim /etc/profile 中的:

ulimit -S -c 0 > /dev/null 2>&1

修改为

ulimit -S -c 1000 > /dev/null 2>&1

 

#source /etc/profile  重启生效

 

 

参考资料: http://hi.baidu.com/jrckkyy/blog/item/2562320a5bdbc534b1351d95.html

 

 

―――――― 2 linux 默认值 open files 和 max user processes 为 1024

#ulimit -n

1024

#ulimit –u

1024

问题描述: 说明 server 只允许同时打开 1024 个文件,处理 1024 个用户进程

使用ulimit -a 可以查看当前系统的所有限制值,使用ulimit -n 可以查看当前的最大打开文件数。

新装的linux 默认只有1024 ,当作负载较大的服务器时,很容易遇到error: too many open files 。因此,需要将其改大。

 

解决方法:

使用 ulimit –n 65535 可即时修改,但重启后就无效了。(注ulimit -SHn 65535 等效 ulimit -n 65535 ,-S 指soft ,-H 指hard)

有如下三种修改方式:

在/etc/rc.local 中增加一行 ulimit -SHn 65535
2. 在/etc/profile 中增加一行 ulimit -SHn 65535
3. 在/etc/security/limits.conf 最后增加:

* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535

具体使用哪种,在 CentOS 中使用第1 种方式无效果,使用第3 种方式有效果,而在Debian 中使用第2 种有效果

 

# ulimit -n

65535

# ulimit -u

65535

 

备注:ulimit 命令本身就有分软硬设置,加-H 就是硬,加-S 就是软默认显示的是软限制

soft 限制指的是当前系统生效的设置值。 hard 限制值可以被普通用户降低。但是不能增加。 soft 限制不能设置的比 hard 限制更高。 只有 root 用户才能够增加 hard 限制值。

 

 

―――――― 发现存在大量 TIME_WAIT 状态的连接

 

问题描述: 发现大量 time_wait 状态的连接 , 有时甚至达到 7000 以上

netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"\t",state[key]}'


 

查找较多 time_wait 连接

netstat -n|grep TIME_WAIT|awk '{print $5}'|sort|uniq -c|sort -rn|head -n20

发现是 DB 服务器的 mysql 和 memcache 的未释放导致。

 

建议:在 PHP 程序开发时,建议大家尽量 mysql_close ,减少 msyql 消耗。

 

 

解决方法: 对于大量使用 tcp 连接的应用来说,也需要对 /etc/sysctl.conf 中的参数进行相应优化 :

vim /etc/sysctl.conf

编辑文件,加入以下内容:

net.ipv4.tcp_syncookies = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_fin_timeout = 30

 

然后执行 /sbin/sysctl -p 让参数生效。

 

优化后:

发现大量的 TIME_WAIT 已不存在, mysql 进程的占用率很快就降下来的。

 

参考资料: http://space.itpub.net/9653957/viewspace-626059

 

―――――― 4 nginx.conf 优化

 

 

 

-------5 php优化

 

open_basedir = /data/www/  #将用户可操作的文件限制在某目录
safe_mode = on #打开php的安全模式,开启重要文件的权限控制
safe_mode_exec_dir = /data/www/ #执行某些程序目录设置。不要执行任何程序,那么就可以指向我们网页目录
disable_functions = chdir,getcwd,scandir,chgrp,chmod,chown #禁用函数
expose_php = Off #关闭PHP版本信息在http头中的泄漏
display_errors = Off #禁止错误
关闭error_reporting
log_errors = On 
post_max_size = 8M
upload_max_filesize = 10M

apc.enabled = 1
apc.cache_by_default = on
apc.shm_segments = 1
apc.shm_size = 32
apc.ttl = 600
apc.user_ttl = 600
apc.num_files_hint = 0
apc.write_lock = On

 

 

 

――――――php-fpm.conf





All relative paths in this config are relative to php's install prefix



Pid file
/usr/local/services/php/logs/php-fpm.pid

Error log file
/data/logs/php/php-fpm.log

Log level
notice

When this amount of php processes exited with SIGSEGV or SIGBUS ...
10

... in a less than this interval of time, a graceful restart will be initiated.
Useful to work around accidental curruptions in accelerator's shared memory.
1m

Time limit on waiting child's reaction on signals from master
5s

Set to 'no' to debug fpm
yes







Name of pool. Used in logs and stats.
default

Address to accept fastcgi requests on.
Valid syntax is 'ip.ad.re.ss:port' or just 'port' or '/path/to/unix/socket'
127.0.0.1:9000



Set listen(2) backlog
-1

Set permissions for unix socket, if one used.
In Linux read/write permissions must be set in order to allow connections from web server.
Many BSD-derrived systems allow connections regardless of permissions.


0666


Additional php.ini defines, specific to this pool of workers.

/usr/sbin/sendmail -t -i
1


Unix user of processes
www

Unix group of processes
www

Process manager settings


Sets style of controling worker process count.
Valid values are 'static' and 'apache-like'
static 

Sets the limit on the number of simultaneous requests that will be served.
Equivalent to Apache MaxClients directive.
Equivalent to PHP_FCGI_CHILDREN environment in original php.fcgi
Used with any pm_style.
128 

Settings group for 'apache-like' pm style


Sets the number of server processes created on startup.
Used only when 'apache-like' pm_style is selected
20

Sets the desired minimum number of idle server processes.
Used only when 'apache-like' pm_style is selected
5

Sets the desired maximum number of idle server processes.
Used only when 'apache-like' pm_style is selected
35





The timeout (in seconds) for serving a single request after which the worker process will be terminated
Should be used when 'max_execution_time' ini option does not stop script execution for some reason
'0s' means 'off'
0s

The timeout (in seconds) for serving of single request after which a php backtrace will be dumped to slow.log file
'0s' means 'off'
0s

The log file for slow requests
/data/logs/php/php-fpm-slow.log

Set open file desc rlimit
65535 

Set max core size rlimit
0

Chroot to this directory at the start, absolute path


Chdir to this directory at the start, absolute path


Redirect workers' stdout and stderr into main error log.
If not set, they will be redirected to /dev/null, according to FastCGI specs
yes

How much requests each process should execute before respawn.
Useful to work around memory leaks in 3rd party libraries.
For endless request processing please specify 0
Equivalent to PHP_FCGI_MAX_REQUESTS
102400 

Comma separated list of ipv4 addresses of FastCGI clients that allowed to connect.
Equivalent to FCGI_WEB_SERVER_ADDRS environment in original php.fcgi (5.2.2+)
Makes sense only with AF_INET listening socket.
127.0.0.1

Pass environment variables like LD_LIBRARY_PATH
All $VARIABLEs are taken from current environment

$HOSTNAME
/usr/local/bin:/usr/bin:/bin
/tmp
/tmp
/tmp
$OSTYPE
$MACHTYPE
2






----------------6 mysql优化

 

 

[client]
#password       = your_password
default-character-set = utf8
port            = 33306
socket          = /data/mysql/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port            = 33306
socket          = /data/mysql/mysql.sock
skip-external-locking
key_buffer_size = 16M
max_allowed_packet = 32M
table_open_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M

open_files_limit    = 10240
max_connections = 3000
max_connect_errors = 6000
table_cache = 1024
back_log = 600
max_allowed_packet = 32M
sort_buffer_size = 2M
join_buffer_size = 2M
thread_cache_size = 64
thread_concurrency = 16
query_cache_size = 32M
query_cache_limit = 2M
query_cache_min_res_unit = 2k
#default-storage-engine = MyISAM
#default_table_type = MyISAM
thread_stack = 192K
transaction_isolation = READ-COMMITTED
tmp_table_size = 256M
max_heap_table_size = 256M
binlog_cache_size = 8M
max_binlog_cache_size = 8M
max_binlog_size = 512M
expire_logs_days = 7
key_buffer_size = 256M
read_buffer_size = 1M
read_rnd_buffer_size = 16M
bulk_insert_buffer_size = 64M

myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover
skip-name-resolve
#master-connect-retry = 10
#slave-skip-errors = 1032,1062,126,1114,1146,1048,1396
innodb_additional_mem_pool_size = 16M
innodb_buffer_pool_size = 1024M
innodb_file_io_threads = 4
innodb_thread_concurrency = 16
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 16M
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 120
innodb_file_per_table = 0




[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[myisamchk]
key_buffer_size = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout