高并发、大流量网卡调优 _转

http://navyaijm.blog.51cto.com/4647068/1334671

 

周五晚上LVS集群的一台LB由于CPU单核耗尽挂掉了,最后查到原因是网卡调优没有生效,今天查了一下网卡调优的资料,感谢同事higkoo给我讲解了一下这方面的东西,下面是关于我们公司网卡调优的相关知识,欢迎大家共同探讨,让我们的机器跑的更high。

1、Broadcom的网卡建议关闭GRO功能

1
2
3
4
ethtool -K eth0 gro off
ethtool -K eth1 gro off
ethtool -K eth2 gro off
ethtool -K eth3 gro off

2、关闭irqbalance服务并手动分配网卡中断

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
service irqbalance stop
chkconfig irqbalance off
# 查看网卡中断号
grep ethx /proc/interrupts
# 分配到每颗颗CPU核上
cat /proc/irq/ {84,85,86,87,88,89,90,91,92,93} /smp_affinity (下面 echo 的值从此获取)
echo 1 > /proc/irq/84/smp_affinity
echo 2 > /proc/irq/85/smp_affinity
echo 4 > /proc/irq/86/smp_affinity
echo 8 > /proc/irq/87/smp_affinity
echo 10 > /proc/irq/88/smp_affinity
echo 20 > /proc/irq/89/smp_affinity
echo 40 > /proc/irq/90/smp_affinity
echo 80 > /proc/irq/91/smp_affinity
echo 100 > /proc/irq/92/smp_affinity
echo 200 > /proc/irq/93/smp_affinity

3、开启网卡的RPS功能 (Linux内核2.6.38或以上版本支持)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Enable RPS (Receive Packet Steering)
rfc=4096
cc=$( grep -c processor /proc/cpuinfo )
rsfe=$( echo $cc*$rfc | bc )
sysctl -w net.core.rps_sock_flow_entries=$rsfe
for fileRps in $( ls /sys/class/net/eth * /queues/rx- * /rps_cpus )
do
echo fff > $fileRps
done
for fileRfc in $( ls /sys/class/net/eth * /queues/rx- * /rps_flow_cnt )
do
echo $rfc > $fileRfc
done
tail /sys/class/net/eth * /queues/rx- */{rps_cpus,rps_flow_cnt}

献上一个完整的脚本:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
vi /opt/sbin/change_irq .sh
#!/bin/bash
ethtool -K eth0 gro off
ethtool -K eth1 gro off
ethtool -K eth2 gro off
ethtool -K eth3 gro off
service irqbalance stop
chkconfig irqbalance off
cat /proc/irq/ {84,85,86,87,88,89,90,91,92,93} /smp_affinity
echo 1 > /proc/irq/84/smp_affinity
echo 2 > /proc/irq/85/smp_affinity
echo 4 > /proc/irq/86/smp_affinity
echo 8 > /proc/irq/87/smp_affinity
echo 10 > /proc/irq/88/smp_affinity
echo 20 > /proc/irq/89/smp_affinity
echo 40 > /proc/irq/90/smp_affinity
echo 80 > /proc/irq/91/smp_affinity
echo 100 > /proc/irq/92/smp_affinity
echo 200 > /proc/irq/93/smp_affinity
# Enable RPS (Receive Packet Steering)
rfc=4096
cc=$( grep -c processor /proc/cpuinfo )
rsfe=$( echo $cc*$rfc | bc )
sysctl -w net.core.rps_sock_flow_entries=$rsfe
for fileRps in $( ls /sys/class/net/eth * /queues/rx- * /rps_cpus )
do
echo fff > $fileRps
done
for fileRfc in $( ls /sys/class/net/eth * /queues/rx- * /rps_flow_cnt )
do
echo $rfc > $fileRfc
done
tail /sys/class/net/eth * /queues/rx- */{rps_cpus,rps_flow_cnt}
chmod +x /opt/sbin/change_irq .sh
echo "/opt/sbin/change_irq.sh" >> /etc/rc . local
PS:记得修改网卡中断号,别直接拿来用哦

转载于:https://www.cnblogs.com/kaka100/p/3455609.html

你可能感兴趣的:(高并发、大流量网卡调优 _转)