解决Dell R410 网卡在Centos5.5 上丢包

解决Dell R410 网卡在Centos5.5 上丢包  

 

 

现象是每几十次访问网页就会一次打不开,数据库,临控等才时常取不到数据.
好像r410的网卡和centos5.5不怎么兼容

uname -a

1. Linux c1gserver 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

dmesg |grep bnx

1. Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.9.3 (March 17, 2009)
2. bnx2: eth0: using MSIX
3. bnx2: eth0 NIC Copper Link is Up, 100 Mbps full duplex

ifconfig
有大量的丢包;dropped:6620400

1. eth0      Link encap:Ethernet  HWaddr 00:26:B9:3B:XX:C3 
2.           inet addr:61.255.xx.xx  Bcast:61.255.xx.xx  Mask:255.255.255.192
3.           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
4.           RX packets:7330597243 errors:1 dropped:6620400 overruns:0 frame:1
5.           TX packets:8116489614 errors:0 dropped:0 overruns:0 carrier:0
6.           collisions:0 txqueuelen:1000
7.           RX bytes:3435468718053 (3.1 TiB)  TX bytes:3656517601592 (3.3 TiB)
8.           Interrupt:82 Memory:da000000-da012800

一.先尝试增加网卡缓存来解决问题
1.ethtool -g eth0

1. Ring parameters for eth0:
2. Pre-set maximums:
3. RX:             1020
4. RX Mini:        0
5. RX Jumbo:       4080
6. TX:             255
7. Current hardware settings:
8. RX:             255
9. RX Mini:        0
10. RX Jumbo:       0
11. TX:             255

2.ethtool -G eth0 rx 1000
ethtool -g eth0

1. Ring parameters for eth0:
2. Pre-set maximums:
3. RX:             1020
4. RX Mini:        0
5. RX Jumbo:       4080
6. TX:             255
7. Current hardware settings:
8. RX:             1000
9. RX Mini:        0
10. RX Jumbo:       0
11. TX:             255

ifconfig

1. eth0      Link encap:Ethernet  HWaddr 00:26:B9:3B:XX:C3 
2.           inet addr:61.255.xx.xx  Bcast:61.255.xx.xx  Mask:255.255.255.192
3.           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
4.           RX packets:1348879 errors:0 dropped:2714 overruns:0 frame:0
5.           TX packets:1449564 errors:0 dropped:0 overruns:0 carrier:0
6.           collisions:0 txqueuelen:1000
7.           RX bytes:646430926 (616.4 MiB)  TX bytes:576842481 (550.1 MiB)
8.           Interrupt:82 Memory:da000000-da012800

ethtool -S eth0

1. NIC statistics:
2.      rx_bytes: 645417070
3.      rx_error_bytes: 0
4.      tx_bytes: 575096746
5.      tx_error_bytes: 0
6.      rx_ucast_packets: 1345249
7.      rx_mcast_packets: 17
8.      rx_bcast_packets: 826
9.      tx_ucast_packets: 1446406
10.      tx_mcast_packets: 0
11.      tx_bcast_packets: 0
12.      tx_mac_errors: 0
13.      tx_carrier_errors: 0
14.      rx_crc_errors: 0
15.      rx_align_errors: 0
16.      tx_single_collisions: 0
17.      tx_multi_collisions: 0
18.      tx_deferred: 0
19.      tx_excess_collisions: 0
20.      tx_late_collisions: 0
21.      tx_total_collisions: 0
22.      rx_fragments: 0
23.      rx_jabbers: 0
24.      rx_undersize_packets: 0
25.      rx_oversize_packets: 0
26.      rx_64_byte_packets: 620303
27.      rx_65_to_127_byte_packets: 87492
28.      rx_128_to_255_byte_packets: 36556
29.      rx_256_to_511_byte_packets: 212099
30.      rx_512_to_1023_byte_packets: 45961
31.      rx_1024_to_1522_byte_packets: 343681
32.      rx_1523_to_9022_byte_packets: 0
33.      tx_64_byte_packets: 368814
34.      tx_65_to_127_byte_packets: 714315
35.      tx_128_to_255_byte_packets: 13048
36.      tx_256_to_511_byte_packets: 18258
37.      tx_512_to_1023_byte_packets: 39752
38.      tx_1024_to_1522_byte_packets: 292219
39.      tx_1523_to_9022_byte_packets: 0
40.      rx_xon_frames: 0
41.      rx_xoff_frames: 0
42.      tx_xon_frames: 0
43.      tx_xoff_frames: 0
44.      rx_mac_ctrl_frames: 0
45.      rx_filtered_packets: 0
46.      rx_discards: 0
47.      rx_fw_discards: 2704

rx_fw_discards: 2704
问题没有解决,依然丢包

二.更新网卡驱动
lsmod |grep bnx2

1. bnx2                  209997  0

ethtool -i eth0

1. driver: bnx2
2. version: 1.9.3
3. firmware-version: 5.0.6 NCSI 2.0.3
4. bus-info: 0000:01:00.0

modinfo bnx2

1. filename:       /lib/modules/2.6.18-164.el5/kernel/drivers/net/bnx2.ko
2. version:        1.9.3
3. license:        GPL
4. description:    Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver
5. author:         Michael Chan <[email protected]>
6. srcversion:     D151EAED8C1037CA480DE9A
7. alias:          pci:v000014E4d0000163Csv*sd*bc*sc*i*
8. alias:          pci:v000014E4d0000163Bsv*sd*bc*sc*i*
9. alias:          pci:v000014E4d0000163Asv*sd*bc*sc*i*
10. alias:          pci:v000014E4d00001639sv*sd*bc*sc*i*
11. alias:          pci:v000014E4d000016ACsv*sd*bc*sc*i*
12. alias:          pci:v000014E4d000016AAsv*sd*bc*sc*i*
13. alias:          pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i*
14. alias:          pci:v000014E4d0000164Csv*sd*bc*sc*i*
15. alias:          pci:v000014E4d0000164Asv*sd*bc*sc*i*
16. alias:          pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i*
17. alias:          pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i*
18. depends:       
19. vermagic:       2.6.18-164.el5 SMP mod_unload gcc-4.1
20. parm:           disable_msi:Disable Message Signaled Interrupt (MSI) (int)
21. parm:           enable_entropy:Allow bnx2 to populate the /dev/random entropy pool (int)
22. module_sig:     883f3504a9f766557f09578a977b7e112ebf209f6aab61295466fb787675fec1378b254df8d186609f705f1f37e06d7e2958667d258f6afe6ac1621228

1.从官网下载驱动
http://zh-cn.broadcom.com/support/ethernet_nic/netxtremeii.php

Linux
iSCSI HBA only supported on RHEL 5.4, SUSE SLES 11 SP1 or newer versions of these distributions
6.2.23 03/18/11
24MB

2.安装包
unzip linux-6.2.23.zip
cd Server/Linux/Driver/
rpm -ivh netxtreme2-6.2.23-1.src.rpm

1. 1:netxtreme2                                                       ########################################### [100%]

3.编译
cd /usr/src/redhat/
rpmbuild -bb SPECS/netxtreme2.spec

1. Wrote: /usr/src/redhat/RPMS/x86_64/netxtreme2-6.2.23-1.x86_64.rpm
2. Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.43778
3. + umask 022
4. + cd /usr/src/redhat/BUILD
5. + cd netxtreme2-6.2.23
6. + rm -rf /var/tmp/netxtreme2-buildroot /usr/src/redhat/BUILD/file.list.netxtreme2
7. + exit 0

exit 0表示成功(1-255是失败)

4.安装生成的rpm包
cd RPMS/x86_64/
rpm -ivh netxtreme2-6.2.23-1.x86_64.rpm

1. Preparing...                                                          ########################################### [100%]
2.    1:netxtreme2                                                       ########################################### [100%]

新的驱动会生成在/lib/modules/2.6.18-164.el5/updates/

5.加载驱动
depmod -a
modprobe bnx2
service network restart
lsmod |grep bnx2

1. bnx2                  209997  0

ethtool -i eth0

1. driver: bnx2
2. version: 1.9.3
3. firmware-version: 5.0.6 NCSI 2.0.3
4. bus-info: 0000:01:00.0

看着好像没区别,modinfo可以看到详细信息
modinfo bnx2

1. filename:       /lib/modules/2.6.18-164.el5/updates/bnx2.ko
2. version:        2.0.23b
3. license:        GPL
4. description:    Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver
5. author:         Michael Chan <[email protected]>
6. srcversion:     6E0DD070AB24C11F50B2712
7. alias:          pci:v000014E4d0000163Csv*sd*bc*sc*i*
8. alias:          pci:v000014E4d0000163Bsv*sd*bc*sc*i*
9. alias:          pci:v000014E4d0000163Asv*sd*bc*sc*i*
10. alias:          pci:v000014E4d00001639sv*sd*bc*sc*i*
11. alias:          pci:v000014E4d000016ACsv*sd*bc*sc*i*
12. alias:          pci:v000014E4d000016AAsv*sd*bc*sc*i*
13. alias:          pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i*
14. alias:          pci:v000014E4d0000164Csv*sd*bc*sc*i*
15. alias:          pci:v000014E4d0000164Asv*sd*bc*sc*i*
16. alias:          pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i*
17. alias:          pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i*
18. depends:       
19. vermagic:       2.6.18-164.el5 SMP mod_unload gcc-4.1
20. parm:           disable_msi:Disable Message Signaled Interrupt (MSI) (int)
21. parm:           stop_on_tx_timeout:For debugging purposes, prevent a chip  reset when a tx timeout occurs (int)

观察一段时间没有问题

20110607更新
附加进一步解决方法

在/etc/modprobe.conf中加入一行

1. options bnx2 disable_msi=1

创建一个脚本来重启服务
vi net.sh

1. /etc/init.d/network stop
2. rmmod bnx2
3. modprobe bnx2
4. /etc/init.d/network start

chmod a+x net.sh
./net.sh

参考:
http://www.pcwind.net/dell-r410.html
http://bbs.linuxtone.org/thread-3813-1-1.html