【Linux】万兆网卡82599驱动安装

本文是在网上找了很久外加自己实际操作而整理出来的,并不是单纯的某一篇博客的转发

在Debian系的Linux中编译并安装ixgbe驱动的教程

复制于在Debian系的Linux中编译并安装ixgbe驱动的教程

Intel的10G网卡（比如，82598、 82599、 x540）由ixgbe驱动支持。现代的Linux发行版已经带有了ixgbe驱动，通过可加载模块的方式使用。然而，有些情况你希望在你机器上的自己编译安装ixgbe驱动，比如，你想要体验ixbge驱动的最新特性时。同样，内核默认自带的ixgbe驱动中的一个问题是不允许你自定义驱动的参数。如果你想要一个完全定制的ixgbe驱动（比如 RSS、多队列、中断阈值等等），你需要手动从源码编译ixgbe驱动。
这里是如何在Ubuntu、Debian或者它们的衍生版中下载安装ixgbe驱动的教程。

第一步： 安装前提
安装之前，需要安装匹配的内核头文件和开发工具包。

$ sudo apt-get install linux-headers-$(uname -r)
$ sudo apt-get install gcc make
第二步： 编译Ixgbe驱动
从最新的ixgbe驱动中下载源码。

$ wget http://sourceforge.net/projects/e1000/files/ixgbe%20stable/3.23.2/ixgbe-3.23.2.tar.gz
如下编译ixgbe驱动。

$ tar xvfvz ixgbe-3.23.2.tar.gz
$ cd ixgbe-3.23.2/src
$ make
第三步： 检查Ixgbe驱动
编译之后，你会看到在ixgbe-3.23.2/src目录下创建了ixgbe.ko。这就是会加载到内核之中的ixgbe驱动。
用modinfo命令检查内核模块的信息。注意你需要指定模块文件的绝对路径（比如 ./ixgbe.ko 或者 /home/xmodulo/ixgbe/ixgbe-3.23.2/src/ixgbe.ko）。输出中会显示ixgbe内核的版本。

$ modinfo ./ixgbe.ko

filename: /home/xmodulo/ixgbe/ixgbe-3.23.2/src/ixgbe.ko
version: 3.23.2
license: GPL
description: Intel(R) 10 Gigabit PCI Express Network Driver
author: Intel Corporation,
srcversion: 2ADA5E537923E983FA9DAE2
alias: pci:v00008086d00001560sv*sd*bc*sc*i*
alias: pci:v00008086d00001558sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Asv*sd*bc*sc*i*
alias: pci:v00008086d00001557sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Fsv*sd*bc*sc*i*
alias: pci:v00008086d0000154Dsv*sd*bc*sc*i*
alias: pci:v00008086d00001528sv*sd*bc*sc*i*
alias: pci:v00008086d000010F8sv*sd*bc*sc*i*
alias: pci:v00008086d0000151Csv*sd*bc*sc*i*
alias: pci:v00008086d00001529sv*sd*bc*sc*i*
alias: pci:v00008086d0000152Asv*sd*bc*sc*i*
alias: pci:v00008086d000010F9sv*sd*bc*sc*i*
alias: pci:v00008086d00001514sv*sd*bc*sc*i*
alias: pci:v00008086d00001507sv*sd*bc*sc*i*
alias: pci:v00008086d000010FBsv*sd*bc*sc*i*
alias: pci:v00008086d00001517sv*sd*bc*sc*i*
alias: pci:v00008086d000010FCsv*sd*bc*sc*i*
alias: pci:v00008086d000010F7sv*sd*bc*sc*i*
alias: pci:v00008086d00001508sv*sd*bc*sc*i*
alias: pci:v00008086d000010DBsv*sd*bc*sc*i*
alias: pci:v00008086d000010F4sv*sd*bc*sc*i*
alias: pci:v00008086d000010E1sv*sd*bc*sc*i*
alias: pci:v00008086d000010F1sv*sd*bc*sc*i*
alias: pci:v00008086d000010ECsv*sd*bc*sc*i*
alias: pci:v00008086d000010DDsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Bsv*sd*bc*sc*i*
alias: pci:v00008086d000010C8sv*sd*bc*sc*i*
alias: pci:v00008086d000010C7sv*sd*bc*sc*i*
alias: pci:v00008086d000010C6sv*sd*bc*sc*i*
alias: pci:v00008086d000010B6sv*sd*bc*sc*i*
depends: ptp,dca
vermagic: 3.11.0-19-generic SMP mod_unload modversions
parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default IntMode (deprecated) (array of int)
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int)
parm: MQ:Disable or enable Multiple Queues, default 1 (array of int)
parm: DCA:Disable or enable Direct Cache Access, 0=disabled, 1=descriptor only, 2=descriptor and data (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues, default 0=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 = disable, 2-16 enable (default=8) (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int)
parm: VEPA:VEPA Bridge Mode: 0 = VEB (default), 1 = VEPA (array of int)
parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (0,1,956-488281), default 1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1) (array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500) (array of int)
parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type (array of int)
parm: LLIVLANP:Low Latency Interrupt on VLAN priority threshold (array of int)
parm: FdirPballoc:Flow Director packet buffer allocation level:
1 = 8k hash filters or 2k perfect filters
2 = 16k hash filters or 4k perfect filters
3 = 32k hash filters or 8k perfect filters (array of int)
parm: AtrSampleRate:Software ATR Tx packet sample rate (array of int)
parm: FCoE:Disable or enable FCoE Offload, default 1 (array of int)
parm: LRO:Large Receive Offload (0,1), default 1 = on (array of int)
parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599 based adapters, default 0 = Disable (array of int)
第四步： 测试Ixgbe驱动
在测试新的模块之前，如果你内核中已存在旧版本ixgbe模块的话你需要先移除它。

$ sudo rmmod ixgbe
接着使用insmod命令插入新编译的ixgbe模块。确保指定一个模块的绝对路径。

$ sudo insmod ./ixgbe.ko
如果上面的命令成功运行，就不会显示任何的信息。
如果你需要，你可以尝试加入额外的参数。比如，设置RSS的队列数量为16：

$ sudo insmod ./ixgbe.ko RSS=16
检查/var/log/kern.log来查看ixgbe驱动是否成功激活。查看日志中的“Intel(R) 10 Gigabit PCI Express Network Driver”。ixgbe的版本信息应该和之前的modinfo的显示应该相同。
Sep 18 14:48:52 spongebob kernel: [684717.906254] Intel(R) 10 Gigabit PCI Express Network Driver - version 3.22.3
第五步： 安装Ixgbe驱动
一旦你验证新的ixgbe驱动可以成功加载，最后一步是在你的系统中安装驱动。

$ sudo make install
ixgbe.ko 会安装在/lib/modules//kernel/drivers/net/ethernet/intel/ixgbe 下。
从这一步起，你可以用下面的modprobe命令加载ixgbe驱动了。注意你不必再指定绝对路径。

$ sudo modprobe ixgbe
如果你希望在启动时加载ixgbe驱动，你可以在/etc/modules的最后加入“ixgb

遇到bug

我安装的是最新版的5.3.7驱动版本.
在使用这个教程的过程里:
在中途出现了

insmod: ERROR: could not insert module ./ixgbe.ko: Unknown symbol in module

在使用dmesg的时候可以看到

Unknown symbol vxlan_get_rx_port (err 0)

实测可以使用的:

这个我是基于Debian安装的,因为是root用户,所以不需要使用sudo,而且Debian没有sudo.
只执行到了第七步…后面的没有继续执行下去.
Step 2: Download the latest version of the ixgbe driver. At the time of this writing, it is 4.1.2

The working directory in this example is /var/tmp
user@techedemic:/var/tmp$ cd /var/tmpuser@techedemic:/var/tmp$ wget http://sourceforge.net/projects/e1000/files/ixgbe%20stable/4.1.2/ixgbe-4.1.2.tar.gz/download

Step 3: Extract files

user@techedemic:/var/tmp$ tar -zxvf ixgbe-4.1.2.tar.gz

Step 4: Build the driver/module

user@techedemic:$ cd /var/tmp/ixgbe-4.1.2/srcuser@techedemic:/var/tmp/ixgbe-4.1.2/src$ sudo make clean #not required but advisableuser@techedemic:/var/tmp/ixgbe-4.1.2/src$ sudo makeuser@techedemic:/var/tmp/ixgbe-4.1.2/src$ sudo make install

Step 5: View module details for the newly built module. Pay special attention to the ‘depends’ section

user@techedemic:/var/tmp/ixgbe-4.1.2/src$ modinfo ./ixgbe.kofilename:       /var/tmp/ixgbe-4.1.2/src/./ixgbe.koversion:        4.1.2license:        GPLdescription:    Intel(R) 10 Gigabit PCI Express Network Driverauthor:         Intel Corporation, srcversion:     11CE41D9B741B44F4507285alias:          pci:v00008086d000015ADsv*sd*bc*sc*i*alias:          pci:v00008086d00001560sv*sd*bc*sc*i*alias:          pci:v00008086d00001558sv*sd*bc*sc*i*alias:          pci:v00008086d0000154Asv*sd*bc*sc*i*alias:          pci:v00008086d00001557sv*sd*bc*sc*i*alias:          pci:v00008086d0000154Fsv*sd*bc*sc*i*alias:          pci:v00008086d0000154Dsv*sd*bc*sc*i*alias:          pci:v00008086d00001528sv*sd*bc*sc*i*alias:          pci:v00008086d000010F8sv*sd*bc*sc*i*alias:          pci:v00008086d0000151Csv*sd*bc*sc*i*alias:          pci:v00008086d00001529sv*sd*bc*sc*i*alias:          pci:v00008086d0000152Asv*sd*bc*sc*i*alias:          pci:v00008086d000010F9sv*sd*bc*sc*i*alias:          pci:v00008086d00001514sv*sd*bc*sc*i*alias:          pci:v00008086d00001507sv*sd*bc*sc*i*alias:          pci:v00008086d000010FBsv*sd*bc*sc*i*alias:          pci:v00008086d00001517sv*sd*bc*sc*i*alias:          pci:v00008086d000010FCsv*sd*bc*sc*i*alias:          pci:v00008086d000010F7sv*sd*bc*sc*i*alias:          pci:v00008086d00001508sv*sd*bc*sc*i*alias:          pci:v00008086d000010DBsv*sd*bc*sc*i*alias:          pci:v00008086d000010F4sv*sd*bc*sc*i*alias:          pci:v00008086d000010E1sv*sd*bc*sc*i*alias:          pci:v00008086d000010F1sv*sd*bc*sc*i*alias:          pci:v00008086d000010ECsv*sd*bc*sc*i*alias:          pci:v00008086d000010DDsv*sd*bc*sc*i*alias:          pci:v00008086d0000150Bsv*sd*bc*sc*i*alias:          pci:v00008086d000010C8sv*sd*bc*sc*i*alias:          pci:v00008086d000010C7sv*sd*bc*sc*i*alias:          pci:v00008086d000010C6sv*sd*bc*sc*i*alias:          pci:v00008086d000010B6sv*sd*bc*sc*i*depends:        ptp,dca,vxlanvermagic:       3.16.0-45-generic SMP mod_unload modversionsparm:           InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default IntMode (deprecated) (array of int)parm:           IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int)parm:           MQ:Disable or enable Multiple Queues, default 1 (array of int)parm:           DCA:Disable or enable Direct Cache Access, 0=disabled, 1=descriptor only, 2=descriptor and data (array of int)parm:           RSS:Number of Receive-Side Scaling Descriptor Queues, default 0=number of cpus (array of int)parm:           VMDQ:Number of Virtual Machine Device Queues: 0/1 = disable, 2-16 enable (default=8) (array of int)parm:           max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int)parm:           VEPA:VEPA Bridge Mode: 0 = VEB (default), 1 = VEPA (array of int)parm:           InterruptThrottleRate:Maximum interrupts per second, per vector, (0,1,956-488281), default 1 (array of int)parm:           LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of int)parm:           LLIPush:Low Latency Interrupt on TCP Push flag (0,1) (array of int)parm:           LLISize:Low Latency Interrupt on Packet Size (0-1500) (array of int)parm:           LLIEType:Low Latency Interrupt Ethernet Protocol Type (array of int)parm:           LLIVLANP:Low Latency Interrupt on VLAN priority threshold (array of int)parm:           FdirPballoc:Flow Director packet buffer allocation level:                        1 = 8k hash filters or 2k perfect filters                        2 = 16k hash filters or 4k perfect filters                        3 = 32k hash filters or 8k perfect filters (array of int)parm:           AtrSampleRate:Software ATR Tx packet sample rate (array of int)parm:           FCoE:Disable or enable FCoE Offload, default 1 (array of int)parm:           LRO:Large Receive Offload (0,1), default 1 = on (array of int)parm:           allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599 based adapters, default 0 = Disable (array of int)parm:           dmac_watchdog:DMA coalescing watchdog in microseconds (0,41-10000), default 0 = off (array of int)parm:           vxlan_rx:VXLAN receive checksum offload (0,1), default 1 = Enable (array of int)

Step 6: Load the vxlan module

user@techedemic:/var/tmp/ixgbe-4.1.2/src$ sudo modprobe vxlan

Step 7: Install the module. If successful there should be no output.

user@techedemic:/var/tmp/ixgbe-4.1.2/src$ sudo insmod ./ixgbe.ko

Step 8: Run dmesg and check to see if new interfaces are detected after loading

the new module. Example output included below
user@techedemic:/var/tmp/ixgbe-4.1.2/src$ dmesg[19097.752081] Intel(R) 10 Gigabit PCI Express Network Driver - version 4.1.2[19097.752090] Copyright (c) 1999-2015 Intel Corporation.[19097.752414] ACPI Warning: \_SB_.PCI0.BR2C._PRT: Return Package has no elements (empty) (20140424/nsprepkg-126)[19099.110658] ixgbe 0000:03:00.0: irq 85 for MSI/MSI-X                                 <-----snipped for easier reading----->[19099.110835] ixgbe 0000:03:00.0: irq 101 for MSI/MSI-X[19099.335410] ixgbe 0000:03:00.0 eth0: MAC: 5, PHY: 6, PBA No: 020000-000[19099.335422] ixgbe 0000:03:00.0: 00:25:90:5d:0f:7e[19099.335432] ixgbe 0000:03:00.0 eth0: Enabled Features: RxQ: 16 TxQ: 16 FdirHash RSC vxlan_rx[19099.354019] ixgbe 0000:03:00.0 eth0: Intel(R) 10 Gigabit Network Connection[19101.100611] ixgbe 0000:03:00.1: irq 102 for MSI/MSI-X                                 <-----snipped for easier reading----->[19101.100790] ixgbe 0000:03:00.1: irq 118 for MSI/MSI-X[19101.321945] ixgbe 0000:03:00.1 eth1: MAC: 5, PHY: 6, PBA No: 020000-000[19101.321959] ixgbe 0000:03:00.1: 00:25:90:5d:0f:7f[19101.321969] ixgbe 0000:03:00.1 eth1: Enabled Features: RxQ: 16 TxQ: 16 FdirHash RSC vxlan_rx[19101.341933] ixgbe 0000:03:00.1 eth1: Intel(R) 10 Gigabit Network Connection[19161.564788] ixgbe 0000:03:00.0: registered PHC device on eth0[19163.468677] ixgbe 0000:03:00.1: registered PHC device on eth1
As can be seen above, my dual port 10gbps interfaces have been detected as eth0 and eth1.

Step 9: Configure your interfaces in /etc/network/interfaces as you would for any other (not shown here) and bring them up. You can then use ethtool to see supported modes.

user@techedemic: /var/tmp/ixgbe-4.1.2/src$ sudo ifup eth0 && sudo ifup eth1 user@techedemic: /var/tmp/ixgbe-4.1.2/src$ ifconfig eth0eth0      Link encap:Ethernet  HWaddr 00:25:90:5d:0f:7e          inet6 addr: fe80::225:90ff:fe5d:f7e/64 Scope:Link          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1          RX packets:0 errors:0 dropped:0 overruns:0 frame:0          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0          collisions:0 txqueuelen:1000          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B) user@techedemic: /var/tmp/ixgbe-4.1.2/src$ sudo ethtool eth0Settings for eth0:        Supported ports: [ TP ]        Supported link modes:   1000baseT/Full                                10000baseT/Full        Supported pause frame use: No        Supports auto-negotiation: Yes        Advertised link modes:  1000baseT/Full                                10000baseT/Full        Advertised pause frame use: Symmetric        Advertised auto-negotiation: Yes        Speed: 10000Mb/s        Duplex: Full        Port: Twisted Pair        PHYAD: 0        Transceiver: external        Auto-negotiation: on        MDI-X: Unknown        Supports Wake-on: umbg        Wake-on: g        Current message level: 0x00000007 (7)                               drv probe link        Link detected: yes

failed to load because an unsupported SFP+ module type was detected.的解决

参考的地址:

centos 万兆网卡使用要点

随着宽带技术的快速发展，服务器使用万兆网卡的概率越来越高。最近装了几台服务器都用的万兆网卡，为了图便宜，网卡和模块都是淘宝上买的，这部还真遇到不少问题。

我的服务器都是centos6.4 64位的，网卡装上后，lspci | grep 82599 能看到网卡
执行lspci |grep 82599能看到网卡在pci设备中
04:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
但是ifcfg -a 看不到网卡比如eth0
(这个和我遇到的一模一样)

使用dmesg | grep ixgbe 发现报错，说发现不支持的光模块，查看intel82599使用手册嘿，里面还真说了，那些网卡经过测试， 意思这些默认是支持的。
ixgbe 0000:04:00.0: setting latency timer to 64
ixgbe 0000:04:00.0: failed to load because an unsupported SFP+ module type was detected.
ixgbe 0000:04:00.0: Reload the driver after installing a supported module.
后然百度发现有篇文章说到光模块兼容性问题，马上试了一下，果然解决了。
rmmod ixgbe 先删除驱动
重新加载驱动，

modprobe ixgbe allow_unsupported_sfp=1,1 

驱动对allow_unsupported_sfp的解释为：
allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599-based adapters (uint)，即允
许82599网卡使用不兼容SFP+模块。
重新加载后，大部分情况下ifconfig就能够看到网卡信息了。
另外如果遇到安装驱动等问题，可以参考intel网卡的readme，里面很全面，不过是英文的。

warning信息

WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics. 
Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter. 
Intel Corporation is not responsible for any harm caused by using untested modules.

dmesg的时候,会看到warning信息…
还是使用Intel配套的光模块更合适

复制,整理以及自己操作所得到的文章,并非直接转载…