Red Hat KVM SR-IOV 配置

SR-IOV是PCI passthrough 的增强，设备本身（例如网卡）要支持SR-IOV

example for Intel 82576 NIC

1. enable Intel VT-d in BIOS

2. using "lspci | grep 82576" to verify the NIC was detected

3. list SR-IOV module using "lsmod | grep igb", if not, "modprobe igb"

4. Activate Virtual Functions

# modprobe -r igb

# modprobe igb max_vfs=7

5. Make the Virtual Functions persistent

# modprobe /etc/modprobe.d/igb.conf igb max_vfs=7

6. Inspect the new Virtual Functions

# lspci | grep 82576   

7. Verify devices exist with virsh

 

# virsh nodedev-list | grep 0b
pci_0000_0b_00_0
pci_0000_0b_00_1
pci_0000_0b_10_0
pci_0000_0b_10_1
pci_0000_0b_10_2
pci_0000_0b_10_3
pci_0000_0b_10_4
pci_0000_0b_10_5
pci_0000_0b_10_6
pci_0000_0b_11_7
pci_0000_0b_11_1
pci_0000_0b_11_2
pci_0000_0b_11_3
pci_0000_0b_11_4
pci_0000_0b_11_5

8. Get device details with virsh

The pci_0000_0b_00_0 is one of the Physical Functions and pci_0000_0b_10_0 is the first corresponding Virtual Function for that Physical Function. Use the virsh nodedev-dumpxml command to get advanced output for both devices.

# virsh nodedev-dumpxml pci_0000_0b_00_0
<device>
   <name>pci_0000_0b_00_0</name>
   <parent>pci_0000_00_01_0</parent>
   <driver>
      <name>igb</name>
   </driver>
   <capability type='pci'>
      <domain>0</domain>
      <bus>11</bus>
      <slot>0</slot>
      <function>0</function>
      <product id='0x10c9'>Intel Corporation</product>
      <vendor id='0x8086'>82576 Gigabit Network Connection</vendor>
   </capability>
</device>

# virsh nodedev-dumpxml pci_0000_0b_10_0
<device>
   <name>pci_0000_0b_10_0</name>
   <parent>pci_0000_00_01_0</parent>
   <driver>
      <name>igbvf</name>
   </driver>
   <capability type='pci'>
      <domain>0</domain>
      <bus>11</bus>
      <slot>16</slot>
      <function>0</function>
      <product id='0x10ca'>Intel Corporation</product>
      <vendor id='0x8086'>82576 Virtual Function</vendor>
   </capability>
</device>

9. Detach the Virtual Functions

Detach the Virtual Function from the host so that the Virtual Function can be used by the guest.

# virsh nodedev-dettach pci_0000_0b_10_0
Device pci_0000_0b_10_0 dettached

10. Add the Virtual Function to the guest

 

a. Shut down the guest.

b. Refer to the output from the virsh nodedev-dumpxml pci_0000_0b_10_0 command for the values required for the configuration file.

Optionally, convert slot and function values to hexadecimal values (from decimal) to get the PCI bus addresses. Append "0x" to the beginning of the output to tell the computer that the value is a hexadecimal number.

The example device has the following values: bus = 3, slot = 16 and function = 1. The decimal configuration uses those three values:

bus='3'
slot='16'
function='1'

If you want to convert to hexadecimal values, you can use the printf utility to convert from decimal values:

$ printf %x 3
3
$ printf %x 16
10
$ printf %x 1
1

The example device would use the following hexadecimal values in the configuration file:

bus='0x0b'
slot='0x10'
function='0x01'

c. Open the XML configuration file with the virsh edit command. This example edits a guest named MyGuest.

# virsh edit MyGuest

d. The default text editor will open the libvirt configuration file for the guest. Add the new device to the devices section of the XML configuration file.

<hostdev mode='subsystem' type='pci' managed='yes'>
   <source>
      <address bus='0x0b' slot='0x10' function='0x01'/>
   </source>
</hostdev>

e. Save the configuration.

11. Restart

Restart the guest to complete the installation.

# virsh start MyGuest