XEN passthrough NIC on SuSE

The idea is to bind the pci device (in your case a nic) to the pciback module in the dom0 guest and then assign it to the target domU.

There are two main ways to achieve that, depending on how long you want the change to last:

a) If you want to assign the pci device to the target domU on a transient basis, do a manual allocation.
The allocation will only last until you reboot the system or the domU guest.

b) If you want the assigment to remain persistent among system reboots, stick to the automatic approach, that basically consists on editing the modprobe conf file, building a new initrd file system and changing the affected domU guest configuration file so that it includes the PCI device id in the pci parameter.

I'll only cover here the transient one.

For illustration purposes I'll use my wireless nic. Adapt the procedure to your system.

a) Manual allocation

Having your system started in Xen mode, from a dom0 bash root session do the following:

-Step 1: Find if your dom0 have support for the pciback module. If you are using open Suse 11, and are using the stock kernel, it should have it:

Code:

lustmord:~ # uname -r

2.6.25.18-0.2-xen

lustmord:~ # lsmod | grep pciback

lustmord:~ # modprobe pciback

lustmord:~ # lsmod | grep pciback

pciback                33940  0 

xenbus_be               8192  4 pciback,netbk,blkbk,blktap

lustmord:~ #

Check that the directory /sys/bus/pci/drivers/pciback exists and looks like this:

Code:

lustmord:/sys/bus/pci/drivers/pciback # ls -l

total 0

--w------- 1 root root 4096 Dec  4 23:19 bind

lrwxrwxrwx 1 root root    0 Dec  4 22:58 module -> ../../../../module/pciback

--w------- 1 root root 4096 Dec  4 23:19 new_id

--w------- 1 root root 4096 Dec  4 23:19 new_slot

-rw------- 1 root root 4096 Dec  4 23:19 permissive

-rw------- 1 root root 4096 Dec  4 23:19 quirks

--w------- 1 root root 4096 Dec  4 23:19 remove_slot

-r-------- 1 root root 4096 Dec  4 23:19 slots

--w------- 1 root root 4096 Dec  4 23:19 uevent

--w------- 1 root root 4096 Dec  4 23:19 unbind

lustmord:/sys/bus/pci/drivers/pciback #

If it does, pciback was loaded correctly.


-Step 2: Find out the PCI device id corresponding to the device you want to allocate to the target domU:

Code:

lustmord:~ # lspci | grep [Ww]ireless

03:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection (rev 61)

lustmord:~ #

The long format for the PCI Id is then : 0000:03:00.0


Step 3: Find out the driver (module) to which this PCI device is currently bound:

Code:

lustmord:~ # find /sys/bus/pci/drivers -name "0000:03:00.0"

/sys/bus/pci/drivers/iwl4965/0000:03:00.0

lustmord:~ #

Good, iwl4965 is bound to it. Double check that the directory content looks like this

Code:

lustmord:/sys/bus/pci/drivers/iwl4965 # ls -lrt

total 0

--w------- 1 root root 4096 Dec  4 21:44 unbind

--w------- 1 root root 4096 Dec  4 21:44 uevent

--w------- 1 root root 4096 Dec  4 21:44 new_id

lrwxrwxrwx 1 root root    0 Dec  4 21:44 module -> ../../../../module/iwl4965

--w------- 1 root root 4096 Dec  4 21:44 bind

lrwxrwxrwx 1 root root    0 Dec  4 21:44 0000:03:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.1/0000:03:00.0

lustmord:/sys/bus/pci/drivers/iwl4965 #

Step 4: Unbind the PCI device from the iwl4965 driver:

Code:

lustmord:~ # echo -n "0000:03:00.0" > /sys/bus/pci/drivers/iwl4965/unbind 

lustmord:~ #

We can verify that, in fact, the driver iwl4965 has been unbound from the PCI device located at 0000:03:00.0 by doing this:

Code:

lustmord:~ # cd /sys/bus/pci/drivers/iwl4965/

lustmord:/sys/bus/pci/drivers/iwl4965 # ls

bind  module  new_id  uevent  unbind

lustmord:/sys/bus/pci/drivers/iwl4965 # ls -lrt

total 0

--w------- 1 root root 4096 Dec  4 21:44 uevent

--w------- 1 root root 4096 Dec  4 21:44 new_id

lrwxrwxrwx 1 root root    0 Dec  4 21:44 module -> ../../../../module/iwl4965

--w------- 1 root root 4096 Dec  4 21:44 bind


--w------- 1 root root    0 Dec  4 23:10 unbind

lustmord:/sys/bus/pci/drivers/iwl4965 #

You will notice that the link to the PCI id vanished. That's what we want.


Step 5: Bound the PCI id 0000:03:00.0 to the pciback module

Code:

lustmord:~ # echo -n "0000:03:00.0" > /sys/bus/pci/drivers/pciback/new_slot 

lustmord:~ # echo -n "0000:03:00.0" > /sys/bus/pci/drivers/pciback/bind

Good. Now, verify that the module pciback is seeing the PCI 0000:03:00.0 device:

Code:

lustmord:~ # cd /sys/bus/pci/drivers/pciback/

lustmord:/sys/bus/pci/drivers/pciback # ls -l

total 0

lrwxrwxrwx 1 root root    0 Dec  4 23:25 0000:03:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.1/0000:03:00.0

--w------- 1 root root    0 Dec  4 23:23 bind

lrwxrwxrwx 1 root root    0 Dec  4 22:58 module -> ../../../../module/pciback

--w------- 1 root root 4096 Dec  4 23:19 new_id

--w------- 1 root root    0 Dec  4 23:23 new_slot

-rw------- 1 root root 4096 Dec  4 23:19 permissive

-rw------- 1 root root 4096 Dec  4 23:19 quirks

--w------- 1 root root 4096 Dec  4 23:19 remove_slot

-r-------- 1 root root 4096 Dec  4 23:19 slots

--w------- 1 root root 4096 Dec  4 23:19 uevent

--w------- 1 root root 4096 Dec  4 23:19 unbind

lustmord:/sys/bus/pci/drivers/pciback #

Right, just as we wanted.


Step 6: Allocate the PCI 0000:03:00.0 device to the target domU:

Now, we are in conditions to attach the device to the xen guest which is going to use it. Having the target domU stopped, run the following command to start it with the new device:

Code:

lustmord:~ # xm list

Name                                        ID   Mem VCPUs      State   Time(s)

Domain-0                                     0  1857     2     r-----    217.7

vm1-sles10                                       512     1                 0.0

vm2-sles10                                       512     1                 0.0

vm3-opensuse11                                   384     1                 0.0

Now, we launch the domU:

Code:

lustmord:~ # xm create pci=0000:03:00.0 /etc/xen/vm/vm3-opensuse11

Using config file "/etc/xen/vm/vm3-opensuse11".

Started domain vm3-opensuse11

lustmord:~ #

Wait until the target domU is running, connect to it and verify that the device was attached correctly using lspci. Once you have done that, proceed like always. Configure the module manually or using yast:

Code:

lustmord:~ # xm list

Name                                        ID   Mem VCPUs      State   Time(s)

Domain-0                                     0  1592     2     r-----    321.6

vm1-sles10                                       512     1                 0.0

vm2-sles10                                       512     1                 0.0

vm3-opensuse11                               1   384     1     -b----     20.2

lustmord:~ #

Now, connect to the domU:

Code:

lustmord:~ # xm console 1

Security Framework initialized

AppArmor: AppArmor initialized <NULL>

AppArmor: Registered secondary security module capability

Capability LSM initialized as secondary

Mount-cache hash table entries: 512

CPU: L1 I cache: 32K, L1 D cache: 32K

....
...
...
EXT3 FS on xvda2, internal journal

EXT3-fs: mounted filesystem with ordered data mode.

iwl4965: Intel(R) Wireless WiFi Link 4965AGN driver for Linux, 1.2.23ks

iwl4965: Copyright(c) 2003-2007 Intel Corporation

PCI: Enabling device 0000:00:00.0 (0000 -> 0002)

iwl4965: Detected Intel Wireless WiFi Link 4965AGN

iwl4965: Tunable channels: 13 802.11bg, 19 802.11a channels

....
....
....
ip_tables: (C) 2000-2006 Netfilter Core Team

nf_conntrack version 0.5.0 (7168 buckets, 28672 max)




Welcome to openSUSE 11.0 (i586) - Kernel 2.6.25.5-1.1-xen (xvc0).




linux-i8s4 login:

Now you log in and run lspci:

Code:

linux-i8s4 login: root 

password: 

Have a lot of fun...

linux-i8s4:~ # lspci

00:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection (rev 61)

linux-i8s4:~ #

There you have it. That is good. Now you run yast and do all the other things....And that's all there is to it.