在 openvswitch 上配置 VXLAN

转载自：http://it.taocms.org/06/894.htm

（本文讲的是 korg 内核中的 openvswitch，而非 openvswitch.org 上提供的openvswitch模块。）

之前讲过 VXLAN，但是一直没有动手实践。其实配置 VXLAN 要比配置 GRE tunnel 简单很多，你所需要做的只要两步：

1. 添加 VXLAN 设备（需要比较新的 iproute2 和内核支持）：

# ./ip/ip li add vxlan0 type vxlan id 42 group 239.1.1.1 local 10.16.43.214 mtu 1500 dev eth0

其中，10.16.43.214 是配置在 eth0 （或其它连接两台主机的设备）上的本地IP地址。

2. 把 vxlan0 添加到 openvswitch 的 bridge 中去：

# ovs-vsctl add-port ovsbr0 vxlan0

最终的结果如下所示：

# ./ip/ip -d li show vxlan0
7: vxlan0:  mtu 1500 qdisc noqueue
state UNKNOWN mode DEFAULT
    link/ether d2:93:a2:08:10:1e brd ff:ff:ff:ff:ff:ff
    vxlan id 42 group 239.1.1.1 local 10.16.43.214 dev eth0 port 32768
61000 ageing 300
# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr AC:16:2D:05:14:3A
          inet addr:10.16.43.214  Bcast:10.16.47.255  Mask:255.255.248.0
          inet6 addr: 2620:52:0:102f:ae16:2dff:fe05:143a/64 Scope:Global
          inet6 addr: fe80::ae16:2dff:fe05:143a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:386702 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1505515 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:42804806 (40.8 MiB)  TX bytes:1202562852 (1.1 GiB)
          Interrupt:20 Memory:fbc00000-fbc20000

# ifconfig vxlan0
vxlan0    Link encap:Ethernet  HWaddr D2:93:A2:08:10:1E
          inet6 addr: fe80::d093:a2ff:fe08:101e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:346746 errors:0 dropped:0 overruns:0 frame:0
          TX packets:752833 errors:36 dropped:0 overruns:0 carrier:36
          collisions:0 txqueuelen:0
          RX bytes:18081642 (17.2 MiB)  TX bytes:1139323399 (1.0 GiB)

# ovs-vsctl show
2dce25d0-ac18-4626-905d-c4995464785d
    Bridge "ovsbr0"
        Port "vnet0"
            Interface "vnet0"
        Port "vxlan0"
            Interface "vxlan0"
        Port "vnet1"
            Interface "vnet1"
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
    ovs_version: "1.7.0"

另外一台机器上基本上完全相同，结果如下：

# ./ip/ip -d li show vxlan0
8: vxlan0:  mtu 1450 qdisc noqueue
state UNKNOWN mode DEFAULT
    link/ether 92:8b:0f:8b:31:f1 brd ff:ff:ff:ff:ff:ff
    vxlan id 42 group 239.1.1.1 local 10.16.43.215 dev eth0 port 32768
61000 ageing 300
# ifconfig vxlan0
vxlan0    Link encap:Ethernet  HWaddr 92:8B:0F:8B:31:F1
          inet6 addr: fe80::908b:fff:fe8b:31f1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:752833 errors:0 dropped:0 overruns:0 frame:0
          TX packets:346770 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1128783737 (1.0 GiB)  TX bytes:22940562 (21.8 MiB)

# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr E8:39:35:30:1D:1E
          inet addr:10.16.43.215  Bcast:10.16.47.255  Mask:255.255.248.0
          inet6 addr: 2620:52:0:102f:ea39:35ff:fe30:1d1e/64 Scope:Global
          inet6 addr: fe80::ea39:35ff:fe30:1d1e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1535775 errors:0 dropped:0 overruns:0 frame:0
          TX packets:346951 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1210625058 (1.1 GiB)  TX bytes:41681835 (39.7 MiB)
          Interrupt:17

# ovs-vsctl show
71f0f455-ccc8-4781-88b2-4b663dd48c5f
    Bridge "ovsbr0"
        Port "vnet1"
            Interface "vnet1"
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
        Port "vnet0"
            Interface "vnet0"
        Port "vxlan0"
            Interface "vxlan0"
    ovs_version: "1.7.0"

配置好了之后，你可以尝试用 tcpdump 在 eth0 上捕捉一下包试试，你会发现 L2 是直接封装到 UDP 包中的：

05:39:05.107906 IP 10.16.43.215.52908 > 10.16.43.214.otv: UDP, length
106
        0x0000:  ac16 2d05 143a e839 3530 1d1e 0800 4500
        0x0010:  0086 abd0 0000 4011 62ca 0a10 2bd7 0a10
        0x0020:  2bd6 ceac 2118 0072 0000 0800 0000 0000
        0x0030:  2a00 5254 00f7 e2e7 5254 0090 5117 0800
        0x0040:  4500 0054 0000 4000 4001 f152 c0a8 6402
        0x0050:  c0a8 6403 0800 073c 3e05 0009 ca8f 0f51
        0x0060:  0000 0000 1102 0900 0000 0000 1011 1213
        0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223
        0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233
        0x0090:  3435 3637