DPDK系列之七：OVS-DPDK的在CentOS安装和测试
一、前言
根据文章《DPDK系列之六：qemu-kvm网络后端的加速技术》，OVS-DPDK可以给虚拟机的部署密度和虚拟机网络性能的提升带来帮助。本文将介绍如何编译和安装DPDK以及如何使用DPDK提升网络性能：
- OS：CentOS 7.4.1708
- OVS：2.8.1
- DPDK：17.08.1
转载自https://blog.csdn.net/cloudvtech
二、操作系统和DPDK的安装
2.1 系统配置
硬件
1 NUMA node with 4 CPU cores
8 GB memory
3 NICs
40 GB disk操作系统
cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
[root@ovs-dpdk ~]# uname -a
Linux ovs-dpdk 3.10.0-693.11.1.el7.x86_64 #1 SMP Mon Dec 4 23:52:40 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux2.2 安装基础依赖软件包
yum makecache
yum -y update
yum install -y epel-release
yum install -y net-tools tcpdump telnet wget zip unzip vim
yum install -y gcc gcc-c++ kernel-devel kernel-headers kernel.x86_64 net-tools
yum install -y numactl-devel.x86_64 numactl-libs.x86_64
yum install -y libpcap.x86_64 libpcap-devel.x86_64
yum install -y pciutils
yum install -y autoconf automake libtool2.3 编译DPDK
wget http://fast.dpdk.org/rel/dpdk-17.08.1.tar.xz
tar -xvf dpdk-17.08.1.tar
cd dpdk-stable-17.08.1/
patch lib/librte_eal/linuxapp/igb_uio/igb_uio.c
export RTE_SDK=/root/dpdk-stable-17.08.1
export RTE_TARGET=x86_64-native-linuxapp-gcc
make config T=x86_64-native-linuxapp-gcc
sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config
add in config/common_linuxapp : CONFIG_RTE_BUILD_COMBINE_LIBS=y
make
cd $RTE_SDK
make install T=x86_64-native-linuxapp-gcc绑定网卡
ifconfig ens38 down
modprobe uio_pci_generic
modprobe uio
insmod /root/dpdk-stable-17.08.1/build/kmod/igb_uio.ko
/root/dpdk-stable-17.08.1/usertools/dpdk-devbind.py --bind=igb_uio 0000:02:06.0
[root@ovs-dpdk openvswitch-2.8.1]# /root/dpdk-stable-17.08.1/usertools/dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=vfio-pci,uio_pci_generic
Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio,vfio-pci,uio_pci_generic *Active*
0000:02:05.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens37 drv=e1000 unused=igb_uio,vfio-pci,uio_pci_generic *Active*三、编译和运行OVS
3.1 编译OVS-DPDK
wget http://openvswitch.org/releases/openvswitch-2.8.1.tar.gz
tar -zxvf openvswitch-2.8.1.tar.gz
cd openvswitch-2.8.1/
./boot.sh
./configure --with-dpdk=/root/dpdk-stable-17.08.1/x86_64-native-linuxapp-gcc/ CFLAGS="-g -Ofast”
make 'CFLAGS=-g -Ofast -march=native' -j10
make install3.2 配置OVS-DPDK
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /root/openvswitch-2.8.1/vswitchd/vswitch.ovsschema
ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x03
export DB_SOCK=/usr/local/var/run/openvswitch/db.sock3.3 运行OVS-DPDK
[root@ovs-dpdk openvswitch-2.8.1]# /root/openvswitch-2.8.1/utilities/ovs-ctl --no-ovsdb-server --db-sock="$DB_SOCK" start
Starting ovs-vswitchd Zone 0: name:, phys:0xa1fb6cc0, len:0x34000, virt:0x7f2b74bb6cc0, socket_id:0, flags:0
[ OK ]
Enabling remote OVSDB managers [ OK ] 转载自https://blog.csdn.net/cloudvtech
四、创建基于DPDK的OVS端口
4.1 创建一个基于宿主机网卡的DPDK端口
ovs-vsctl add-port br0 myportnameone -- set Interface myportnameone type=dpdk options:dpdk-devargs=0000:02:06.0
4.2 添加另外一个基于宿主机网卡的端口
ifconfig ens37 down
/root/dpdk-stable-17.08.1/usertools/dpdk-devbind.py --bind=igb_uio 0000:02:05.0
/root/dpdk-stable-17.08.1/usertools/dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
0000:02:05.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=vfio-pci,uio_pci_generic
0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=vfio-pci,uio_pci_generic
Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio,vfio-pci,uio_pci_generic *Active*
ovs-vsctl add-port br0 myportnametwo -- set Interface myportnametwo type=dpdk options:dpdk-devargs=0000:02:05.0对应的日志信息：
ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
2018-01-01T16:04:45.327Z|00044|memory|INFO|46340 kB peak resident set size after 115.7 seconds
2018-01-01T16:04:45.334Z|00045|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports recirculation
2018-01-01T16:04:45.334Z|00046|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN header stack length probed as 1
2018-01-01T16:04:45.334Z|00047|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS label stack length probed as 3
2018-01-01T16:04:45.334Z|00048|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports truncate action
2018-01-01T16:04:45.334Z|00049|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports unique flow ids
2018-01-01T16:04:45.334Z|00050|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports clone action
2018-01-01T16:04:45.334Z|00051|ofproto_dpif|INFO|netdev@ovs-netdev: Max sample nesting level probed as 10
2018-01-01T16:04:45.334Z|00052|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports eventmask in conntrack action
2018-01-01T16:04:45.334Z|00053|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_state
2018-01-01T16:04:45.334Z|00054|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_zone
2018-01-01T16:04:45.334Z|00055|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_mark
2018-01-01T16:04:45.334Z|00056|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_label
2018-01-01T16:04:45.334Z|00057|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_state_nat
2018-01-01T16:04:45.334Z|00058|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_orig_tuple
2018-01-01T16:04:45.334Z|00059|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_orig_tuple6
2018-01-01T16:04:45.344Z|00060|bridge|INFO|bridge br0: added interface br0 on port 65534
2018-01-01T16:04:45.345Z|00061|bridge|INFO|bridge br0: using datapath ID 00007e7fb388574d
2018-01-01T16:04:45.345Z|00062|connmgr|INFO|br0: added service controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
2018-01-01T16:04:55.351Z|00063|memory|INFO|peak resident set size grew 100% in last 10.0 seconds, from 46340 kB to 92756 kB
2018-01-01T16:04:55.351Z|00064|memory|INFO|handlers:2 ports:1 revalidators:2 rules:5
ovs-vsctl add-port br0 myportnametwo -- set Interface myportnametwo type=dpdk options:dpdk-devargs=0000:02:05.0
2018-01-01T16:05:44.914Z|00065|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 0 created.
2018-01-01T16:05:44.914Z|00066|dpif_netdev|INFO|There are 1 pmd threads on numa node 0
2018-01-01T16:05:45.052Z|00067|netdev_dpdk|INFO|Port 0: 00:0c:29:b9:82:0f
2018-01-01T16:05:45.053Z|00068|bridge|INFO|bridge br0: added interface myportnametwo on port 1
2018-01-01T16:05:45.055Z|00069|bridge|INFO|bridge br0: using datapath ID 0000000c29b9820f
2018-01-01T16:05:45.061Z|00070|netdev_dpdk|WARN|Failed to enable flow control on device 0
ovs-vsctl add-port br0 myportnameone -- set Interface myportnameone type=dpdk options:dpdk-devargs=0000:02:06.0
2018-01-01T16:07:08.295Z|00071|netdev_dpdk|INFO|Port 1: 00:0c:29:b9:82:19
2018-01-01T16:07:08.297Z|00072|bridge|INFO|bridge br0: added interface myportnameone on port 24.3 查看OVS信息
内存页信息
转载自https://blog.csdn.net/cloudvtech