How-to: Build VPP FD.IO with Mellanox DPDK PMD on top CentOS 7.7 with inbox drivers.
目录
References
Prerequisites
Installation, Compilation and Configuration
This short document will guide you how to compile and install VPP with Mellanox DPDK PMD over a Mellanox ConnectX-4 and ConnectX-5 interfaces.
References
- Performance tuning for VPP - https://wiki.fd.io/view/VPP/How_To_Optimize_Performance_(System_Tuning)
- Routing and Switching tutorial - https://wiki.fd.io/view/VPP/Tutorial_Routing_and_Switching
- VPP CLI guide - https://wiki.fd.io/view/VPP/Command-line_Interface_(CLI)_Guide
- Compiling VPP 17.10 with OFED: https://community.mellanox.com/s/article/how-to-build-vpp-fd-io--160--development-environment-with-mellanox-dpdk-pmd-for-connectx-4-and-connectx-5
Prerequisites
1. Install CentOS 7.7
2. Set up an account to use the Linux Foundation resources. To set up the account, refer to The Linux Foundation website: https://identity.linuxfoundation.org.
3. Set up a Gerrit account and apply an ssh public key. Follow the instructions on the FD.IO wiki web page: https://wiki.fd.io/view/DEV/Setting_up_Gerrit#Log_into_Gerrit
4. Install the following prerequisites packages:
yum install -y git cpp gcc rpm-build openssl-devel libmnl-devel numactl-devel epel-release net-tools rdma-core-devel nasmInstallation, Compilation and Configuration
1. Download VPP main master branch.
git clone ssh://@gerrit.fd.io:29418/vpp.git
git checkout origin/stable/<"Release_version_number"> - Use your Linux Foundation user account name (see Prerequisites above) instead of
as shows in above example. - To find desired <"Release_version_number"> do: "git -r branch" form VPP root git directory.
2. For VPP 18.07 ONLY: Enable the mlx5 PMD driver compilation as follow:
cd /vpp
sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk 3. Compile and install VPP with DLopen linkage
cd /vpp
make install-dep
make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=y
## Copy manually newly compiled shared module: ##
cp /opt/vpp/external/x86_64/lib/librte_pmd_mlx5_glue.so* /usr/lib64/
## For VPP 18.07 ONLY, check vpp-dpdk-devel package is installed: ##
rpm -qa | grep vpp-dpdk-devel
## Compile new binary: ##
make build-release vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y
## Building RPM packages: ##
make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y
## Install VPP RPMs: ##
rpm -ivh build-root/vpp-sel*.rpm
rpm -ivh build-root/vpp-lib*.rpm
rpm -ivh build-root/vpp-19*.rpm
rpm -ivh build-root/vpp-plug*.rpm 4. Compile and install without DLopen linkage
cd /vpp
make install-dep
make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n
## Compile new binary: ##
make build-release DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n
## Building RPM packages: ##
make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=n
## Install VPP RPMs: ##
rpm -ivh build-root/vpp-sel*.rpm
rpm -ivh build-root/vpp-lib*.rpm
rpm -ivh build-root/vpp-19*.rpm
rpm -ivh build-root/vpp-plug*.rpm 5. Edit the /etc/vpp/startup.conf start file to include the following:
Important: Uncomment no-mulit-seg parameter. It disables jumbo packets handling and improves performances significantly.
File
unix {
nodaemon
log /tmp/vpp.log
full-coredump
}
dpdk {
dev 0000:07:00.0
dev 0000:07:00.1
no-multi-seg
dev default {
num-rx-queues 2
}
}Insert PCI slot numbers of ConnectX-4/5 interfaces with correct IDs at dpdk { dev } section, as above example.
To verify the correct IDs use:
lspci -nn | grep -i mel
07:00.0 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]
07:00.1 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]6. Start VPP.
vpp -c /etc/vpp/startup.conf7. Enter the VPP Command Line Interface.
vppctl8. Configure L3 interfaces as shown in the example below:
VPP Cli
_______ _ _ _____ ___
__/ __/ _ \ (_)__ | | / / _ \/ _ \
_/ _// // / / / _ \ | |/ / ___/ ___/
/_/ /____(_)_/\___/ |___/_/ /_/
vpp# set int ip address HundredGigabitEthernet7/0/0 1.1.1.1/24
vpp# set interface state HundredGigabitEthernet7/0/0 up
vpp# set int ip address HundredGigabitEthernet7/0/1 2.2.2.2/24
vpp# set interface state HundredGigabitEthernet7/0/1 up
vpp# show int
Name Idx State Counter Count
HundredGigabitEthernet7/0/0 1 up
HundredGigabitEthernet7/0/1 2 up
vpp# show int address
HundredGigabitEthernet7/0/0 (up):
1.1.1.1/24
HundredGigabitEthernet7/0/1 (up):
2.2.2.2/24
local0 (dn):