SDN in Action: Practice VXLAN L2 Gateway with GNS3 and CSR1000v
Virtual Extensible LAN (VXLAN) is a popular encapsulation protocol for running an overlay network, allowing us to create a Layer 2 network on top of a Layer 3 network. In a cloud computing environement, each tenant requires its own logical network and its network ID. Traditionally VLANs are used to isolate apps and tenants, but its limitation of 4K network IDs severely restricts its adoption in a large cloud computing environment. The primary goal of VXLAN is to extend the network address space to 16 million by adding a 24-bit segment ID. In addtion, with the MAC-in-UDP encapsulation, millions of isolated Layer 2 VXLAN networks can co-exist on a common and long-distance Layer 3 infrastructure. Nowadays, VXLAN has been widely deployed not only in the DC and Cloud environment, but also in the wide area networks.
Today we are going to build a mini-lab environment with GNS3 and CSR1000v, and get some hands-on experience with VXLAN. Below is the network topology and desgin, we will create 2 Layer 2 VXLAN networks on top of a Layer 3 underlay network:
-VXLAN 4096 with VPCS-1, VPCS-2 and VPCS-3, 10.110.0.0/16;
-VXLAN 4097 with VPCS-4 and VPCS-5, 10.111.0.0/16.
- Setup the mini-lab environment
- Understand VXLAN and its configuration on CSR1000v
- Configure VXLAN L2 Gateway: Multicast Mode
- Configure VXLAN L2 Gateway: Unicast Mode
Setup the mini-lab environment
Enable ‘Intel VT-x/EPT’ in the VM settings, and install Ubuntu:
gset@ubuntu:~$ egrep -o '(vmx|svm)' /proc/cpuinfo
vmx
vmx
gset@ubuntu:~$ sudo apt-get update
gset@ubuntu:~$ sudo apt-get upgrade
gset@ubuntu:~$ sudo apt-get dist-upgrade
Install, reboot and run GNS3:
https://docs.gns3.com/1QXVIihk7dsOL7Xr7Bmz4zRzTsJ02wklfImGuHwTlaA4/index.html
gset@ubuntu:~$ sudo add-apt-repository ppa:gns3/ppa
gset@ubuntu:~$ sudo apt-get update
gset@ubuntu:~$ sudo apt-get install gns3-gui
// Should non-superusers be able to run GNS3 ? Yes
// Should non-superusers be able to capture packages ? Yes
gset@ubuntu:~$ reboot
gset@ubuntu:~$ gns3
Download the appliance file for CSR1000v and its image:
- ‘cisco-csr1000v.gns3a’ ;
- ‘csr1000v-universalk9.16.07.01-serial.qcow2’.
https://docs.gns3.com/appliances/cisco-csr1000v.html#appliance_supported
Import ‘cisco-csr1000v.gns3a’ to GNS3 and run the appliance on the local computer:
Create a new version for the appliance and import its image – ‘csr1000v-universalk9.16.07.01-serial.qcow2’:
Customize the template of Qemu VM:
Edit > Preferences > QEMU > Qemu VMs > Edit
Install Konsole:
gset@ubuntu:~$ sudo apt-get install konsole
Config GNS3 to use Konsole insteadl gnome-terminal:
Edit > > Preferences > General > Console applications
gnome-terminal -t "%d" -e "telnet %h %p" -----> /usr/bin/konsole --new-tab -p tabtitle=%d -e telnet %h %p
Run CSR1000v the first time: 'CSR1000v - packages.conf'
Would you like to enter the initial configuration dialog? [yes/no]:no
Would you like to terminate autoinstall? [yes]:yes
Router>en
Router#conf
Configuring from terminal, memory, or network [terminal]?
Router(config)#hostname r1
r1(config)#exit
r1#write
Building configuration...
[OK]
r1#
r1#reload
Proceed with reload? [confirm]
Understand VXLAN and its configuration on CSR1000v
Based on the data plane MAC address learning and flooding of multidestination traffic using IP multicast or unicast, CSR1000v can act as a Layer 2 VXLAN gateway and provide support to bridge traffic across VxLAN segments and Ethernet/VLANs.
CSR1000v implements the Virtual Tunnel Endpoint (VTEP) function, which encapsulates the Layer 2 packet on the IP/UDP tunnel with the VXLAN header (VNI) information before sending it with multicast or unicast, and after receiving the packet, removes the VXLAN header, identifies the bridge domain under which the VNI is configured and then bridges the inner L2 packet to the Ethernet/VLAN side.
An NVE interface is the overlay interface that terminates VXLAN tunnels. Multiple VXLAN VNIs can be assoicated with a NVE interface.
Bridge domain represents a Layer 2 broadcast domain with a set of logical ports, and by default, it maintains its own forwarding database of MAC.
Service-instance is to bind some traffic based to their tag(s) or VLAN(s) to a service (L2 or L3), and connects a bridge domain to a physical or logical port.
The most commonly used VXLAN control plane is multicast. When configuring VXLAN with multicast control plane, every node configured with a VTEP for a VXLAN with certain VNI will join the same multicast group, and multicast configuration must support Any-Source Multicast (ASM). With the head-end replication, VXLAN can also support the unicast-only mode.
Configure VXLAN L2 Gateway: Multicast Mode
Configuration for VPCS |
VXLAN: 4096
VPCS> set pcname VPCS-1 VPCS-1> ip 10.110.0.1 255.255.0.0
VPCS-2> set pcname VPCS-2 VPCS-2> ip 10.110.0.2 255.255.0.0
VPCS-3> set pcname VPCS-3 VPCS-3> ip 10.110.0.3 255.255.0.0 |
VXLAN: 4097
VPCS-4> set pcname VPCS-4 VPCS-4> ip 10.111.0.1 255.255.0.0
VPCS-5> set pcname VPCS-4 VPCS-5> ip 10.111.0.2 255.255.0.0 |
show ip save ping |
Configuration for routers – r1,r2,r3,r4 |
||
r2#show run Building configuration... ! hostname r2 ip multicast-routing distributed ! interface Loopback100 ip address 100.100.100.100 255.255.255.255 ip pim sparse-mode ip ospf 100 area 100 ! interface GigabitEthernet2 ip address 12.1.1.2 255.255.255.0 ip pim sparse-mode ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface GigabitEthernet3 ip address 11.1.1.2 255.255.255.0 ip pim sparse-mode ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface GigabitEthernet4 ip address 10.1.1.2 255.255.255.0 ip pim sparse-mode ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! router ospf 100 router-id 2.2.2.2 ! ip pim bidir-enable ip pim rp-address 100.100.100.100 bidir |
||
r1#show run Building configuration... ! hostname r1 ip multicast-routing distributed ! bridge-domain 10 member vni 4096 // Apply the service instance member GigabitEthernet1 service-instance 10 ! bridge-domain 11 member vni 4097 // Apply the service instance member GigabitEthernet2 service-instance 11 ! interface Loopback100 ip address 10.10.10.10 255.255.255.255 ip pim sparse-mode ip ospf 100 area 100 ! interface GigabitEthernet1 no ip address negotiation auto no mop enabled no mop sysid // define a service instance service instance 10 ethernet encapsulation untagged ! interface GigabitEthernet2 no ip address negotiation auto no mop enabled no mop sysid // define a service instance service instance 11 ethernet encapsulation untagged ! interface GigabitEthernet4 ip address 10.1.1.1 255.255.255.0 ip pim sparse-mode ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface nve1 no ip address source-interface Loopback100 // 1 multicast distribution tree for multiple VNIs member vni 4096 mcast-group 225.1.1.1 member vni 4097 mcast-group 225.1.1.1 no mop enabled no mop sysid ! router ospf 100 router-id 1.1.1.1 ! ip pim bidir-enable ip pim rp-address 100.100.100.100 bidir |
r3#show run Building configuration... ! hostname r3 ip multicast-routing distributed ! bridge-domain 10 member vni 4096 // Apply the service instance member GigabitEthernet1 service-instance 10 ! bridge-domain 11 member vni 4097 // Apply the service instance member GigabitEthernet2 service-instance 11 ! interface Loopback100 ip address 11.11.11.11 255.255.255.255 ip pim sparse-mode ip ospf 100 area 100 ! interface GigabitEthernet1 no ip address negotiation auto no mop enabled no mop sysid // define a service instance service instance 10 ethernet encapsulation untagged ! interface GigabitEthernet2 no ip address negotiation auto no mop enabled no mop sysid // define a service instance service instance 11 ethernet encapsulation untagged ! interface GigabitEthernet3 ip address 11.1.1.1 255.255.255.0 ip pim sparse-mode ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface nve1 no ip address source-interface Loopback100 // 1 multicast distribution tree for multiple VNIs member vni 4096 mcast-group 225.1.1.1 member vni 4097 mcast-group 225.1.1.1 no mop enabled no mop sysid ! router ospf 100 router-id 3.3.3.3 ! ip pim bidir-enable ip pim rp-address 100.100.100.100 bidir |
r4#show run Building configuration... ! hostname r4 ip multicast-routing distributed ! bridge-domain 10 member vni 4096 // Apply the service instance member GigabitEthernet1 service-instance 10 !
interface Loopback100 ip address 12.12.12.12 255.255.255.255 ip pim sparse-mode ip ospf 100 area 100 ! interface GigabitEthernet1 no ip address negotiation auto no mop enabled no mop sysid // define a service instance service instance 10 ethernet encapsulation untagged !
interface GigabitEthernet2 ip address 12.1.1.1 255.255.255.0 ip pim sparse-mode ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface nve1 no ip address source-interface Loopback100 // 1 multicast distribution tree for multiple VNIs member vni 4096 mcast-group 225.1.1.1
no mop enabled no mop sysid ! router ospf 100 router-id 4.4.4.4 ! ip pim bidir-enable ip pim rp-address 100.100.100.100 bidir
|
r1#show nve vni
r1#show nve peers
r1#show nve inter nve 1 detail
r1#show ip route
r2#show ip route
r1#show ip mroute
r2#show ip igmp interface loopback 100
r2#show ip pim neighbor
r2#show ip pim rp
VPCS-1> ping 10.110.0.1
VPCS-1> ping 10.110.0.2
VPCS-1> ping 10.110.0.3
VPCS-4> ping 10.111.0.1
VPCS-4> ping 10.111.0.2
Configure VXLAN L2 Gateway: Unicast Mode
Configuration for routers – r1,r2,r3,r4 |
||
r2#show run Building configuration... ! hostname r2 ! interface Loopback100 ip address 100.100.100.100 255.255.255.255 ip ospf 100 area 100 ! interface GigabitEthernet2 ip address 12.1.1.2 255.255.255.0 ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface GigabitEthernet3 ip address 11.1.1.2 255.255.255.0 ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface GigabitEthernet4 ip address 10.1.1.2 255.255.255.0 ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! router ospf 100 router-id 2.2.2.2 ! |
||
r1#show run Building configuration... ! hostname r1 ! bridge-domain 10 member vni 4096 member GigabitEthernet1 service-instance 10 ! bridge-domain 11 member vni 4097 member GigabitEthernet2 service-instance 11 ! interface Loopback100 ip address 10.10.10.10 255.255.255.255 ip ospf 100 area 100 ! interface GigabitEthernet1 no ip address negotiation auto no mop enabled no mop sysid service instance 10 ethernet encapsulation untagged ! interface GigabitEthernet2 no ip address negotiation auto no mop enabled no mop sysid service instance 11 ethernet encapsulation untagged ! interface GigabitEthernet4 ip address 10.1.1.1 255.255.255.0 ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface nve1 no ip address source-interface Loopback100 // Head-end replication enables unicast-only mode member vni 4096 ingress-replication 11.11.11.11 ingress-replication 12.12.12.12 member vni 4097 ingress-replication 11.11.11.11 no mop enabled no mop sysid ! router ospf 100 router-id 1.1.1.1 ! |
r3#show run Building configuration... ! hostname r3 ! bridge-domain 10 member vni 4096 member GigabitEthernet1 service-instance 10 ! bridge-domain 11 member vni 4097 member GigabitEthernet2 service-instance 11 ! interface Loopback100 ip address 11.11.11.11 255.255.255.255 ip ospf 100 area 100 ! interface GigabitEthernet1 no ip address negotiation auto no mop enabled no mop sysid service instance 10 ethernet encapsulation untagged ! interface GigabitEthernet2 no ip address negotiation auto no mop enabled no mop sysid service instance 11 ethernet encapsulation untagged ! interface GigabitEthernet3 ip address 11.1.1.1 255.255.255.0 ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface nve1 no ip address source-interface Loopback100 // Head-end replication enables unicast-only mode member vni 4096 ingress-replication 10.10.10.10 ingress-replication 12.12.12.12 member vni 4097 ingress-replication 10.10.10.10 no mop enabled no mop sysid ! router ospf 100 router-id 3.3.3.3 !
|
r4#show run Building configuration... ! hostname r4 ! bridge-domain 10 member vni 4096 member GigabitEthernet1 service-instance 10 ! ! ! ! ! ! interface Loopback100 ip address 12.12.12.12 255.255.255.255 ip ospf 100 area 100 ! interface GigabitEthernet1 no ip address negotiation auto no mop enabled no mop sysid service instance 10 ethernet encapsulation untagged !
interface GigabitEthernet2 ip address 12.1.1.1 255.255.255.0 ip ospf 100 area 100 negotiation auto no mop enabled no mop sysid ! interface nve1 no ip address source-interface Loopback100 // Head-end replication enables unicast-only mode member vni 4096 ingress-replication 10.10.10.10 ingress-replication 11.11.11.11
no mop enabled no mop sysid ! router ospf 100 router-id 4.4.4.4 !
|
r1#show nve vni
r1#show nve peers
r1#show nve inter nve 1 detail
r4#show nve vni
r4#show nve peers
r4#show nve inter nve 1 detail
GNS3 Installation on Linux:
https://docs.gns3.com/1QXVIihk7dsOL7Xr7Bmz4zRzTsJ02wklfImGuHwTlaA4/index.html
Cisco CSR1000v appliance:
https://docs.gns3.com/appliances/cisco-csr1000v.html#appliance_supported
Cisco CSR 1000V VxLAN Support:
https://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/vxlan/m_csr-vxlan-support-book.html
VXLAN Basics:
http://stayinginit.blogspot.com/2014/02/vxlan-basics.html
Networking Basics As I Currently Understand
http://stayinginit.blogspot.in/2014/10/vxlan-unicast.html