VxLAN数据中心L2/L3互通（端到端）
VxLAN数据中心端到端方式实现L2/L3互连，这种实现方式可以使数据中心属于同一个EVPN-VXLAN域，相较于hand-off方式通过端到端实现数据中心L2互连可以满足Mac mobility、ARP suppression等特性。
实现思路
DC1的Border-Leaf和DC2的Border-Leaf之间运行EBGP EVPN协议传递MAC路由或者主机路由信息，不改变MAC路由或者主机路由的下一跳地址，从而在跨数据中心的VTEP之间建立端到端VXLAN隧道。
配置步骤
1.配置各节点IP地址。
2.配置路由实现各节点之间的互通。
3.在DC1和DC2内分别创建EBGP邻居，配置BGP EVPN协议创建VXLAN隧道。
4.WAN-PE之间L3互通，使两个DC之间的Lo0能互通。
5.DC1-BL与DC2-BL之间创建EBGP EVPN邻居。
拓扑图
这里起了两个vlan，并配置分布式网关，
VPC6、VPC35 两个主机在vlan100；veos55和veos56在vlan200，两个vlan属于同租户；
每个DC内有一台leaf、两台border-leaf做malg，剩下一台leaf忽略了。
WAN模拟相对简单只是L3互连，没有涉及到MPLS。
数据准备
IP
| DC1设备 | 接口 | IP | 对端设备 | 接口 | IP |
|---|---|---|---|---|---|
| DC1-Spine1 | eth1 | 10.1.2.1/30 | DC1-Leaf1 | eth1 | 10.1.2.2/30 |
| DC1-Spine1 | eth2 | 10.1.3.1/30 | DC1-Leaf2 | eth1 | 10.1.3.2/30 |
| DC1-Spine1 | eth3 | 10.1.4.1/30 | DC1-BL1 | eth1 | 10.1.4.2/30 |
| DC1-Spine1 | eth4 | 10.1.5.1/30 | DC1-BL2 | eth1 | 10.1.5.2/30 |
| DC1-Spine2 | eth1 | 10.2.2.1/30 | DC1-Leaf1 | eth2 | 10.2.2.2/30 |
| DC1-Spine2 | eth2 | 10.2.3.1/30 | DC1-Leaf2 | eth2 | 10.2.3.2/30 |
| DC1-Spine2 | eth3 | 10.2.4.1/30 | DC1-BL1 | eth2 | 10.2.4.2/30 |
| DC1-Spine2 | eth4 | 10.2.5.1/30 | DC1-BL2 | eth2 | 10.2.5.2/30 |
| DC1-BL1 | eth4 | 172.16.0.1/30 | DC1-BL2 | eth4 | 172.16.0.2/30 |
| DC1-Spine1 | Lo0 | 10.0.0.1/32 | DC1-Spine2 | Lo0 | 10.0.0.2/32 |
| DC1-Leaf1 | Lo0 | 10.0.0.3/32 | DC1-Leaf2 | Lo0 | 10.0.0.4/32 |
| DC1-BL1 | Lo0 | 10.0.0.5/32 | DC1-BL2 | Lo0 | 10.0.0.6/32 |
| DC1-BL1/2 | Lo1 | 1.1.1.1/32 BL1&2Lo1相同 | |||
| VPC6 | eth0 | 192.168.100.6 | VPC34 | eth0 | 192.168.100.34 |
| veos55 | Po3 | 192.168.200.55 |
| WAN设备 | 接口 | IP | 对端设备 | 接口 | IP |
|---|---|---|---|---|---|
| PE1 | Po2 | 172.17.0.0/31 | PE2 | Po2 | 172.17.0.1/31 |
| PE1 | eth1 | 172.31.0.1/31 | DC1-BL1 | eth3 | 172.31.0.0/31 |
| PE1 | eth2 | 172.32.0.1/31 | DC1-BL2 | eth3 | 172.32.0.0/31 |
| PE2 | eth1 | 172.33.0.0/31 | DC2-BL1 | eth3 | 172.33.0.1/31 |
| PE2 | eth2 | 172.34.0.0/31 | DC2-BL2 | eth3 | 172.34.0.1/31 |
| PE2 | Lo0 | 10.0.0.26/32 | PE2 | Lo0 | 10.0.0.50/32 |
| DC2设备 | 接口 | IP | 对端设备 | 接口 | IP |
|---|---|---|---|---|---|
| DC2-Spine1 | eth1 | 20.1.1.1/30 | DC2-BL1 | eth1 | 20.1.1.2/30 |
| DC2-Spine1 | eth2 | 20.1.2.1/30 | DC2-BL2 | eth1 | 20.1.2.2/30 |
| DC2-Spine1 | eth3 | 20.1.3.1/30 | DC2-Leaf1 | eth1 | 20.1.3.2/30 |
| DC2-Spine1 | eth4 | 20.1.4.1/30 | DC2-Leaf2 | eth1 | 20.1.4.2/30 |
| DC2-Spine2 | eth1 | 20.2.1.1/30 | DC2-BL1 | eth2 | 20.2.1.2/30 |
| DC2-Spine2 | eth2 | 20.2.2.1/30 | DC2-BL2 | eth2 | 20.2.2.2/30 |
| DC2-Spine2 | eth3 | 20.2.3.1/30 | DC2-Leaf1 | eth2 | 20.2.3.2/30 |
| DC2-Spine2 | eth4 | 20.2.4.1/30 | DC2-Leaf2 | eth2 | 20.2.4.2/30 |
| DC2-BL1 | eth4 | 192.168.2.1/30 | DC1-BL2 | eth4 | 192.168.2.2/30 |
| DC2-Spine1 | Lo0 | 10.0.0.51/32 | DC2-Spine2 | Lo0 | 10.0.0.52/32 |
| DC2-Leaf1 | Lo0 | 10.0.0.53/32 | DC2-Leaf2 | Lo0 | 10.0.0.54/32 |
| DC2-BL1 | Lo0 | 10.0.0.28/32 | DC1-BL2 | Lo0 | 10.0.0.29/32 |
| DC2-BL1/2 | Lo1 | 2.2.2.2/32 BL1&2Lo1相同 | |||
| VPC35 | eth0 | 192.168.100.35 | VPC7 | eth0 | 192.168.100.7 |
| veos56 | Po3 | 192.168.200.55 |
AS
| 设备 | AS | 设备 | AS |
|---|---|---|---|
| DC1-Spine1 | 100 | DC1-Spine2 | 99 |
| DC1-Leaf1 | 101 | DC1-Leaf2 | 102 |
| DC1-BL1 | 103 | DC1-BL2 | 104 |
| DCI-VTEP1 | 60026 | DCI-VTEP2 | 65050 |
| DC2-Spine1 | 65051 | DC2-Spine2 | 65052 |
| DC2-Leaf1 | 65053 | DC2-Leaf2 | 65054 |
| DC2-BL1 | 65028 | DC2-BL2 | 65029 |
详细配置
每个角色的配置各举一个。
Spine，以DC1-Spine1为例：
service routing protocols model multi-agent
hostname DC1-Spine1
spanning-tree mode none
interface Ethernet1
mtu 9200
no switchport
ip address 10.1.2.1/30
interface Ethernet2
mtu 9200
switchport access vlan 1800
no switchport
ip address 10.1.3.1/30
interface Ethernet3
mtu 9200
no switchport
ip address 10.1.4.1/30
interface Ethernet4
mtu 9200
no switchport
ip address 10.1.5.1/30
interface Loopback0
ip address 10.0.0.1/32
ip routing
router bgp 100
router-id 10.0.0.1
no bgp default ipv4-unicast
maximum-paths 128
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor underlay peer group
neighbor underlay maximum-routes 0
neighbor 10.0.0.3 peer group overlay
neighbor 10.0.0.3 remote-as 101
neighbor 10.0.0.4 peer group overlay
neighbor 10.0.0.4 remote-as 102
neighbor 10.0.0.5 peer group overlay
neighbor 10.0.0.5 remote-as 103
neighbor 10.0.0.6 peer group overlay
neighbor 10.0.0.6 remote-as 104
neighbor 10.1.2.2 peer group underlay
neighbor 10.1.2.2 remote-as 101
neighbor 10.1.3.2 peer group underlay
neighbor 10.1.3.2 remote-as 102
neighbor 10.1.4.2 peer group underlay
neighbor 10.1.4.2 remote-as 103
neighbor 10.1.5.2 peer group underlay
neighbor 10.1.5.2 remote-as 104
address-family e
neighbor overlay activate
address-family ipv4
neighbor underlay activate
network 10.0.0.1/32
Leaf，以DC1-Leaf1为例：
service routing protocols model multi-agent
hostname DC1-Leaf1
spanning-tree mode none
vlan 100,200
vrf instance 100
interface Ethernet1
mtu 9200
no switchport
ip address 10.1.2.2/30
interface Ethernet2
no switchport
ip address 10.2.2.2/30
interface Ethernet3
mtu 9200
switchport access vlan 100
interface Loopback0
ip address 10.0.0.3/32
interface Vlan100
vrf 100
ip address virtual 192.168.100.254/24
interface Vlan200
vrf 100
ip address virtual 192.168.200.254/24
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 100,200 vni 10100,10200
ip virtual-router mac-address 00:00:00:00:00:01
ip routing
ip routing vrf 100
router bgp 101
router-id 10.0.0.3
no bgp default ipv4-unicast
maximum-paths 128
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor underlay peer group
neighbor underlay maximum-routes 0
neighbor 10.0.0.1 peer group overlay
neighbor 10.0.0.1 remote-as 100
neighbor 10.0.0.2 peer group overlay
neighbor 10.0.0.2 remote-as 99
neighbor 10.1.2.1 peer group underlay
neighbor 10.1.2.1 remote-as 100
neighbor 10.2.2.1 peer group underlay
neighbor 10.2.2.1 remote-as 99
vlan-aware-bundle vlans-1
rd 1:101
route-target both 1:1
redistribute learned
vlan 100,200
address-family e
neighbor overlay activate
address-family ipv4
neighbor underlay activate
network 10.0.0.3/32
vrf 100
rd 100:1
route-target import e 100:100
route-target export e 100:100
redistribute connected
Border-Leaf，以DC1-BL1为例：
service routing protocols model multi-agent
hostname DC1-BL1
spanning-tree mode mstp
no spanning-tree vlan-id 4094
vlan 10,100,200
vlan 4094
name mlag
trunk group mlagpeer
vrf instance 100
interface Port-Channel3
mtu 9200
switchport trunk allowed vlan 100,200
switchport mode trunk
mlag 3
interface Ethernet1
mtu 9200
no switchport
ip address 10.1.4.2/30
interface Ethernet2
mtu 9200
no switchport
ip address 10.2.4.2/30
interface Ethernet3
mtu 9200
no switchport
ip address 172.31.0.0/31
interface Ethernet4
mtu 9200
switchport mode trunk
switchport trunk group mlagpeer
interface Loopback0
ip address 10.0.0.5/32
interface Loopback1
ip address 1.1.1.1/32
interface Vlan100
vrf 100
ip address virtual 192.168.100.254/24
interface Vlan200
vrf 100
ip address virtual 192.168.200.254/24
interface Vlan4094
ip address 172.16.0.1/30
interface Vxlan1
vxlan source-interface Loopback1
vxlan udp-port 4789
vxlan vlan 100,200 vni 10100,10200
ip virtual-router mac-address 00:00:00:00:00:01
ip routing
ip routing vrf 100
mlag configuration
domain-id mlag-domain
local-interface Vlan4094
peer-address 172.16.0.2
peer-link Ethernet4
dual-primary detection delay 5 action errdisable all-interfaces
router bgp 103
router-id 10.0.0.5
no bgp default ipv4-unicast
maximum-paths 128
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor underlay peer group
neighbor underlay maximum-routes 0
neighbor 10.0.0.1 peer group overlay
neighbor 10.0.0.1 remote-as 100
neighbor 10.0.0.2 peer group overlay
neighbor 10.0.0.2 remote-as 99
neighbor 10.0.0.28 peer group overlay
neighbor 10.0.0.28 remote-as 65028
neighbor 10.0.0.29 peer group overlay
neighbor 10.0.0.29 remote-as 65029
neighbor 10.1.4.1 peer group underlay
neighbor 10.1.4.1 remote-as 100
neighbor 10.2.4.1 peer group underlay
neighbor 10.2.4.1 remote-as 99
neighbor 172.16.0.2 remote-as 104
neighbor 172.31.0.1 peer group underlay
neighbor 172.31.0.1 remote-as 65026
vlan-aware-bundle vlans-1
rd 1:103
route-target both 1:1
redistribute learned
vlan 100,200
address-family e
neighbor overlay activate
address-family ipv4
neighbor underlay activate
neighbor 172.16.0.2 activate
network 1.1.1.1/32
network 10.0.0.5/32
vrf 100
rd 100:3
route-target import e 100:100
route-target export e 100:100
redistribute connected
WAN-PE，以PE1为例：
service routing protocols model multi-agent
hostname WAN-PE1
spanning-tree mode none
interface Port-Channel2
mtu 9200
no switchport
ip address 172.17.0.0/31
interface Ethernet1
mtu 9200
no switchport
ip address 172.31.0.1/31
interface Ethernet2
no switchport
ip address 172.32.0.1/31
interface Ethernet3
channel-group 2 mode active
interface Ethernet4
channel-group 2 mode active
interface Loopback0
ip address 10.0.0.26/32
ip routing
router bgp 65026
no bgp default ipv4-unicast
neighbor underlay-ebgp peer group
neighbor underlay-ebgp maximum-routes 0
neighbor 172.17.0.1 peer group underlay-ebgp
neighbor 172.17.0.1 remote-as 65050
neighbor 172.31.0.0 peer group underlay-ebgp
neighbor 172.31.0.0 remote-as 103
neighbor 172.32.0.0 peer group underlay-ebgp
neighbor 172.32.0.0 remote-as 104
address-family ipv4
neighbor underlay-ebgp activate
network 10.0.0.26/32
状态检查
DC1-BL1包括：DC内部BGP EVPN，两个DC-BL之间的BGP IPv4族，到WAN-PE1的BGP IPv4族，以及还有两个到DC2-BL的BGP EVPN 状态Established
DC1-Leaf1的远端vtep，2.2.2.2是DC2-BL，10.0.0.53是DC2-Leaf1，可以看到跨数据中心是端到端的隧道
DC1-Leaf1的mac table
DC1-Leaf1的arp table
DC1-Leaf1上看VPC35的MAC-IP路由，下一跳是DC2-Leaf1（10.0.0.53）
VPC6,ping 通本网段VPC35，ping通跨网段veos55、veos56
总结
DCI 端到端互连方案中，WAN-PE只是起到路由传递，和外层数据包转发的作用。
相比hand-off，端到端支持了Mac mobility、ARP suppression。