Lesson Contents
In the first two VXLAN lessons, we used static ingress replication and multicast to learn MAC addresses. These are two flood-and-learn options that operate on the data plane. In this lesson, we’ll examine how to use MP-BGP EVPN to learn MAC addresses for VXLAN.
VXLAN MP-BGP EVPN separates the control plane (MP-BGP for MAC/IP advertisement) from the data plane (VXLAN encapsulation). We use MP-BGP to advertise MAC and IP address information between leaf switches. This eliminates the need for flood-and-learn mechanisms for unicast traffic and is a more efficient and scalable option.
EVPN uses a route distinguisher and route-target, similar to MPLS VPN. The RD is what makes a route unique. The RT defines where we want to export and import these routes. On Cisco NX-OS, the RD and RT are automatically generated by default.
Despite MP-BGP EVPN’s advantages, we still need something for Broadcast, Unknown Unicast, and Multicast (BUM) traffic. You’ll need to configure static ingress replication or multicast for BUM traffic. In this lesson, I’ll explain how to configure MP-BGP EVPN, and we’ll use multicast for BUM traffic.
Configuration
Here is the topology I’ll use:
I’m using Cisco Nexus 9300v switches running version 9.3(9). The hosts are simple Ubuntu Docker containers. The spine switch will be the BGP route reflector. I already configured the underlay network with OSPF, PIM, and the spine switch as the RP.
Configurations
Want to take a look for yourself? Here, you will find the startup configuration of each device.
SPINE1
hostname SPINE1
feature ospf
feature pim
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
interface Ethernet1/1
no switchport
mac-address 0050.c253.1001
ip address 192.168.12.1/24
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
no shutdown
interface Ethernet1/2
no switchport
mac-address 0050.c253.1002
ip address 192.168.13.1/24
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
no shutdown
interface loopback0
ip address 1.1.1.1/32
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
icam monitor scale
router ospf 1
LEAF1
hostname LEAF1
feature ospf
feature pim
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
interface Ethernet1/1
no switchport
mac-address 0050.c253.2001
ip address 192.168.12.2/24
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
no shutdown
interface Ethernet1/2
switchport access vlan 10
interface loopback0
ip address 2.2.2.2/32
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
router ospf 1
LEAF2
hostname LEAF2
feature ospf
feature pim
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
interface Ethernet1/1
no switchport
mac-address 0050.c253.3001
ip address 192.168.13.3/24
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
no shutdown
interface Ethernet1/2
switchport access vlan 10
interface loopback0
ip address 3.3.3.3/32
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
router ospf 1
MP-BGP
Let’s start with the MP-BGP EVPN configuration. There are two global items we need to configure:
SPINE1, LEAF1 & LEAF2
(config)# feature bgp
(config)# nv overlay evpn
This is why we need to enable these two commands:
- You need
feature bgp
so you can use therouter bgp
command. - You need
nx overlay evpn
otherwise, you can’t configureaddress-family l2vpn evpn
underrouter bgp
.
Now, we can configure MP-BGPN. I’ll start with the spine switch:
SPINE1(config)# router bgp 123
SPINE1(config-router)# neighbor 2.2.2.2
SPINE1(config-router-neighbor)# remote-as 123
SPINE1(config-router-neighbor)# update-source loopback 0
SPINE1(config-router-neighbor)# address-family l2vpn evpn
SPINE1(config-router-neighbor-af)# send-community
SPINE1(config-router-neighbor-af)# send-community extended
SPINE1(config-router-neighbor-af)# route-reflector-client
SPINE1(config-router)# neighbor 3.3.3.3
SPINE1(config-router-neighbor)# remote-as 123
SPINE1(config-router-neighbor)# update-source loopback 0
SPINE1(config-router-neighbor)# address-family l2vpn evpn
SPINE1(config-router-neighbor-af)# send-community
SPINE1(config-router-neighbor-af)# send-community extended
SPINE1(config-router-neighbor-af)# route-reflector-client
This turns SPINE1 into a route reflector for the L2VPN EVPN address family. We also need extended communities enabled.
We’ll configure the leaf switches as route reflector clients:
LEAF1 & LEAF2
(config)# router bgp 123
(config-router)# neighbor 1.1.1.1
(config-router-neighbor)# remote-as 123
(config-router-neighbor)# update-source loopback 0
(config-router-neighbor)# address-family l2vpn evpn
(config-router-neighbor-af)# send-community
(config-router-neighbor-af)# send-community extended
that’s all for bgp…
VNI and NVE
Let’s configure the VNI and NVE interface. We’ll do this on both leaf switches:
LEAF1 & LEAF2
(config)#feature vn-segment-vlan-based
(config)#feature nv overlay
(config)# vlan 10
(config-vlan)# vn-segment 10010
(config-vlan)# exit
(config)# interface nve1
(config-if-nve)# no shutdown
(config-if-nve)# host-reachability protocol bgp
(config-if-nve)# source-interface loopback 0
(config-if-nve)# member vni 10010
(config-if-nve-vni)# mcast-group 239.1.1.1
This configuration is exactly the same as we have seen previously, with one exception. We now use the host-reachability protocol bgp
command. This tells the leaf switches to learn MAC addresses in the control plane (using MP-BGP) and not on the data plane.
This completes our configuration.
Verification
Let’s verify our work. The NVE interface is up:
LEAF1# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 10010 239.1.1.1 Up CP L2 [10]
Note that the mode now shows up as Control Plane (CP). Let’s send some traffic between the hosts:
What if the spine switch had a different AS number?
Hello @dimasgb ,
There are some pros and cons of using eBGP or iBGP on the fabric. I’ll create an entire lesson for this with some other options for the underlay network. I’ll add a link here when it’s finished.
Hello Amit
Indeed MP-BGP EVPN resolves the issue of the flood and learn mechanism that multicast uses, enabling a more efficient communication using VXLAN. However, MP-BGP EVPN functions only for unicast traffic. Many services and devices will still be using BUM traffic (Broadcast, Unknown Unicast, Multicast), and this kind of traffic cannot be dealt with just using MP-BGP EVPN. There
... Continue reading in our forumHello Amit
Yes that’s correct, MP-BGP EVPN alone doesn’t natively handle BUM traffic. That’s why the multicast underlay must be configured in addition. It works alongside MP-BGP EVPN to handle the BUM traffic.
I hope this has been helpful!
Laz
Hello There is a line I never gave attention to, and now that I’m trying to understand it, i can’t find the logic ^^
it is :
Flags: (0x000102) (high32 00000000) on xmit-list, is not in l2rib/evpn
Especially the
not in l2rib
, for a locally generated route. A local route injected into evpn comes necessarily from L2RIB. There must be a logic, but i can’t see itthanks a lot in advance
cheers
Imen