In the introduction to VXLAN lesson, I explained what VXLAN is and how it works. In this lesson, I’ll show you how to configure VXLAN where we use the multicast “flood and learn” system to learn the mapping between a VTEP IP address and a MAC address.
Here’s the topology we’ll use:
All devices are CSR1000V routers running Cisco IOS XE Software, version 16.06.01. I’m using CSR1000V routers since anyone can use these. I use custom MAC addresses because those are easy to recognize when we do a packet capture.
VTEP1 and VTEP2 are our VTEP devices. The core router is there to simulate our “IP network”. We are going to create a VXLAN tunnel with VNI 5012 so that H1 and H2 can communicate directly over layer 2.
I pre-configured OSPF so that we have connectivity between the VTEP devices and the core router.
Want to take a look for yourself? Here you will find the startup configuration of each device.
hostname CORE ! interface Loopback0 ip address 22.214.171.124 255.255.255.255 ! interface GigabitEthernet2 mac-address 0000.5e00.5303 ip address 192.168.13.3 255.255.255.0 ! interface GigabitEthernet3 mac-address 0000.5e00.5333 ip address 192.168.23.3 255.255.255.0 ! router ospf 1 network 126.96.36.199 0.0.0.0 area 0 network 192.168.13.0 0.0.0.255 area 0 network 192.168.23.0 0.0.0.255 area 0 ! end
hostname H1 ! interface GigabitEthernet2 mac-address 0000.5e00.5365 ip address 192.168.12.101 255.255.255.0 ! end
hostname H2 ! interface GigabitEthernet2 mac-address 0000.5e00.5366 ip address 192.168.12.102 255.255.255.0 ! end
hostname VTEP1 ! interface Loopback0 ip address 188.8.131.52 255.255.255.255 ! interface GigabitEthernet2 mac-address 0000.5e00.5301 ! interface GigabitEthernet3 mac-address 0000.5e00.5311 ip address 192.168.13.1 255.255.255.0 ! router ospf 1 network 184.108.40.206 0.0.0.0 area 0 network 192.168.13.0 0.0.0.255 area 0 ! end
hostname VTEP2 ! interface Loopback0 ip address 220.127.116.11 255.255.255.255 ! interface GigabitEthernet2 mac-address 0000.5e00.5302 ! interface GigabitEthernet3 mac-address 0000.5e00.5322 ip address 192.168.23.2 255.255.255.0 ! router ospf 1 network 18.104.22.168 0.0.0.0 area 0 network 192.168.23.0 0.0.0.255 area 0 ! end
Let’s start with the configuration of multicast. With VXLAN, we don’t have a typical scenario where we have a few sources and many receivers. All VTEP devices communicate with each other so it makes sense to use bidirectional PIM. The core router will be the RP in this network.
Let’s enable multicast routing and bidirectional PIM on all VTEP devices and the core router:
VTEP1, VTEP2 & CORE (config)#ip multicast-routing distributed (config)#ip pim bidir-enable
We need to enable PIM sparse mode on all physical interfaces that connect to the IP network:
VTEP1 & VTEP2 & CORE (config)#interface GigabitEthernet 3 (config-if)#ip pim sparse-mode
CORE(config)#interface GigabitEthernet 2 CORE(config-if)#ip pim sparse-mode
And don’t forget the loopback interfaces:
VTEP1, VTEP2 & CORE (config)#interface Loopback 0 (config-if)#ip pim sparse-mode
Last but not least, configure the RP address:
VTEP1, VTEP2 & CORE (config)#ip pim rp-address 22.214.171.124 bidir
This completes the multicast configuration.
We need to create a Network Virtualization Endpoint (NVE) interface. This is where we configure the VNI and multicast group that we will use. We source this interface from the loopback 0 interface, use VNI 5012, and use multicast group 126.96.36.199.
Here’s how to configure the NVE interface:
VTEP1 & VTEP2 (config)#interface NVE 1 (config-if)#no shutdown (config-if)#source-interface Loopback 0 (config-if)#member vni 5012 mcast-group 188.8.131.52
Now we need to configure the Ethernet Flow Point (EFP) service instance. This is a logical interface that connects a bridge domain to a physical port (or EtherChannel). Under the service instance, we configure whether the incoming traffic is tagged or untagged. In our case, the hosts send untagged traffic. This is how to configure it:
VTEP1 & VTEP2 (config)#interface GigabitEthernet 2 (config-if)#service instance 1 ethernet (config-if-srv)#encapsulation untagged (config-if-srv)#exit (config-if)#exit
Last but not least, we need to configure the Bridge Domain Interface (BDI):
This is where we combine the VNI, physical interface, and service-instance:
VTEP1 & VTEP2 (config)#bridge-domain 1 (config-bdomain)#member vni 5012 (config-bdomain)#member GigabitEthernet 2 service-instance 1
This completes our VXLAN configuration.
exitcommands on purpose because I configure the bridge-domain globally. You can also configure the bridge-domain under the service instance.
Let’s verify our work.
First, I’ll check if our multicast configuration is correct:
VTEP1#show ip mroute 184.108.40.206 IP Multicast Routing Table (*, 220.127.116.11), 00:00:36/00:02:25, RP 18.104.22.168, flags: BCx Bidir-Upstream: GigabitEthernet3, RPF nbr 192.168.13.3 Outgoing interface list: Tunnel0, Forward/Sparse-Dense, 00:00:36/00:02:25 GigabitEthernet3, Bidir-Upstream/Sparse, 00:00:36/stopped
VTEP2#show ip mroute 22.214.171.124 IP Multicast Routing Table (*, 126.96.36.199), 00:00:36/00:02:24, RP 188.8.131.52, flags: BCx Bidir-Upstream: GigabitEthernet3, RPF nbr 192.168.23.3 Outgoing interface list: Tunnel0, Forward/Sparse-Dense, 00:00:36/00:02:24 GigabitEthernet3, Bidir-Upstream/Sparse, 00:00:36/stopped
CORE#show ip mroute 184.108.40.206 IP Multicast Routing Table (*, 220.127.116.11), 00:00:49/00:02:45, RP 18.104.22.168, flags: B Bidir-Upstream: Null, RPF nbr 0.0.0.0 Outgoing interface list: GigabitEthernet3, Forward/Sparse, 00:00:44/00:02:45 GigabitEthernet2, Forward/Sparse, 00:00:49/00:02:40
I’m seeing the (*,G) entry for the multicast group 22.214.171.124 and outgoing interfaces. This is looking good.
Let’s try some VXLAN specific commands. First, we’ll check if the NVE interface is up:
Rene the command does work: “show nve peers”
I have been using VXlan with BGP underlay in spine and leaf architecture, we use F&L with out multicast , the mechanism for F&L is Head end replication and manually map vlans-vni’s and update flood list’s. Our Vxlan infrastructure is small and we use another vendors switch’s then Cisco. We’re looking to go to Cisco solution with BGP Vxlan evpn.
Thanks again for another great... Continue reading in our forum
show nve peerscommand was fixed somewhere in between Version 16.06.01 and 16.09.01. Time for me to upgrade VIRL
VXLAN is fun to work with and this lesson was way overdue. Glad to hear you like it! I’ll create an example for BGP EVPN later once I finish some of the missing ENCOR/ENARSI topics.
I am so glad you added this VxLAN section! Please add BGP Evpn and ospf examples. I am currently implementing this solution for VxRail utilizing cat9ks switches and NCS55 ios Xr across two sites. I would love to hear more about this.
Thanks for your very nice lesson as always …
I have tried to understand the fundamental of Service Instance and bridge domain but failed . Can you please help me to understand about the two topic and your configuration here in your magical clear text . It will be great for me .Thanks again
The truth is, there’s a lot there to get your head around! I’ll try to clear it up for you.
First of all we have the Network Virtualization Endpoint (NVE). This is the entity on which the VNI (VXLAN Network Identifier) is configured, and from where the multicast group functions. The NVE is “sourced” on a loopback interface. In other words, it adds the capability of VNI membership and multicast participation to the loopback interface it is sourced from.
Next we have the Ethernet Flow Point (EFP) service instance. This is a logical interface t... Continue reading in our forum