VXLAN Flood and Learn with Multicast

In the introduction to VXLAN lesson, I explained what VXLAN is and how it works. In this lesson, I’ll show you how to configure VXLAN where we use the multicast “flood and learn” system to learn the mapping between a VTEP IP address and a MAC address.

Configuration

Here’s the topology we’ll use:

All devices are CSR1000V routers running Cisco IOS XE Software, version 16.06.01. I’m using CSR1000V routers since anyone can use these. I use custom MAC addresses because those are easy to recognize when we do a packet capture.

VTEP1 and VTEP2 are our VTEP devices. The core router is there to simulate our “IP network”.  We are going to create a VXLAN tunnel with VNI 5012 so that H1 and H2 can communicate directly over layer 2.

I pre-configured OSPF so that we have connectivity between the VTEP devices and the core router.

hostname CORE
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet2
 mac-address 0000.5e00.5303
 ip address 192.168.13.3 255.255.255.0
!
interface GigabitEthernet3
 mac-address 0000.5e00.5333
 ip address 192.168.23.3 255.255.255.0
!
router ospf 1
 network 3.3.3.3 0.0.0.0 area 0
 network 192.168.13.0 0.0.0.255 area 0
 network 192.168.23.0 0.0.0.255 area 0
!
end

hostname H1
!
interface GigabitEthernet2
 mac-address 0000.5e00.5365
 ip address 192.168.12.101 255.255.255.0
!
end

hostname H2
!
interface GigabitEthernet2
 mac-address 0000.5e00.5366
 ip address 192.168.12.102 255.255.255.0
!
end

hostname VTEP1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface GigabitEthernet2
 mac-address 0000.5e00.5301
!
interface GigabitEthernet3
 mac-address 0000.5e00.5311
 ip address 192.168.13.1 255.255.255.0
!
router ospf 1
 network 1.1.1.1 0.0.0.0 area 0
 network 192.168.13.0 0.0.0.255 area 0
!
end

hostname VTEP2
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface GigabitEthernet2
 mac-address 0000.5e00.5302
!
interface GigabitEthernet3
 mac-address 0000.5e00.5322
 ip address 192.168.23.2 255.255.255.0
!
router ospf 1
 network 2.2.2.2 0.0.0.0 area 0
 network 192.168.23.0 0.0.0.255 area 0
!
end

Multicast

Let’s start with the configuration of multicast. With VXLAN, we don’t have a typical scenario where we have a few sources and many receivers. All VTEP devices communicate with each other so it makes sense to use bidirectional PIM. The core router will be the RP in this network.

Let’s enable multicast routing and bidirectional PIM on all VTEP devices and the core router:

VTEP1, VTEP2 & CORE
(config)#ip multicast-routing distributed
(config)#ip pim bidir-enable

We need to enable PIM sparse mode on all physical interfaces that connect to the IP network:

VTEP1 & VTEP2 & CORE
(config)#interface GigabitEthernet 3
(config-if)#ip pim sparse-mode

CORE(config)#interface GigabitEthernet 2
CORE(config-if)#ip pim sparse-mode

And don’t forget the loopback interfaces:

VTEP1, VTEP2 & CORE
(config)#interface Loopback 0
(config-if)#ip pim sparse-mode

Last but not least, configure the RP address:

VTEP1, VTEP2 & CORE
(config)#ip pim rp-address 3.3.3.3 bidir

This completes the multicast configuration.

VXLAN

We need to create a Network Virtualization Endpoint (NVE) interface. This is where we configure the VNI and multicast group that we will use. We source this interface from the loopback 0 interface, use VNI 5012, and use multicast group 239.1.1.1.

Here’s how to configure the NVE interface:

VTEP1 & VTEP2
(config)#interface NVE 1
(config-if)#no shutdown
(config-if)#source-interface Loopback 0
(config-if)#member vni 5012 mcast-group 239.1.1.1

Now we need to configure the Ethernet Flow Point (EFP) service instance. This is a logical interface that connects a bridge domain to a physical port (or EtherChannel). Under the service instance, we configure whether the incoming traffic is tagged or untagged. In our case, the hosts send untagged traffic. This is how to configure it:

VTEP1 & VTEP2
(config)#interface GigabitEthernet 2
(config-if)#service instance 1 ethernet
(config-if-srv)#encapsulation untagged
(config-if-srv)#exit
(config-if)#exit

Last but not least, we need to configure the Bridge Domain Interface (BDI):

This is where we combine the VNI, physical interface, and service-instance:

VTEP1 & VTEP2
(config)#bridge-domain 1
(config-bdomain)#member vni 5012
(config-bdomain)#member GigabitEthernet 2 service-instance 1

This completes our VXLAN configuration.

Verification

Let’s verify our work.

Multicast

First, I’ll check if our multicast configuration is correct:

VTEP1#show ip mroute 239.1.1.1
IP Multicast Routing Table

(*, 239.1.1.1), 00:00:36/00:02:25, RP 3.3.3.3, flags: BCx
  Bidir-Upstream: GigabitEthernet3, RPF nbr 192.168.13.3
  Outgoing interface list:
    Tunnel0, Forward/Sparse-Dense, 00:00:36/00:02:25
    GigabitEthernet3, Bidir-Upstream/Sparse, 00:00:36/stopped

VTEP2#show ip mroute 239.1.1.1
IP Multicast Routing Table

(*, 239.1.1.1), 00:00:36/00:02:24, RP 3.3.3.3, flags: BCx
  Bidir-Upstream: GigabitEthernet3, RPF nbr 192.168.23.3
  Outgoing interface list:
    Tunnel0, Forward/Sparse-Dense, 00:00:36/00:02:24
    GigabitEthernet3, Bidir-Upstream/Sparse, 00:00:36/stopped

CORE#show ip mroute 239.1.1.1
IP Multicast Routing Table

(*, 239.1.1.1), 00:00:49/00:02:45, RP 3.3.3.3, flags: B
  Bidir-Upstream: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet3, Forward/Sparse, 00:00:44/00:02:45
    GigabitEthernet2, Forward/Sparse, 00:00:49/00:02:40

I’m seeing the (*,G) entry for the multicast group 239.1.1.1 and outgoing interfaces. This is looking good.

VXLAN

Let’s try some VXLAN specific commands. First, we’ll check if the NVE interface is up: