Get Unlimited Access to 811 Cisco Lessons Now Get $1 Trial
You are here: Home » VXLAN

VXLAN Underlay eBGP Two-AS

It’s possible to use eBGP for your VXLAN underlay as well. This has some advantages, including having a single routing protocol for both the underlay and overlay networks. BGP is a stable routing protocol that scales well, even for larger fabrics. It’s not as chatty as OSPF or IS-IS and has no flooding issues. The initial configuration is more work and not as straightforward as OSPF or IS-IS, but once configured, it’s stable and easy to maintain.

We’ll use a single instance of BGP. For the underlay network, we establish neighbor adjacencies between the spine and leaf switches using IP addresses on the physical interfaces. We’ll use this to advertise loopback interfaces. These loopback interfaces are used to establish neighbor adjacencies for the overlay network using the L2VPN EVPN address family.

With the two-AS model, all spine switches are in one AS, and all leaf switches are in another AS.

Normally, I would only show the configuration of the underlay. However, we’ll do both the underlay and overlay networks. The two networks are so tied together with this setup, and there are eBGP-related issues that you have to solve in both the underlay and overlay networks to make this work. Here are some of the issues we’ll run into:

  • Leaf switches use the same AS number, so we’ll run into eBGP issues where eBGP neighbors refuse to install and advertise routes because of the same AS number:
    • Spine switches won’t advertise routes from one leaf switch to another.
    • Leaf switches refuse to accept routes with their own AS number
  • ECMP is disabled by default
  • Spine switches refuse to install and advertise EVPN routes because they are not configured as VTEPs.
  • Next hop issues because eBGP changes the next hop address.

We’ll deal with all these issues step-by-step.

Before you continue with this lesson, make sure you are familiar with VXLAN in general, MP-BGP, and simpler underlay networks such as OSPF or IS-IS.

Having said that, let’s get started.

Configuration

Here is the topology we’ll use:

Vxlan Underlay Ebgp Topology

I use a larger topology with two spine switches and four leaf switches. The spine and leaf switches run Cisco Nexus 9000v version 10.4(5). The hosts are Ubuntu containers. These lightweight devices are only needed to test connectivity.

Configurations

Want to take a look for yourself? Here you will find the startup configuration for each device.

LEAF1

hostname LEAF1

feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.3001
  ip address 192.168.13.3/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.3002
  ip address 192.168.23.3/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10

interface loopback0
  ip address 3.3.3.3/32
  ip pim sparse-mode

LEAF2

hostname LEAF2

feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.4001
  ip address 192.168.14.4/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.4002
  ip address 192.168.24.4/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10
  
interface loopback0
  ip address 4.4.4.4/32
  ip pim sparse-mode

LEAF3

hostname LEAF3

feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.5001
  ip address 192.168.15.5/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.5002
  ip address 192.168.25.5/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10

interface loopback0
  ip address 5.5.5.5/32
  ip pim sparse-mode

LEAF4

hostname LEAF4

feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.6001
  ip address 192.168.16.6/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.6002
  ip address 192.168.26.6/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10

interface loopback0
  ip address 6.6.6.6/32
  ip pim sparse-mode

SPINE1

hostname SPINE1

feature pim

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.1001
  ip address 192.168.13.1/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.1002
  ip address 192.168.14.1/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 9216
  mac-address 0050.c253.1003
  ip address 192.168.15.1/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  no switchport
  mtu 9216
  mac-address 0050.c253.1004
  ip address 192.168.16.1/24
  ip pim sparse-mode
  no shutdown
  
interface loopback0
  ip address 1.1.1.1/32
  ip pim sparse-mode

SPINE2

hostname SPINE2

feature pim

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.2001
  ip address 192.168.23.2/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.2002
  ip address 192.168.24.2/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 9216
  mac-address 0050.c253.2003
  ip address 192.168.25.2/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  no switchport
  mtu 9216
  mac-address 0050.c253.2004
  ip address 192.168.26.2/24
  ip pim sparse-mode
  no shutdown
  
interface loopback0
  ip address 2.2.2.2/32
  ip pim sparse-mode

There are a couple of things that I have preconfigured:

  • The leaf switches have NVE interfaces.
  • Multicast for BUM traffic. SPINE1 is the RP.
Usually, in a topology like this, you would use anycast RP. To keep it simple and keep the focus on the configuration of eBGP for the underlay, I decided to make SPINE1 the only RP.

Underlay Network

Let’s start with the configuration of the underlay.




Neighbor Adjacencies

Here’s what we are going to do:

  • Use the unicast address family.
  • Establish neighbor adjacencies between spine and leaf switches.
  • Use the IP addresses on the physical interfaces for the neighbor adjacencies.
  • Advertise the loopback interfaces.

We’ll start with the spine switches. Let’s enable BGP:

SPINE1 & SPINE2
(config)# feature bgp

Now we can configure BGP. We’ll use the unicast address family to establish neighbor adjacencies with the leaf switches. We’ll also advertise all loopbacks in BGP. We’ll need these loopback interfaces for the overlay network. Here is SPINE1:

SPINE1(config)# router bgp 1
SPINE1(config-router)# router-id 1.1.1.1

SPINE1(config-router)# address-family ipv4 unicast 
SPINE1(config-router-af)# network 1.1.1.1/32

SPINE1(config-router-af)# neighbor 192.168.13.3
SPINE1(config-router-neighbor)# remote-as 2
SPINE1(config-router-neighbor)# address-family ipv4 unicast 

SPINE1(config-router)# neighbor 192.168.14.4
SPINE1(config-router-neighbor)# remote-as 2
SPINE1(config-router-neighbor)# address-family ipv4 unicast 

SPINE1(config-router)# neighbor 192.168.15.5
SPINE1(config-router-neighbor)# remote-as 2
SPINE1(config-router-neighbor)# address-family ipv4 unicast 

SPINE1(config-router)# neighbor 192.168.16.6
SPINE1(config-router-neighbor)# remote-as 2
SPINE1(config-router-neighbor)# address-family ipv4 unicast

And SPINE2:

SPINE2(config)# router bgp 1
SPINE2(config-router)# router-id 2.2.2.2

SPINE2(config-router)# address-family ipv4 unicast 
SPINE2(config-router-af)# network 2.2.2.2/32

SPINE2(config-router-af)# neighbor 192.168.23.3
SPINE2(config-router-neighbor)# remote-as 2
SPINE2(config-router-neighbor)# address-family ipv4 unicast 

SPINE2(config-router)# neighbor 192.168.24.4
SPINE2(config-router-neighbor)# remote-as 2
SPINE2(config-router-neighbor)# address-family ipv4 unicast 

SPINE2(config-router)# neighbor 192.168.25.5
SPINE2(config-router-neighbor)# remote-as 2
SPINE2(config-router-neighbor)# address-family ipv4 unicast 

SPINE2(config-router)# neighbor 192.168.26.6
SPINE2(config-router-neighbor)# remote-as 2
SPINE2(config-router-neighbor)# address-family ipv4 unicast

Now we’ll do the same thing on all leaf switches. First, enable BGP:

LEAF1, LEAF2, LEAF3, LEAF4
(config)# feature bgp

Now we can advertise all loopback interfaces and neighbor adjacencies with the spine switches. Here is LEAF1:

LEAF1(config)# router bgp 2
LEAF1(config-router)# router-id 3.3.3.3
LEAF1(config-router)# address-family ipv4 unicast
LEAF1(config-router-af)# network 3.3.3.3/32

LEAF1(config-router)# neighbor 192.168.13.1
LEAF1(config-router-neighbor)# remote-as 1
LEAF1(config-router-neighbor)# address-family ipv4 unicast

LEAF1(config-router)# neighbor 192.168.23.2
LEAF1(config-router-neighbor)# remote-as 1
LEAF1(config-router-neighbor)# address-family ipv4 unicast

And we’ll do the same thing on LEAF2, LEAF3, and LEAF4:

LEAF2(config)# router bgp 2
LEAF2(config-router)# router-id 4.4.4.4
LEAF2(config-router)# address-family ipv4 unicast
LEAF2(config-router-af)# network 4.4.4.4/32

LEAF2(config-router)# neighbor 192.168.14.1
LEAF2(config-router-neighbor)# remote-as 1
LEAF2(config-router-neighbor)# address-family ipv4 unicast 

LEAF2(config-router)# neighbor 192.168.24.2 
LEAF2(config-router-neighbor)# remote-as 1
LEAF2(config-router-neighbor)# address-family ipv4 unicast
LEAF3(config)# router bgp 2
LEAF3(config-router)# router-id 5.5.5.5
LEAF3(config-router)# address-family ipv4 unicast
LEAF3(config-router-af)# network 5.5.5.5/32

LEAF3(config-router)# neighbor 192.168.15.1
LEAF3(config-router-neighbor)# remote-as 1
LEAF3(config-router-neighbor)# address-family ipv4 unicast

LEAF3(config-router)# neighbor 192.168.25.2 
LEAF3(config-router-neighbor)# remote-as 1
LEAF3(config-router-neighbor)# address-family ipv4 unicast
LEAF4(config)# router bgp 2
LEAF4(config-router)# router-id 6.6.6.6
LEAF4(config-router)# address-family ipv4 unicast
LEAF4(config-router-af)# network 6.6.6.6/32

LEAF4(config-router)# neighbor 192.168.16.1
LEAF4(config-router-neighbor)# remote-as 1
LEAF4(config-router-neighbor)# address-family ipv4 unicast

LEAF4(config-router)# neighbor 192.168.26.2
LEAF4(config-router-neighbor)# remote-as 1
LEAF4(config-router-neighbor)# address-family ipv4 unicast

That takes care of the leaf switches.

In a production network, it would be a good idea to use BGP templates to reduce the number of configuration lines that we have to add. To keep it simple, I decided to configure all neighbors manually in this lab.

Let’s check if we have neighbor adjacencies. I’ll start with the spine switches:

SPINE1# show ip bgp summary
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 1.1.1.1, local AS number 1
BGP table version is 10, IPv4 Unicast config peers 4, capable peers 4
5 network entries and 5 paths using 1460 bytes of memory
BGP attribute entries [2/736], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS    MsgRcvd    MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.13.3    4     2         27         27       10    0    0 00:20:55 1         
192.168.14.4    4     2         24         24       10    0    0 00:17:32 1         
192.168.15.5    4     2         21         20       10    0    0 00:14:17 1         
192.168.16.6    4     2         15         14       10    0    0 00:08:26 1 
SPINE2# show ip bgp summary 
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 2.2.2.2, local AS number 1
BGP table version is 10, IPv4 Unicast config peers 4, capable peers 4
5 network entries and 5 paths using 1460 bytes of memory
BGP attribute entries [2/736], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS    MsgRcvd    MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.23.3    4     2         28         28       10    0    0 00:21:46 1         
192.168.24.4    4     2         25         24       10    0    0 00:18:29 1         
192.168.25.5    4     2         22         21       10    0    0 00:15:14 1         
192.168.26.6    4     2         16         16       10    0    0 00:09:18 1

The spine switches see all leaf switches as neighbors, so that’s good. They also received one prefix from each leaf switch. Let’s take a closer look:

SPINE1# show ip bgp 
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 10, Local Router ID is 1.1.1.1
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>l1.1.1.1/32         0.0.0.0                           100      32768 i
*>e3.3.3.3/32         192.168.13.3                                   0 2 i
*>e4.4.4.4/32         192.168.14.4                                   0 2 i
*>e5.5.5.5/32         192.168.15.5                                   0 2 i
*>e6.6.6.6/32         192.168.16.6                                   0 2 i
SPINE2# show ip bgp
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 10, Local Router ID is 2.2.2.2
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>l2.2.2.2/32         0.0.0.0                           100      32768 i
*>e3.3.3.3/32         192.168.23.3                                   0 2 i
*>e4.4.4.4/32         192.168.24.4                                   0 2 i
*>e5.5.5.5/32         192.168.25.5                                   0 2 i
*>e6.6.6.6/32         192.168.26.6                                   0 2 i

This looks good. The spine switches learned about the loopback interfaces of the leaf switches.

Disable Peer AS Check

Let’s check one of the leaf switches:

LEAF1# show ip bgp
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 6, Local Router ID is 3.3.3.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>e1.1.1.1/32         192.168.13.1                                   0 1 i
*>e2.2.2.2/32         192.168.23.2                                   0 1 i
*>l3.3.3.3/32         0.0.0.0                           100      32768 i

LEAF1 only knows about the loopback interfaces of the spine switches. It doesn’t have the loopback interfaces of the other leaf switches. To understand why, we’ll take a closer look at SPINE1:

SPINE1# show ip bgp neighbors 192.168.13.3 advertised-routes 

Peer 192.168.13.3 routes for address family IPv4 Unicast:
BGP table version is 10, Local Router ID is 1.1.1.1
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>l1.1.1.1/32         0.0.0.0                           100      32768 i

The spine switches don’t advertise prefixes from one leaf switch to another. That makes sense because the leaf switches use the same AS number. We can fix this by adding the disable-peer-as-check command on the spine switches:

Here is SPINE1:

SPINE1(config)# router bgp 1
SPINE1(config-router)# neighbor 192.168.13.3
SPINE1(config-router-neighbor)# address-family ipv4 unicast
SPINE1(config-router-neighbor-af)# disable-peer-as-check

SPINE1(config-router)# neighbor 192.168.14.4
SPINE1(config-router-neighbor)# address-family ipv4 unicast
SPINE1(config-router-neighbor-af)# disable-peer-as-check

SPINE1(config-router)# neighbor 192.168.15.5
SPINE1(config-router-neighbor)# address-family ipv4 unicast
SPINE1(config-router-neighbor-af)# disable-peer-as-check

SPINE1(config-router)# neighbor 192.168.16.6
SPINE1(config-router-neighbor)# address-family ipv4 unicast
SPINE1(config-router-neighbor-af)# disable-peer-as-check

And SPINE2:

SPINE2(config)# router bgp 1
SPINE2(config-router)# neighbor 192.168.23.3
SPINE2(config-router-neighbor)# address-family ipv4 unicast
SPINE2(config-router-neighbor-af)# disable-peer-as-check

SPINE2(config-router)# neighbor 192.168.24.4
SPINE2(config-router-neighbor)# address-family ipv4 unicast
SPINE2(config-router-neighbor-af)# disable-peer-as-check

SPINE2(config-router)# neighbor 192.168.25.5
SPINE2(config-router-neighbor)# address-family ipv4 unicast
SPINE2(config-router-neighbor-af)# disable-peer-as-check

SPINE2(config-router)# neighbor 192.168.26.6
SPINE2(config-router-neighbor)# address-family ipv4 unicast
SPINE2(config-router-neighbor-af)# disable-peer-as-check

Now, take another look at what SPINE1 advertises to LEAF1:

SPINE1# show ip bgp neighbors 192.168.13.3 advertised-routes 

Peer 192.168.13.3 routes for address family IPv4 Unicast:
BGP table version is 14, Local Router ID is 1.1.1.1
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>l1.1.1.1/32         0.0.0.0                           100      32768 i
*>e4.4.4.4/32         192.168.14.4                                   0 2 i
*>e5.5.5.5/32         192.168.15.5                                   0 2 i
*>e6.6.6.6/32         192.168.16.6                                   0 2 i

SPINE1 now advertises the loopback interfaces from LEAF2, LEAF3, and LEAF4 to LEAF1.

Allow AS In

When we look at LEAF1, though, you can see it doesn’t install anything:

LEAF1# show ip bgp
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 6, Local Router ID is 3.3.3.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>e1.1.1.1/32         192.168.13.1                                   0 1 i
*>e2.2.2.2/32         192.168.23.2                                   0 1 i
*>l3.3.3.3/32         0.0.0.0                           100      32768 i

What happens here is that although SPINE1 (or SPINE2) advertises those prefixes to the leaf switches, the leaf switches won’t install anything because they see their own AS number. We can fix that with the allowas-in command.

Another scenario where you see allowas-in is with MPLS VPN where the CE routers use the same AS number.

Here’s what it looks like:

LEAF1(config)# router bgp 2 
LEAF1(config-router)# neighbor 192.168.13.1
LEAF1(config-router-neighbor)# address-family ipv4 unicast
LEAF1(config-router-neighbor-af)# allowas-in ?
      
  <1-10>  Number of occurrences of AS number, default is 3

LEAF1(config-router-neighbor-af)# allowas-in

You can specify the number of occurrences. In our case, the default is fine because the leaf switches see their own AS number only once. Let’s configure it:

LEAF1(config-router)# neighbor 192.168.23.2
LEAF1(config-router-neighbor)# address-family ipv4 unicast
LEAF1(config-router-neighbor-af)# allowas-in

Make sure to enable allowas-in on all leaf switches:

LEAF2(config)# router bgp 2

LEAF2(config-router)# neighbor 192.168.14.1
LEAF2(config-router-neighbor)# address-family ipv4 unicast
LEAF2(config-router-neighbor-af)# allowas-in 

LEAF2(config-router)# neighbor 192.168.24.2 
LEAF2(config-router-neighbor)# address-family ipv4 unicast
LEAF2(config-router-neighbor-af)# allowas-in
LEAF3(config)# router bgp 2

LEAF3(config-router)# neighbor 192.168.15.1
LEAF3(config-router-neighbor)# address-family ipv4 unicast
LEAF3(config-router-neighbor-af)# allowas-in

LEAF3(config-router-neighbor-af)# neighbor 192.168.25.2
LEAF3(config-router-neighbor)# address-family ipv4 unicast
LEAF3(config-router-neighbor-af)# allowas-in
LEAF4(config)# router bgp 2

LEAF4(config-router)# neighbor 192.168.16.1
LEAF4(config-router-neighbor)#  address-family ipv4 unicast
LEAF4(config-router-neighbor-af)# allowas-in 

LEAF4(config-router)# neighbor 192.168.26.2
LEAF4(config-router-neighbor)# address-family ipv4 unicast
LEAF4(config-router-neighbor-af)# allowas-in

That’s it. Now let’s check the BGP table on LEAF1 once more:

LEAF1# show ip bgp
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 24, Local Router ID is 3.3.3.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>e1.1.1.1/32         192.168.13.1                                   0 1 i
*>e2.2.2.2/32         192.168.23.2                                   0 1 i
*>l3.3.3.3/32         0.0.0.0                           100      32768 i
* e4.4.4.4/32         192.168.23.2                                   0 1 2 i
*>e                   192.168.13.1                                   0 1 2 i
* e5.5.5.5/32         192.168.23.2                                   0 1 2 i
*>e                   192.168.13.1                                   0 1 2 i
* e6.6.6.6/32         192.168.23.2                                   0 1 2 i
*>e                   192.168.13.1                                   0 1 2 i

Now we see the loopback interfaces of the other leaf switches!

Equal-Cost Multi Path (ECMP)

There is one more thing, though…if you look closely, you can see that LEAF1 has selected SPINE1 as the default next hop. That works, but we won’t have Equal-Cost Multi-Path (ECMP). We’ll add the maximum-paths command on all leaf switches to take care of that:

LEAF1, LEAF2, LEAF3, LEAF4
(config)# router bgp 2
(config-router)# address-family ipv4 unicast
(config-router-af)# maximum-paths 2

Let’s check the BGP table again:

LEAF1# show ip bgp
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 30, Local Router ID is 3.3.3.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>e1.1.1.1/32         192.168.13.1                                   0 1 i
*>e2.2.2.2/32         192.168.23.2                                   0 1 i
*>l3.3.3.3/32         0.0.0.0                           100      32768 i
*|e4.4.4.4/32         192.168.23.2                                   0 1 2 i
*>e                   192.168.13.1                                   0 1 2 i
*|e5.5.5.5/32         192.168.23.2                                   0 1 2 i
*>e                   192.168.13.1                                   0 1 2 i
*|e6.6.6.6/32         192.168.23.2                                   0 1 2 i
*>e                   192.168.13.1                                   0 1 2 i

Multipath is enabled, so we have ECMP. We can also verify this by looking at the routing table:

LEAF1# show ip route bgp
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%' in via output denotes VRF 

1.1.1.1/32, ubest/mbest: 1/0
    *via 192.168.13.1, [20/0], 00:00:17, bgp-2, external, tag 1
2.2.2.2/32, ubest/mbest: 1/0
    *via 192.168.23.2, [20/0], 00:00:17, bgp-2, external, tag 1
4.4.4.4/32, ubest/mbest: 2/0
    *via 192.168.13.1, [20/0], 00:00:17, bgp-2, external, tag 1
    *via 192.168.23.2, [20/0], 00:00:17, bgp-2, external, tag 1
5.5.5.5/32, ubest/mbest: 2/0
    *via 192.168.13.1, [20/0], 00:00:17, bgp-2, external, tag 1
    *via 192.168.23.2, [20/0], 00:00:17, bgp-2, external, tag 1
6.6.6.6/32, ubest/mbest: 2/0
    *via 192.168.13.1, [20/0], 00:00:17, bgp-2, external, tag 1
    *via 192.168.23.2, [20/0], 00:00:17, bgp-2, external, tag 1

LEAF1 now has two routes for each of the loopback interfaces of the other leaf switches. That’s all there is to it. The underlay network is now up and running.

Configurations

Want to take a look for yourself? Here you will find the final configuration of the underlay for each device.

LEAF1

hostname LEAF1

feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.3001
  ip address 192.168.13.3/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.3002
  ip address 192.168.23.3/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10

interface loopback0
  ip address 3.3.3.3/32
  ip pim sparse-mode
  
router bgp 2
  router-id 3.3.3.3
  address-family ipv4 unicast
    network 3.3.3.3/32
    maximum-paths 2
  neighbor 192.168.13.1
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3
  neighbor 192.168.23.2
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3

LEAF2

hostname LEAF2

feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.4001
  ip address 192.168.14.4/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.4002
  ip address 192.168.24.4/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10
  
interface loopback0
  ip address 4.4.4.4/32
  ip pim sparse-mode
  
router bgp 2
  router-id 4.4.4.4
  address-family ipv4 unicast
    network 4.4.4.4/32
    maximum-paths 2
  neighbor 192.168.14.1
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3
  neighbor 192.168.24.2
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3

LEAF3

hostname LEAF3

feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.5001
  ip address 192.168.15.5/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.5002
  ip address 192.168.25.5/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10

interface loopback0
  ip address 5.5.5.5/32
  ip pim sparse-mode
  
router bgp 2
  router-id 5.5.5.5
  address-family ipv4 unicast
    network 5.5.5.5/32
    maximum-paths 2
  neighbor 192.168.15.1
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3
  neighbor 192.168.25.2
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3

LEAF4

hostname LEAF4

feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1,10

vlan 10
  vn-segment 10010

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010
    mcast-group 239.1.1.1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.6001
  ip address 192.168.16.6/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.6002
  ip address 192.168.26.6/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  switchport access vlan 10

interface loopback0
  ip address 6.6.6.6/32
  ip pim sparse-mode

router bgp 2
  router-id 6.6.6.6
  address-family ipv4 unicast
    network 6.6.6.6/32
    maximum-paths 2
  neighbor 192.168.16.1
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3
  neighbor 192.168.26.2
    remote-as 1
    address-family ipv4 unicast
      allowas-in 3

SPINE1

hostname SPINE1

feature bgp
feature pim

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.1001
  ip address 192.168.13.1/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.1002
  ip address 192.168.14.1/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 9216
  mac-address 0050.c253.1003
  ip address 192.168.15.1/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  no switchport
  mtu 9216
  mac-address 0050.c253.1004
  ip address 192.168.16.1/24
  ip pim sparse-mode
  no shutdown
  
interface loopback0
  ip address 1.1.1.1/32
  ip pim sparse-mode
  
router bgp 1
  router-id 1.1.1.1
  address-family ipv4 unicast
    network 1.1.1.1/32
  neighbor 192.168.13.3
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check
  neighbor 192.168.14.4
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check
  neighbor 192.168.15.5
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check
  neighbor 192.168.16.6
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check

SPINE2

hostname SPINE2

feature bgp
feature pim

ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4

vlan 1

interface Ethernet1/1
  no switchport
  mtu 9216
  mac-address 0050.c253.2001
  ip address 192.168.23.2/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9216
  mac-address 0050.c253.2002
  ip address 192.168.24.2/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 9216
  mac-address 0050.c253.2003
  ip address 192.168.25.2/24
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  no switchport
  mtu 9216
  mac-address 0050.c253.2004
  ip address 192.168.26.2/24
  ip pim sparse-mode
  no shutdown
  
interface loopback0
  ip address 2.2.2.2/32
  ip pim sparse-mode

router bgp 1
  router-id 2.2.2.2
  address-family ipv4 unicast
    network 2.2.2.2/32
  neighbor 192.168.23.3
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check
  neighbor 192.168.24.4
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check
  neighbor 192.168.25.5
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check
  neighbor 192.168.26.6
    remote-as 2
    address-family ipv4 unicast
      disable-peer-as-check

Overlay Network

With the underlay network up and running, we can configure the overlay network.




We need to add the nv overlay evpn command on all switches, otherwise you can’t use the L2VPN EVPN address family. Enable this on all switches:

SPINE1, SPINE2, LEAF1, LEAF2, LEAF3, LEAF4
(config)# nv overlay evpn

Neighbor Adjacencies

Now we can configure the neighbor adjacencies. We’ll start with the spine switches:

We're Sorry, Full Content Access is for Members Only...

If you like to keep on reading, Become a Member Now!

  • Learn any CCNA, CCNP and CCIE R&S Topic. Explained As Simple As Possible.
  • Try for Just $1. The Best Dollar You’ve Ever Spent on Your Cisco Career!
  • Full Access to our 811 Lessons. More Lessons Added Every Week!
  • Content created by Rene Molenaar (CCIE #41726)
Give Membership a try - it's just $1 ►
223 Sign Ups in the last 30 days
satisfaction-guaranteed

  • 100% Satisfaction Guaranteed!
  • You may cancel your monthly membership at any time.
  • No Questions Asked!

Ask a question or start a discussion by visiting our Community Forum