Introduction to Virtual Extensible LAN (VXLAN)

Virtual eXtensible Local Area Network (VXLAN) is a tunneling protocol that tunnels Ethernet (layer 2) traffic over an IP (layer 3) network.



Traditional layer 2 networks have issues because of three main reasons:

  • Spanning-tree.
  • Limited amount of VLANs.
  • Large MAC address tables.

Spanning-tree blocks any redundant links to avoid loops. Blocking links to create a loop-free topology gets the job done, but it also means we pay for links we can’t use. We could switch to a layer 3 network, but some technology requires layer 2 networking.

The VLAN ID is 12-bit, which means we can create 4094 VLANs (0 and 4095 are reserved). Only 4094 available VLANs can be an issue for data centers. For example, imagine we have a service provider with 500 customers. With 4094 available VLANs, they can only offer 8 VLANs to each customer.

Because of server virtualization, the number of addresses in the MAC address tables of our switches has grown exponentially. Before server virtualization, a switch only had to learn one MAC address per switchport. With server virtualization, we run many virtual machines (VM) or containers on a single physical server. Each VM has a virtual NIC and a virtual MAC address. The switch has to learn many MAC addresses on a single switchport.

A Top of Rack (ToR) switch in a data center could connect to 24 or 48 physical servers. A data center could have many racks, so each switch has to store the MAC addresses of all VMs that communicate with each other. We require much larger MAC address tables compared to networks without server virtualization.

In this lesson, I’ll explain what VXLAN is, how it works, and how it solves the above layer 2 issues.

Overlay vs Underlay

VXLAN uses an overlay and underlay network:

Overlay Underlay Network

An overlay network is a virtual network that runs on top of a physical underlay network. Even if you never heard about this terminology before, you have probably seen it. A GRE tunnel is a simple example of an overlay network. The GRE tunnel runs on top of a physical underlay network.

With VXLAN, the overlay is a layer 2 Ethernet network. The underlay network is a layer 3 IP network. Another name for the underlay network is a transport network.

The underlay network is simple; its only job is to get packets from A to B. We don’t use any layer 2 here, only layer 3. When we use layer 3, we can use an IGP like OSPF or EIGRP and load balance traffic on redundant links.

Another advantage is that the overlay and underlay network are independent. The overlay network is virtual and requires an underlay network, but whatever changes you make in the overlay network won’t affect the underlay network. You can add and remove links in the underlay network, and as long as your routing protocol can reach the destination, your overlay network will remain unchanged.

VNI

The VXLAN Network Identifier (VNI) identifies the VXLAN and has a similar function as the VLAN ID for regular VLANs. We use 24 bits for the VNI, which means we can create 16,777,215 ( ~16 million) VXLANs. That’s a lot, compared to those 4094 VLANs with a 12-bit VLAN ID. We can create plenty of VXLANs, which means a large service provider with even thousands of customers can use as many VXLANs per customer as needed.

VTEP

The VXLAN tunnel endpoint (VTEP) is the device that’s responsible for encapsulating and de-encapsulating layer 2 traffic. This device is the connection between the overlay and the underlay network. The VTEP comes in two forms:

  • Software (host-based)
  • Hardware (gateway)

Let’s look at these two options.

Software

When I’m talking about hosts, I mean hypervisors like VMWare’s ESXi or Microsoft’s Hyper-V. These hypervisors use virtual switches, and some of them support VXLAN. Here’s an illustration:

Vxlan Vtep Software Hypervisor Virtual Switch

The VXLAN tunnels are between the virtual switches of the hypervisors. The underlay network is unaware of VXLAN.

Hardware

A hardware VTEP is a router, switch, or firewall which supports VXLAN. We also call a hardware VTEP a VXLAN gateway because it combines a regular VLAN and VXLAN segment into a single layer 2 domain. Some switches have VXLAN support with ASICs, offering better VXLAN performance than a software VTEP. Here’s what it looks like:

Vxlan Vtep Hardware

In the above picture, the VXLAN tunnels are between the physical switches. The devices that connect to the physical switches are unaware of VXLAN.

Interfaces

Each VTEP has two interfaces types:

  • VTEP IP interface: Connects the VTEP to the underlay network with a unique IP address. This interface encapsulates and de-encapsulates Ethernet frames.
  • VNI interface: A virtual interface that keeps network traffic separated on the physical interface. Similar to an SVI interface.

A VTEP can have multiple VNI interfaces, but they associate with the same VTEP IP interface. Here’s a picture to help you visualize this:

We're Sorry, Full Content Access is for Members Only...

If you like to keep on reading, Become a Member Now! Here is why:

  • Learn any CCNA, CCNP and CCIE R&S Topic. Explained As Simple As Possible.
  • Try for Just $1. The Best Dollar You’ve Ever Spent on Your Cisco Career!
  • Full Access to our 710 Lessons. More Lessons Added Every Week!
  • Content created by Rene Molenaar (CCIE #41726)

473 Sign Ups in the last 30 days

satisfaction-guaranteed
100% Satisfaction Guaranteed!
You may cancel your monthly membership at any time.
No Questions Asked!

Tags:


Forum Replies

  1. Hi Rene, this is a very good lesson on VXLAN, straight to the point and well written.
    I’m particularly passionate to this topic because I worked on a VXLAN EVPN Fabric the last year and love this technology.
    If I had done a lesson like this when I started It would have been easier to learn the topic…

  2. Hi Rene,
    How can we configure vxlan. Could you please explain.

  3. I fixed this, somehow a draft post got published :slight_smile: Thanks for letting us know!

  4. Hi
    Thanks for this lesson.

    <Traditional layer 2 networks have issues because of three main reasons:

    Spanning-tree.
    Limited amount of VLANs.
    Large MAC address tables.>

    My understanding is as follows. Could you please explain ?

    1. Spine-leaf topology solved the Spanning tree issue. STP is not an issue in VxLAN.
    2. Limited amount of VLANs : This is the essential why vxlan needs.
    3. Large MAC address tables : I’am not sure how vxlan solve this problem. This issue is common in virtualization.

    Spine-leaf was not ip network. It becomes ip underlay to bind MAC learning to a

    ... Continue reading in our forum

  5. Thank you very much Laz for the clarify the issue.

11 more replies! Ask a question or join the discussion by visiting our Community Forum