CEF (Cisco Express Forwarding)

Perhaps you have heard about the term “wirespeed” before. It’s something the marketing department likes to use when it comes to selling networking equipment. It means that packets can be forwarded without any noticeable delay. Oh btw, for the remaining of this lesson the words “multilayer switch” and “router” are the same thing. Everything that I explain about the multilayer switches from now on also applies to routers.Let’s take a look at the difference between layer 2 and multilayer switches from the switch’s perspective:

You know that layer 2 switches only will switch Ethernet frames within a VLAN, and if we want we can filter traffic based on layer 2 (for example with port-security). The multilayer switch can do the same but is also able to route between VLANS and filter on layer 3 or 4 using access-lists.

Forwarding on layer 2 is based on the destination MAC address. Our switch learns the source MAC addresses on incoming frames and it builds the MAC address table. Whenever an Ethernet frame enters one of our interfaces, we’ll check the MAC address table to find the destination MAC address and we’ll send it out the correct interface.

Forwarding on layer 3 is based on the destination IP address. Forwarding happens when the switch receives an IP packet where the source IP address is in a different subnet than the destination IP address.

When our multilayer switch receives an IP packet with its own MAC address as the destination in the Ethernet header there are two possibilities:

• If the destination IP address is an address that is configured on the multilayer switch then the IP packet was destined for this switch.
• If the destination IP address is an address that is not configured on the multilayer switch then we have to act as a gateway and “route” the packet. This means we’ll have to do a lookup in the routing table to check for the longest match. Also we have to check if the IP packet is allowed if you configured an ACL.

Back in the days…switching was done at hardware speed while routing was done in software. Nowadays both switching and routing is done at hardware speed. In the remaining of this lesson you’ll learn why.

Let’s take a look at the difference between handling Ethernet Frames and IP Packets:

The life of a layer 2 switch is simple:

1. The switch will verify the checksum of the Ethernet frame to make it sure it’s not corrupted or altered.
2. The switch receives an Ethernet frame and adds the source MAC address to the MAC address table.
3. The switch forwards the Ethernet frame to the correct interface if it knows the destination MAC address. If not, it will be flooded.

There is no alteration of the Ethernet frame!

Now let’s see what we have to do when we receive an IP packet on a multilayer switch:

In the example above H1 is sending an IP packet towards H2. Note that they are in different subnets so we will have to route it. When our multilayer switch receives the IP packet this is what will happen:

1. The switch will verify the checksum of the Ethernet frame to make it sure it’s not corrupted or altered.
2. The switch will verify the checksum of the IP packet to make it sure it’s not corrupted or altered.

The multilayer switch will check the routing table, notices that 192.168.20.0/24 is directly connected and the following will happen:

1. Check the ARP table to see if there’s a layer 2 to 3 mapping for H2. If there is no mapping the multilayer switch will send an ARP request.
2. The destination MAC address changes from FFF (Multilayer switch Fa0/1 ) to BBB (H2).
3. The source MAC address changes from AAA (H1) to GGG (Multilayer switch Fa0/2).
4. The TTL (time to live) field in the IP packet is decreased by 1 and because of this the IP header checksum will be recalculated.
5. The Ethernet frame checksum must be recalculated.
6. The Ethernet frame carrying the IP packet will be sent out of the interface towards H2.

As you can see there are quite some steps involved if we want to route IP packets.

When we look at multilayer switches there is a “separation of duties”. We have to build a table for the MAC addresses, fill a routing table, ARP requests, check if an IP packet matches an access-list etc and we need to forward our IP packets. These tasks are divided between the “control plane” and the “data plane”. Let me give you an illustration:

The control plane is responsible for exchanging routing information using routing protocols, building a routing table and ARP table.The data plane is responsible for the actual forwarding of IP packets. The routing table isn’t very suitable for fast forwarding because we have to deal with recursive routing. What is recursive routing? Let me give you an example:

In the example above I have three routers. R3 has a loopback interface that we want to reach from R1. I will use static routes for reachability:

R1(config)#ip route 3.3.3.0 255.255.255.0 192.168.23.3
R1(config)#ip route 192.168.23.0 255.255.255.0 192.168.12.2

The first static route is to reach the loopback0 interface of R3 and points to the FastEthernet0/0 interface of R3. The second static route is required to reach network 192.168.23.0/24.

R1#show ip route         
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 -IS-IS level-2
       ia - IS-IS inter area, * - candidate default, per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     3.0.0.0/24 is subnetted, 1 subnets
S       3.3.3.0 [1/0] via 192.168.23.3
S    192.168.23.0/24 [1/0] via 192.168.12.2

Whenever R1 wants to reach 3.3.3.0/24 we have to do 3 lookups:

• The first lookup is to check the entry for 3.3.3.0 /24. It’s there and the next hop IP address is 192.168.23.3
• The second lookup is for 192.168.23.3. There’s an entry and the next hop IP address is 192.168.12.2.
• The third and last lookup is for 192.168.12.2. There’s an entry and it is directly connected.

R1 has to check the routing table 3 times before it knows where to send its traffic. Doesn’t sound very efficient right? Doing multiple lookups to reach a certain network is called recursive routing.

Most of the time all incoming and outgoing IP packets will be processed and forwarded by the data plane but there are some exceptions, first let me show you this picture:

Most of the IP packets can be forwarded by the data plane. However there are some “special” IP packets that can’t be forwarded by the data plane immediately and they are sent to the control plane, here are some examples:

• IP packets that are destined for one of the IP addresses of the multilayer switch.
• Routing protocol traffic like OSPF, EIGRP or BGP.
• IP packets that have some of the options set in the IP header.
• IP packets with an expired TTL.

The control plane can forward outgoing IP packets to the data plane or use its own forwarding mechanism to determine the outgoing interface and the next hop IP address. An example of this is local policy based routing. If you have never heard about policy based routing, don’t worry…it’s covered in CCNP ROUTE.

Our multilayer switch has many more steps to take than the layer 2 switches so theoretically it should be slower right?

One reason that multilayer switches are able to forward frames and packets at wirespeed is because of special hardware called ASICs in the dataplane.

Information like MAC addresses, the routing table or access-lists are stored into these ASICs. The tables are stored in content-addressable memory (CAM) and ternary content addressable memory (TCAM).

• The CAM table is used to store layer 2 information like:
  • The source MAC address.
  • The interface where we learned the MAC address on.
  • To which VLAN the MAC address belongs.

Table lookups are fast! Whenever the switch receives an Ethernet frame it will use a hashing algorithm to create a “key” for the destination MAC address + VLAN and it will compare this hash to the already hashed information in the CAM table. This way it is able to quickly lookup information in the CAM table.

• The TCAM table is used to store “higher layer” information like:
  • Access-lists.
  • Quality of service information.
  • Routing table.
• The TCAM table can match on 3 different values:
  • 0 = must be 0.
  • 1 = must be 1.
  • X = 0 or 1 both acceptable.
• Longest match will return a hit.
• Useful for a lookup where we don’t need an exact match. (routing table or ACLs for example).

Because there are 3 values we call it ternary.

So why are there 2 types of tables?

When we look for a MAC address we always require an exact match. We require the exact MAC address if we want to forward an Ethernet frame. The MAC address table is stored in a CAM table.

Whenever we need to match an IP packet against the routing table or an access-list we don’t always need an exact match. For example an IP packet with destination address 192.168.20.44 will match:

• 192.168.20.44 /32
• 192.168.20.0 /24
• 192.168.0.0 /16

Information like the routing table are stored in a TCAM table for this reason. We can decide whether all or some bits have to match.

Here’s an example of a TCAM table:

If we want to match IP address 192.168.10.22 the multilayer switch will first see if there’s a “most specific match”. There is nothing that matches 192.168.10.22 /32 so we’ll continue if there is anything else that matches. In this case there is an entry that matches 192.168.10.0 /24. The example above applies to routing table lookups, access-lists but also quality of service, VLAN access-lists and more.

Now you know all the steps a multilayer switch has to take when it has to forward ip packets, the control/data plane and that we use different tables stored in special hardware called ASICs. Let’s take a closer look at the actual ‘forwarding’ of IP packets.

There are different switching methods to forward IP packets. Here are the different switching options:

• Process switching:
  • All packets are examined by the CPU and all forwarding decisions are made in software…very slow!
• Fast switching (also known as route caching):
  • The first packet in a flow is examined by the CPU; the forwarding decision is cached in hardware for the next packets in the same flow. This is a faster method.
• (CEF) Cisco Express Forwarding (also known as topology based switching):
  • Forwarding table created in hardware beforehand. All packets will be switched using hardware. This is the fastest method but there are some limitations. Multilayer switches and routers use CEF.

When using process switching the router will remove the header for each Ethernet frame, look for the destination IP address in the routing table for each IP packet and then forward the Ethernet frame with the rewritten MAC addresses and CRC to the outgoing interface. Everything is done in software so this is very CPU-intensive.