Troubleshooting OSPF Route Advertisement

In the first OSPF troubleshooting lesson I explained how to fix problems with the OSPF neighbor adjacency. Once this is working, you can focus on problems with route advertisements. Most of the times you are expecting to see something in the routing table while it’s not there or you see something different than what you are expecting.

This lesson explains some of the most common issues and how to fix them.

OSPF Network Command

Let’s start with a simple scenario:

Let’s start with an easy scenario. 2 routers running single area OSPF and each router has a loopback interface advertised in OSPF. Here’s the output of the routing tables:

R1#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O       2.2.2.2/32 [110/2] via 192.168.12.2, 00:00:14, FastEthernet0/0
C       1.1.1.0/24 is directly connected, Loopback0

R2#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     2.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0

We can see that R1 has learned about network 2.2.2.2 /32 from R2 but there’s nothing in the routing table of R2. What’s going on? Let’s check if OSPF is enabled:

R1#show ip ospf interface loopback 0
%OSPF: OSPF not enabled on Loopback0

R2#show ip ospf interface loopback 0
Loopback0 is up, line protocol is up 
  Internet Address 2.2.2.2/24, Area 0 
  Process ID 1, Router ID 2.2.2.2, Network Type LOOPBACK, Cost: 1
  Loopback interface is treated as a stub Host

We can see that OSPF is not enabled on the loopback0 interface of R1, so what networks are we advertising then?

R1#show ip protocols | begin Networks
  Routing for Networks:
    10.1.1.1 0.0.0.0 area 0
    192.168.12.0 0.0.0.255 area 0

Looks like we are advertising network 10.10.1.1 /32 but this network is not configured on any interface. Network 1.1.1.1 /32 is configured on the loopback0 interface of R1. Here’s the configuration:

R1#show run | section router ospf
router ospf 1
 log-adjacency-changes
 network 10.1.1.1 0.0.0.0 area 0
 network 192.168.12.0 0.0.0.255 area 0

Here you can see the wrong network command, let’s get rid of it:

R1(config)#router ospf 1
R1(config-router)#no network 10.1.1.1 0.0.0.0 area 0        
R1(config-router)#network 1.1.1.0 0.0.0.255 area 0

We’ll make sure we have the correct network command configured here. Now take a look at R2:

R2#show ip route | include 1.1.1. 
O       1.1.1.1/32 [110/2] via 192.168.12.1, 00:01:29, FastEthernet0/0

Problem solved! This issue might sound a bit lame but using the wrong network statements is something that happens all the time. Especially if we are using smaller subnets ( like /27 or /28 or something) people tend to make errors with the wildcard masks.

Lesson learned: Make sure you configure the correct network address, wildcard bits and area.

OSPF Interface Issue

Next scenario, same two routers:

Here are the routing tables:

R1#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O       2.2.2.2/32 [110/2] via 192.168.12.2, 00:00:14, FastEthernet0/0

R2#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     10.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0

Once again R2 has not learned about network 1.1.1.0 /24. Note that R1 doesn’t have network 1.1.1.0 /24 in its routing table as directly connected. Let’s take a look at the OSPF configuration:

R1#show ip protocols | begin Networks
  Routing for Networks:
    1.1.1.0 0.0.0.255 area 0
    192.168.12.0 0.0.0.255 area 0

We can verify that R1 is using the correct network command. Since R1 doesn’t even have the network in its routing table we can assume that something is wrong with the interface. Let’s take a look:

R1#show ip interface brief 
Interface               IP-Address      OK? Method Status                Protocol
FastEthernet0/0         192.168.12.1    YES manual up                    up      
Loopback0               1.1.1.1        YES manual administratively down down

Seems someone had a TGIF moment and forgot to do a “no shutdown” on the interface. Let’s fix that:

R1(config)#interface loopback 0
R1(config-if)#no shutdown

That should fix the problem, take a look at the routing table of R2:

R2#show ip route | include 1.1.1
O       1.1.1.1/32 [110/2] via 192.168.12.1, 00:00:32, FastEthernet0/0

And now it shows up in the routing table of R2.

Lesson learned: You can not advertise what you don’t have!

OSPF Distribute-List

Next issue! Same two routers:

Single area, same 2 routers…we would like to have “full connectivity” but something is off…here is the output of the routing tables:

R1#show ip route ospf

R2#show ip route ospf 
     1.0.0.0/32 is subnetted, 1 subnets
O       1.1.1.1 [110/2] via 192.168.12.1, 00:08:46, FastEthernet0/0

R1 doesn’t show any OSPF routes, R2 does…let’s find out what is going wrong:

R2#show ip protocols 
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 2.2.2.2
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    2.2.2.0 0.0.0.255 area 0
    192.168.12.0 0.0.0.255 area 0

I can take a quick look at R2 to confirm that it’s indeed advertising the correct network(s) which is the case. Let’s take a look at R1:

R1#show ip protocols 
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is 1
  Router ID 1.1.1.1
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    1.1.1.0 0.0.0.255 area 0
    192.168.12.0 0.0.0.255 area 0
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
    2.2.2.2              110      00:07:23
    192.168.12.2         110      00:24:54
  Distance: (default is 110)

The output of R1 is more interesting…you can see that it has an inbound distribute-list configured. Let’s take a closer look at it:

R1#show access-lists 
Standard IP access list 1
    10 deny   2.2.2.0, wildcard bits 0.0.0.255 (2 matches)
    20 permit any

Here’s our problem. Let’s get rid of the distribute-list:

R1(config)#router ospf 1
R1(config-router)#no distribute-list 1 in

This will disable it. Now look at the routing table again:

R1#show ip route ospf 
     2.0.0.0/32 is subnetted, 1 subnets
O       2.2.2.2 [110/2] via 192.168.12.2, 00:00:24, FastEthernet0/0

Problem solved!

Lesson learned: Be aware of distribute-lists that prevent the advertising and/or installing of prefixes in the routing table.

OSPF Virtual Link

Let’s look at some more advanced OSPF issues:

 

In the picture above we have R1 and R2 but this time we have a multi-area OSPF configuration. Here’s the OSPF configuration of both routers:

R1#show run | section router ospf
router ospf 1
 log-adjacency-changes
 area 12 virtual-link 192.168.12.2
 network 1.1.1.0 0.0.0.255 area 0
 network 192.168.12.0 0.0.0.255 area 12

R2#show run | section ospf
router ospf 1
 log-adjacency-changes
 area 12 virtual-link 192.168.12.1
 network 2.2.2.0 0.0.0.255 area 2
 network 192.168.12.0 0.0.0.255 area 12

As you can see all networks have been advertised. Area 2 is not directly connected to area 0 so a virtual link has been created. However, not everything is working as expected. Take a look at the routing tables:

R1#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/24 is subnetted, 1 subnets
C       1.1.1.0 is directly connected, Loopback0

R2#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/32 is subnetted, 1 subnets
O IA    1.1.1.1 [110/2] via 192.168.12.1, 00:07:34, FastEthernet0/0
     2.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0

R1 didn’t learn about network 2.2.2.0 /24 from R2 but R2 did learn about network 1.1.1.0 /24. Since area 2 is not directly connected to area 0 we require the virtual link. Let’s see if it’s working or not:

R1#show ip ospf virtual-links 
Virtual Link OSPF_VL1 to router 192.168.12.2 is down

R2#show ip ospf virtual-links 
Virtual Link OSPF_VL0 to router 192.168.12.1 is down

Hmm this doesn’t look good. The virtual link is down. Note the IP addresses that we see here, those are the IP addresses that are configured on the FastEthernet interfaces of both routers. Is this correct? Take a look below:

R1#show ip ospf neighbor 

Neighbor ID  Pri   State           Dead Time   Address         Interface
2.2.2.2       1   FULL/DR         00:00:31    192.168.12.2    FastEthernet0/0

R2#show ip ospf neighbor 

Neighbor ID  Pri   State           Dead Time   Address         Interface
1.1.1.1       1   FULL/BDR        00:00:38    192.168.12.1    FastEthernet0/0

Whenever we configure a virtual link we need to configure the OSPF router ID of the other side, not the IP address of the other side! Take a look at the virtual link configuration again:

R1#show run | incl virtual-link
 area 12 virtual-link 192.168.12.2

R2#show run | incl virtual-link
 area 12 virtual-link 192.168.12.1

Let’s get rid of this configuration and use the router IDs for the virtual link:

R1(config)#router ospf 1
R1(config-router)#no area 12 virtual-link 192.168.12.2
R1(config-router)#area 12 virtual-link 2.2.2.2

R2(config)#router ospf 1
R2(config-router)#no area 12 virtual-link 192.168.12.1
R2(config-router)#area 12 virtual-link 1.1.1.1

This is what the virtual link should look like, configured between the OSPF router IDs. Our work is rewarded with a nice message on the consoles:

R1# %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on OSPF_VL2 from LOADING to FULL, Loading Done

R2# OSPF-5-ADJCHG: Process 1, Nbr 1.1.1.1 on OSPF_VL1 from LOADING to FULL, Loading Done

Right after typing in the correct commands you’ll see this message on your console. Did this help?

R1#show ip route ospf
     2.0.0.0/32 is subnetted, 1 subnets
O IA    2.2.2.2 [110/2] via 192.168.12.2, 00:01:38, FastEthernet0/0

Now we see the OSPF entry for network 2.2.2.0 /24. Problem solved!

Lesson learned: When you configure an OSPF virtual link, always use the router IDs.

OSPF Stub Area

Let’s take a look at another issue:

Same two routers but there’s an “external routing domain”. This could be another routing protocol like RIP or EIGRP, it’s something that we will redistribute into OSPF. R2 will redistribute network 2.2.2.0 /24 into OSPF but for some reason it’s not showing up on R1. To keep things exciting I’m not going to show you the OSPF configuration of both routers…

Let’s look at the routing tables:

R1#show ip route

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/24 is subnetted, 1 subnets
C       1.1.1.0 is directly connected, Loopback0

I’m not seeing network 2.2.2.0 /24 on R1 so let’s take a look at R2:

R2#show ip route | include 2.2.2
C       2.2.2.0 is directly connected, Loopback0

I can see the network is in the routing table of R2 as directly connected. Let’s see if R2 is redistributing this network into OSPF:

R2#show ip protocols 
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 2.2.2.2
  It is an autonomous system boundary router
  Redistributing External Routes from,
    connected, includes subnets in redistribution

I can see R2 has been configured to redistribute directly connected networks. This should include network 2.2.2.0 /24 from the loopback0 interface. Let’s see if we can find this network in the LSDB of R2:

R2#show ip ospf database external 

            OSPF Router with ID (2.2.2.2) (Process ID 1)

I don’t find anything in the OSPF database? What could be a possible reason for this? Let’s find out what kind of area this is:

R2#show ip protocols | include areas
  Number of areas in this router is 1. 0 normal 1 stub 0 nssa

Ahhh that explains, it’s a stub area! Stub areas do not allow LSA type 5 (external routes). We can either turn this area into a normal area or a NSSA. Let’s turn it into a NSSA:

R1(config)#router ospf 1
R1(config-router)#no area 12 stub
R1(config-router)#area 12 nssa

R2(config)#router ospf 1
R2(config-router)#no area 12 stub
R2(config-router)#area 12 nssa

I’ll change the area type on both routers. The NSSA area allows external routes by using LSA type 7. Take a look at the LSDB again: