Lesson Contents
In the first OSPF troubleshooting lesson I explained how to fix problems with the OSPF neighbor adjacency. Once this is working, you can focus on problems with route advertisements. Most of the times you are expecting to see something in the routing table while it’s not there or you see something different than what you are expecting.
This lesson explains some of the most common issues and how to fix them.
OSPF Network Command
Let’s start with a simple scenario:
Let’s start with an easy scenario. 2 routers running single area OSPF and each router has a loopback interface advertised in OSPF. Here’s the output of the routing tables:
R1#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
1.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O 2.2.2.2/32 [110/2] via 192.168.12.2, 00:00:14, FastEthernet0/0
C 1.1.1.0/24 is directly connected, Loopback0
R2#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
2.0.0.0/24 is subnetted, 1 subnets
C 2.2.2.0 is directly connected, Loopback0
We can see that R1 has learned about network 2.2.2.2 /32 from R2 but there’s nothing in the routing table of R2. What’s going on? Let’s check if OSPF is enabled:
R1#show ip ospf interface loopback 0
%OSPF: OSPF not enabled on Loopback0
R2#show ip ospf interface loopback 0
Loopback0 is up, line protocol is up
Internet Address 2.2.2.2/24, Area 0
Process ID 1, Router ID 2.2.2.2, Network Type LOOPBACK, Cost: 1
Loopback interface is treated as a stub Host
We can see that OSPF is not enabled on the loopback0 interface of R1, so what networks are we advertising then?
R1#show ip protocols | begin Networks
Routing for Networks:
10.1.1.1 0.0.0.0 area 0
192.168.12.0 0.0.0.255 area 0
Looks like we are advertising network 10.10.1.1 /32 but this network is not configured on any interface. Network 1.1.1.1 /32 is configured on the loopback0 interface of R1. Here’s the configuration:
R1#show run | section router ospf
router ospf 1
log-adjacency-changes
network 10.1.1.1 0.0.0.0 area 0
network 192.168.12.0 0.0.0.255 area 0
Here you can see the wrong network command, let’s get rid of it:
R1(config)#router ospf 1
R1(config-router)#no network 10.1.1.1 0.0.0.0 area 0
R1(config-router)#network 1.1.1.0 0.0.0.255 area 0
We’ll make sure we have the correct network command configured here. Now take a look at R2:
R2#show ip route | include 1.1.1.
O 1.1.1.1/32 [110/2] via 192.168.12.1, 00:01:29, FastEthernet0/0
Problem solved! This issue might sound a bit lame but using the wrong network statements is something that happens all the time. Especially if we are using smaller subnets ( like /27 or /28 or something) people tend to make errors with the wildcard masks.
Lesson learned: Make sure you configure the correct network address, wildcard bits and area.
OSPF Interface Issue
Next scenario, same two routers:
Here are the routing tables:
R1#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O 2.2.2.2/32 [110/2] via 192.168.12.2, 00:00:14, FastEthernet0/0
R2#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 2.2.2.0 is directly connected, Loopback0
Once again R2 has not learned about network 1.1.1.0 /24. Note that R1 doesn’t have network 1.1.1.0 /24 in its routing table as directly connected. Let’s take a look at the OSPF configuration:
R1#show ip protocols | begin Networks
Routing for Networks:
1.1.1.0 0.0.0.255 area 0
192.168.12.0 0.0.0.255 area 0
We can verify that R1 is using the correct network command. Since R1 doesn’t even have the network in its routing table we can assume that something is wrong with the interface. Let’s take a look:
R1#show ip interface brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 192.168.12.1 YES manual up up
Loopback0 1.1.1.1 YES manual administratively down down
Seems someone had a TGIF moment and forgot to do a “no shutdown” on the interface. Let’s fix that:
R1(config)#interface loopback 0
R1(config-if)#no shutdown
That should fix the problem, take a look at the routing table of R2:
R2#show ip route | include 1.1.1
O 1.1.1.1/32 [110/2] via 192.168.12.1, 00:00:32, FastEthernet0/0
And now it shows up in the routing table of R2.
Lesson learned: You can not advertise what you don’t have!
OSPF Distribute-List
Next issue! Same two routers:
Single area, same 2 routers…we would like to have “full connectivity” but something is off…here is the output of the routing tables:
R1#show ip route ospf
R2#show ip route ospf
1.0.0.0/32 is subnetted, 1 subnets
O 1.1.1.1 [110/2] via 192.168.12.1, 00:08:46, FastEthernet0/0
R1 doesn’t show any OSPF routes, R2 does…let’s find out what is going wrong:
R2#show ip protocols
Routing Protocol is "ospf 1"
Outgoing update filter list for all interfaces is not set
Incoming update filter list for all interfaces is not set
Router ID 2.2.2.2
Number of areas in this router is 1. 1 normal 0 stub 0 nssa
Maximum path: 4
Routing for Networks:
2.2.2.0 0.0.0.255 area 0
192.168.12.0 0.0.0.255 area 0
I can take a quick look at R2 to confirm that it’s indeed advertising the correct network(s) which is the case. Let’s take a look at R1:
R1#show ip protocols
Routing Protocol is "ospf 1"
Outgoing update filter list for all interfaces is not set
Incoming update filter list for all interfaces is 1
Router ID 1.1.1.1
Number of areas in this router is 1. 1 normal 0 stub 0 nssa
Maximum path: 4
Routing for Networks:
1.1.1.0 0.0.0.255 area 0
192.168.12.0 0.0.0.255 area 0
Reference bandwidth unit is 100 mbps
Routing Information Sources:
Gateway Distance Last Update
2.2.2.2 110 00:07:23
192.168.12.2 110 00:24:54
Distance: (default is 110)
The output of R1 is more interesting…you can see that it has an inbound distribute-list configured. Let’s take a closer look at it:
R1#show access-lists
Standard IP access list 1
10 deny 2.2.2.0, wildcard bits 0.0.0.255 (2 matches)
20 permit any
Here’s our problem. Let’s get rid of the distribute-list:
R1(config)#router ospf 1
R1(config-router)#no distribute-list 1 in
This will disable it. Now look at the routing table again:
R1#show ip route ospf
2.0.0.0/32 is subnetted, 1 subnets
O 2.2.2.2 [110/2] via 192.168.12.2, 00:00:24, FastEthernet0/0
Problem solved!
Lesson learned: Be aware of distribute-lists that prevent the advertising and/or installing of prefixes in the routing table.
OSPF Virtual Link
Let’s look at some more advanced OSPF issues:
In the picture above we have R1 and R2 but this time we have a multi-area OSPF configuration. Here’s the OSPF configuration of both routers:
R1#show run | section router ospf
router ospf 1
log-adjacency-changes
area 12 virtual-link 192.168.12.2
network 1.1.1.0 0.0.0.255 area 0
network 192.168.12.0 0.0.0.255 area 12
R2#show run | section ospf
router ospf 1
log-adjacency-changes
area 12 virtual-link 192.168.12.1
network 2.2.2.0 0.0.0.255 area 2
network 192.168.12.0 0.0.0.255 area 12
As you can see all networks have been advertised. Area 2 is not directly connected to area 0 so a virtual link has been created. However, not everything is working as expected. Take a look at the routing tables:
R1#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
1.0.0.0/24 is subnetted, 1 subnets
C 1.1.1.0 is directly connected, Loopback0
R2#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
1.0.0.0/32 is subnetted, 1 subnets
O IA 1.1.1.1 [110/2] via 192.168.12.1, 00:07:34, FastEthernet0/0
2.0.0.0/24 is subnetted, 1 subnets
C 2.2.2.0 is directly connected, Loopback0
R1 didn’t learn about network 2.2.2.0 /24 from R2 but R2 did learn about network 1.1.1.0 /24. Since area 2 is not directly connected to area 0 we require the virtual link. Let’s see if it’s working or not:
R1#show ip ospf virtual-links
Virtual Link OSPF_VL1 to router 192.168.12.2 is down
R2#show ip ospf virtual-links
Virtual Link OSPF_VL0 to router 192.168.12.1 is down
Hmm this doesn’t look good. The virtual link is down. Note the IP addresses that we see here, those are the IP addresses that are configured on the FastEthernet interfaces of both routers. Is this correct? Take a look below:
R1#show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
2.2.2.2 1 FULL/DR 00:00:31 192.168.12.2 FastEthernet0/0
R2#show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
1.1.1.1 1 FULL/BDR 00:00:38 192.168.12.1 FastEthernet0/0
Whenever we configure a virtual link we need to configure the OSPF router ID of the other side, not the IP address of the other side! Take a look at the virtual link configuration again:
R1#show run | incl virtual-link
area 12 virtual-link 192.168.12.2
R2#show run | incl virtual-link
area 12 virtual-link 192.168.12.1
Let’s get rid of this configuration and use the router IDs for the virtual link:
R1(config)#router ospf 1
R1(config-router)#no area 12 virtual-link 192.168.12.2
R1(config-router)#area 12 virtual-link 2.2.2.2
R2(config)#router ospf 1
R2(config-router)#no area 12 virtual-link 192.168.12.1
R2(config-router)#area 12 virtual-link 1.1.1.1
This is what the virtual link should look like, configured between the OSPF router IDs. Our work is rewarded with a nice message on the consoles:
R1# %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on OSPF_VL2 from LOADING to FULL, Loading Done
R2# OSPF-5-ADJCHG: Process 1, Nbr 1.1.1.1 on OSPF_VL1 from LOADING to FULL, Loading Done
Right after typing in the correct commands you’ll see this message on your console. Did this help?
R1#show ip route ospf
2.0.0.0/32 is subnetted, 1 subnets
O IA 2.2.2.2 [110/2] via 192.168.12.2, 00:01:38, FastEthernet0/0
Now we see the OSPF entry for network 2.2.2.0 /24. Problem solved!
Lesson learned: When you configure an OSPF virtual link, always use the router IDs.
OSPF Stub Area
Let’s take a look at another issue:
Same two routers but there’s an “external routing domain”. This could be another routing protocol like RIP or EIGRP, it’s something that we will redistribute into OSPF. R2 will redistribute network 2.2.2.0 /24 into OSPF but for some reason it’s not showing up on R1. To keep things exciting I’m not going to show you the OSPF configuration of both routers…
Let’s look at the routing tables:
R1#show ip route
C 192.168.12.0/24 is directly connected, FastEthernet0/0
1.0.0.0/24 is subnetted, 1 subnets
C 1.1.1.0 is directly connected, Loopback0
I’m not seeing network 2.2.2.0 /24 on R1 so let’s take a look at R2:
R2#show ip route | include 2.2.2
C 2.2.2.0 is directly connected, Loopback0
I can see the network is in the routing table of R2 as directly connected. Let’s see if R2 is redistributing this network into OSPF:
R2#show ip protocols
Routing Protocol is "ospf 1"
Outgoing update filter list for all interfaces is not set
Incoming update filter list for all interfaces is not set
Router ID 2.2.2.2
It is an autonomous system boundary router
Redistributing External Routes from,
connected, includes subnets in redistribution
I can see R2 has been configured to redistribute directly connected networks. This should include network 2.2.2.0 /24 from the loopback0 interface. Let’s see if we can find this network in the LSDB of R2:
R2#show ip ospf database external
OSPF Router with ID (2.2.2.2) (Process ID 1)
I don’t find anything in the OSPF database? What could be a possible reason for this? Let’s find out what kind of area this is:
R2#show ip protocols | include areas
Number of areas in this router is 1. 0 normal 1 stub 0 nssa
Ahhh that explains, it’s a stub area! Stub areas do not allow LSA type 5 (external routes). We can either turn this area into a normal area or a NSSA. Let’s turn it into a NSSA:
R1(config)#router ospf 1
R1(config-router)#no area 12 stub
R1(config-router)#area 12 nssa
R2(config)#router ospf 1
R2(config-router)#no area 12 stub
R2(config-router)#area 12 nssa
I’ll change the area type on both routers. The NSSA area allows external routes by using LSA type 7. Take a look at the LSDB again:
Thank you for explaining OSPF troubleshooting in a simple language. This helps in understanding OSPF better.
The first scenario doesn’t match with the troubleshooting. For example, “We can see that R1 has learned about network 10.2.2.0 /24 from R2 but there’s nothing in the routing table of R2. What’s going on? Let’s check if OSPF is enabled:”. There is nowhere on the scenario that states, configure network address 10.2.2.0 /24, so I’m not expecting to see network 10.2.2.0 /24 in my routing table.
Hi Alex,
You are right, this should be the 2.2.2.0/24 network. I just fixed it in the article. Thanks for letting me know!
Rene
Super Explanation
Completely agree , Super Explanation !
Nice concept to include such a topic and very well explained !