BGP PIC (Prefix Independent Convergence) Core & Edge

PIC (Prefix Independent Convergence) is a feature to decrease the data plane convergence time. There are two flavors:

  • BGP PIC Core: decreases convergence time when a core router fails and your IGP has to find a new best path to your PE router.
  • BGP PIC Edge: decreases convergence time when a PE router fails and BGP has to switch to a different PE router.

PIC is best explained by showing it in action. We’ll use the following topology:

Bgp Pic Topology Ce Pe P Routers

Above, we have a small provider network in AS 2 with routers running iBGP. P2 is our route reflector in this network. We have two customers, CE1 and CE2. CE2 is advertising two prefixes in BGP. Within AS 2, we use OSPF as the IGP. I increased the cost on the interfaces of the P2 router so that P1 is the preferred path.

Configurations

Want to take a look for yourself? Here you will find the startup configuration of each device.

CE1

hostname CE1
!
ip cef

!
interface GigabitEthernet0/1
 ip address 192.168.12.1 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.13.1 255.255.255.0
!
router bgp 1
 bgp router-id 1.1.1.1
 neighbor 192.168.12.2 remote-as 2
 neighbor 192.168.13.3 remote-as 2
!
end

CE2

hostname CE2
!
ip cef
!
interface Loopback0
 ip address 8.8.8.8 255.255.255.255
!
interface Loopback1
 ip address 88.88.88.88 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.68.8 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.78.8 255.255.255.0
!
router bgp 8
 bgp router-id 8.8.8.8
 network 8.8.8.8 mask 255.255.255.255
 network 88.88.88.88 mask 255.255.255.255
 neighbor 192.168.68.6 remote-as 2
 neighbor 192.168.78.7 remote-as 2
!
end

P1

hostname P1
!
ip cef
!
interface Loopback0
 ip address 4.4.4.4 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.24.4 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.46.4 255.255.255.0
!
interface GigabitEthernet0/3
 ip address 192.168.34.4 255.255.255.0
!
interface GigabitEthernet0/4
 ip address 192.168.47.4 255.255.255.0
!
router ospf 1
 network 4.4.4.4 0.0.0.0 area 0
 network 192.168.24.0 0.0.0.255 area 0
 network 192.168.34.0 0.0.0.255 area 0
 network 192.168.46.0 0.0.0.255 area 0
 network 192.168.47.0 0.0.0.255 area 0
!
router bgp 2
 bgp router-id 4.4.4.4
 neighbor 5.5.5.5 remote-as 2
 neighbor 5.5.5.5 update-source Loopback0
!
end

P2

hostname P2
!
ip cef
!
interface Loopback0
 ip address 5.5.5.5 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.35.5 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.57.5 255.255.255.0
!
interface GigabitEthernet0/3
 ip address 192.168.25.5 255.255.255.0
!
interface GigabitEthernet0/4
 ip address 192.168.56.5 255.255.255.0
!
router ospf 1
 network 5.5.5.5 0.0.0.0 area 0
 network 192.168.25.0 0.0.0.255 area 0
 network 192.168.35.0 0.0.0.255 area 0
 network 192.168.56.0 0.0.0.255 area 0
 network 192.168.57.0 0.0.0.255 area 0
!
router bgp 2
 bgp router-id 5.5.5.5
 neighbor 2.2.2.2 remote-as 2
 neighbor 2.2.2.2 update-source Loopback0
 neighbor 2.2.2.2 route-reflector-client
 neighbor 3.3.3.3 remote-as 2
 neighbor 3.3.3.3 update-source Loopback0
 neighbor 3.3.3.3 route-reflector-client
 neighbor 4.4.4.4 remote-as 2
 neighbor 4.4.4.4 update-source Loopback0
 neighbor 4.4.4.4 route-reflector-client
 neighbor 6.6.6.6 remote-as 2
 neighbor 6.6.6.6 update-source Loopback0
 neighbor 6.6.6.6 route-reflector-client
 neighbor 7.7.7.7 remote-as 2
 neighbor 7.7.7.7 update-source Loopback0
 neighbor 7.7.7.7 route-reflector-client
!
end

PE1

hostname PE1
!
ip cef
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.24.2 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.25.2 255.255.255.0
!
interface GigabitEthernet0/3
 ip address 192.168.12.2 255.255.255.0
!
router ospf 1
 network 2.2.2.2 0.0.0.0 area 0
 network 192.168.24.0 0.0.0.255 area 0
 network 192.168.25.0 0.0.0.255 area 0
!
router bgp 2
 bgp router-id 2.2.2.2
 neighbor 5.5.5.5 remote-as 2
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 5.5.5.5 next-hop-self
 neighbor 192.168.12.1 remote-as 1
!
end

PE2

hostname PE2
!
ip cef
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.35.3 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.34.3 255.255.255.0
!
interface GigabitEthernet0/3
 ip address 192.168.13.3 255.255.255.0
!
router ospf 1
 network 3.3.3.3 0.0.0.0 area 0
 network 192.168.34.0 0.0.0.255 area 0
 network 192.168.35.0 0.0.0.255 area 0
!
router bgp 2
 bgp router-id 3.3.3.3
 neighbor 5.5.5.5 remote-as 2
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 192.168.13.1 remote-as 1
!
end

PE3

hostname PE3
!
ip cef
!
interface Loopback0
 ip address 6.6.6.6 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.46.6 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.56.6 255.255.255.0
!
interface GigabitEthernet0/3
 ip address 192.168.68.6 255.255.255.0
!
router ospf 1
 network 6.6.6.6 0.0.0.0 area 0
 network 192.168.46.0 0.0.0.255 area 0
 network 192.168.56.0 0.0.0.255 area 0
!
router bgp 2
 bgp router-id 6.6.6.6
 neighbor 5.5.5.5 remote-as 2
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 5.5.5.5 next-hop-self
 neighbor 192.168.68.8 remote-as 8
!
end

PE4

hostname PE4
!
ip cef
!
interface Loopback0
 ip address 7.7.7.7 255.255.255.255
!
interface GigabitEthernet0/1
 ip address 192.168.57.7 255.255.255.0
!
interface GigabitEthernet0/2
 ip address 192.168.47.7 255.255.255.0
!
interface GigabitEthernet0/3
 ip address 192.168.78.7 255.255.255.0
!
router ospf 1
 network 7.7.7.7 0.0.0.0 area 0
 network 192.168.47.0 0.0.0.255 area 0
 network 192.168.57.0 0.0.0.255 area 0
!
router bgp 2
 bgp router-id 7.7.7.7
 neighbor 5.5.5.5 remote-as 2
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 5.5.5.5 next-hop-self
 neighbor 192.168.78.8 remote-as 8
!
end

BGP PIC Core

PIC core helps to speed up convergence when there is a failure in the core of your network and the BGP next hop does not change.




Let’s have a look at PE1:

PE1#show ip bgp
BGP table version is 23, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>i  8.8.8.8/32       6.6.6.6                  0    100      0 8 i
 *>i  88.88.88.88/32   6.6.6.6                  0    100      0 8 i

Let’s focus on 8.8.8.8/32 and 88.88.88.88/32. These two prefixes were advertised by CE2 and PE1 learned them from P2 (our route reflector). The next hop for both prefixes is 6.6.6.6. When PE1 wants to reach CE2, our traffic path looks like this:

Bgp Pic Core Traffic Path Pe1 Ce2

How does 6.6.6.6 get resolved? Let’s have a look:

PE1#show ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 3, type intra area
  Last update from 192.168.24.4 on GigabitEthernet0/1, 00:44:32 ago
  Routing Descriptor Blocks:
  * 192.168.24.4, from 6.6.6.6, 00:44:32 ago, via GigabitEthernet0/1
      Route metric is 3, traffic share count is 1

We learn about 6.6.6.6 through OSPF and to get to this next hop, we use next hop 192.168.24.4 through interface GigabitEthernet0/1. This is what gets installed in the FIB (CEF table):

PE1#show ip cef 6.6.6.6
6.6.6.6/32
  nexthop 192.168.24.4 GigabitEthernet0/1

What about those two prefixes? This is how they show up in the FIB:

PE1#show ip cef | include 8.8
8.8.8.8/32           192.168.24.4         GigabitEthernet0/1
88.88.88.88/32       192.168.24.4         GigabitEthernet0/1

Here’s a quick summary of the recursive routing we saw:

  • BGP: to get to 8.8.8.8/32 and 88.88.88.88/32 we use next hop 6.6.6.6.
  • IGP: to get to 6.6.6.6/32 we use next hop 192.168.24.4 with interface GigabitEthernet0/1.
  • FIB:
    • 8.8.8.8/32 via 192.168.24.4 interface GigabitEthernet0/1
    • 88.88.88.88/32 via 192.168.24.4 interface GigabitEthernet0/1

For each prefix you have learned, you find an entry like this in the FIB. We only have two prefixes but if you learned a million prefixes through BGP, you’ll see them here.

Disaster strikes our provider and suddenly P1 fails:

Bgp Pic Core P1 Failure

How does this impact PE1 and how it gets to CE2?

Let’s find out! To see what happens in real-time, I’ll create an access-list that matches the next hop (6.6.6.6) and our two prefixes (8.8.8.8/32 and 88.88.88.88/32). We attach the access-list to a debug:

PE1(config)#access-list 1 permit host 6.6.6.6
PE1(config)#access-list 1 permit host 8.8.8.8
PE1(config)#access-list 1 permit host 88.88.88.88

PE1#debug ip routing 1
IP routing debugging is on for access list 1

Let’s shut all interfaces of P1 now:

P1(config)#interface range GigabitEthernet 0/1 - 4
P1(config-if-range)#shutdown

Now it’s up to OSPF to figure out that P1 is gone and find a new path to 6.6.6.6:

PE1#
RT: updating ospf 6.6.6.6/32 (0x0)  :
    via 192.168.25.5 Gi0/2  0 1048578

RT: closer admin distance for 6.6.6.6, flushing 1 routes
RT: add 6.6.6.6/32 via 192.168.25.5, ospf metric [110/4]
RT: updating bgp 8.8.8.8/32 (0x0)  :
    via 6.6.6.6   0 1048577

RT: closer admin distance for 8.8.8.8, flushing 1 routes
RT: add 8.8.8.8/32 via 6.6.6.6, bgp metric [200/0]
RT: updating bgp 88.88.88.88/32 (0x0)  :
    via 6.6.6.6   0 1048577

RT: closer admin distance for 88.88.88.88, flushing 1 routes
RT: add 88.88.88.88/32 via 6.6.6.6, bgp metric [200/0]

We see that OSPF reconverges and installs 192.168.25.5 (P2) as the new next hop to get to 6.6.6.6/32. BGP is also updated. Let’s take a look at the BGP table:

PE1#show ip bgp
BGP table version is 33, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>i  8.8.8.8/32       6.6.6.6                  0    100      0 8 i
 *>i  88.88.88.88/32   6.6.6.6                  0    100      0 8 i

Nothing changed in the BGP table, this makes sense since our next hop did not change. Only the path to get to the next hop. Let’s take a look at OSPF:

PE1#show ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 4, type intra area
  Last update from 192.168.25.5 on GigabitEthernet0/2, 00:02:06 ago
  Routing Descriptor Blocks:
  * 192.168.25.5, from 6.6.6.6, 00:02:06 ago, via GigabitEthernet0/2
      Route metric is 4, traffic share count is 1

We see that OSPF now uses 192.168.25.5 (P2) to get to 6.6.6.6/32 via the GigabitEthernet0/2 interface. Let’s take a look at the FIB for our next hop:

PE1#show ip cef 6.6.6.6
6.6.6.6/32
  nexthop 192.168.25.5 GigabitEthernet0/2

Above, we see the new next hop for 6.6.6.6/32 and interface in the FIB. Let’s take a look at the FIB entries for our two prefixes:

PE1#show ip cef | include 8.8    
8.8.8.8/32           192.168.25.5         GigabitEthernet0/2
88.88.88.88/32       192.168.25.5         GigabitEthernet0/2

The FIB entries for our two prefixes has changed as well to reflect the new next hop and outgoing interface. Think about this for a second…

Our router had to figure out a new path to get to 6.6.6.6/32 and update all entries in the FIB that used the old next hop (192.168.24.4) to the new next hop (192.168.25.5). With two prefixes, this is a piece of cake but what if we have one million prefixes in our FIB? They all have to get updated one-by-one and this takes a LONG time (minutes).

The problem here is how our FIB works. We use what we call a “flat” or “flattened” FIB structure:

We're Sorry, Full Content Access is for Members Only...

If you like to keep on reading, Become a Member Now!

  • Learn any CCNA, CCNP and CCIE R&S Topic. Explained As Simple As Possible.
  • Try for Just $1. The Best Dollar You’ve Ever Spent on Your Cisco Career!
  • Full Access to our 801 Lessons. More Lessons Added Every Week!
  • Content created by Rene Molenaar (CCIE #41726)
510 Sign Ups in the last 30 days
satisfaction-guaranteed

  • 100% Satisfaction Guaranteed!
  • You may cancel your monthly membership at any time.
  • No Questions Asked!

Tags:


Forum Replies

  1. Great lesson, Rene!

    I studied this from the Cisco documentation but they never explained in clear terms what was going on, but you managed it!

    Small typo (should be 88.88.88.88/32?):

    88.88.88.88/43 via 192.168.24.4 interface GigabitEthernet0/1

    Also could you provide the configs for PIC edge, at the end?

    ===

    Just came across this quote from Cisco:

    PIC core ensures fast convergence for BGP routes when there is a link or node failure in the core that causes a change in the IGP reachability to a remote BGP next-hop address.

    PIC edge ensures fast convergence to a

    ... Continue reading in our forum

  2. Hello Chris

    Thanks for pointing that out, I’ll let Rene know…

    I’ll also let him know about the request for the configurations for PIC edge.

    The Cisco comment is indeed correct. Even though iBGP is run between PE

    ... Continue reading in our forum

  3. Thanks Laz.

    Still don’t really agree though - PIC edge is clearly addressing data plane convergence for IBGP neighbours in a service provider environment.

    I just can’t see how that can be possible be called “external neighbours”, which implies the BGP neighbour operates in a different AS when in fact it doesn’t. The external customer may not even be running BGP!

    Oh well!

  4. Hi Rene,
    Wonderful explanation, probably not found anywhere.
    My question is until when PE1 will use PE4 as NH in PIC edge? In case PE3 is up again, will it be revertive ?
    2nd question-: if MPLS is enabled in ISP, will we have H-LFIB as well?

25 more replies! Ask a question or join the discussion by visiting our Community Forum