BGP Prefix Origin AS Validation with RPKI

Resource Public Key Infrastructure (RPKI) is used to validate whether certain prefixes belong to the AS that advertises them. Before you continue with this lesson, make sure you have a good understanding of what BGP route leaking is and how RPKI can help.

In this lesson, we’ll configure an RPKI validator using Docker and make a Cisco IOS XE router validated prefixes. We’ll see how the router deals with valid, invalid, and not found prefixes. You’ll also learn how to forward this RPKI state to other routers.

Configuration

Let’s start with our topology:

Let me explain what we have:

• Routinator is open-source RPKI relying party software (RPKI validator) which downloads and verifies RPKI data.
• R1 and R2 are in AS 12 and will use RPKI to validate routes.
• R4 and R5 advertise prefixes to R1 through eBGP.
• R3 is only needed so I can reach Routinator from R1.

I use CSR1000v routers running Cisco IOS Software [Amsterdam], Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 17.3.6.

hostname R1
!
interface GigabitEthernet1
 ip address 192.168.1.1 255.255.255.0
!
interface GigabitEthernet2
 ip address 192.168.12.1 255.255.255.0
!
router bgp 12
 neighbor 192.168.1.4 remote-as 8075
 neighbor 192.168.1.5 remote-as 29256
 neighbor 192.168.12.2 remote-as 12
 neighbor 192.168.12.2 next-hop-self
 neighbor 192.168.12.2 send-community extended
 neighbor 192.168.12.2 announce rpki state
!
ip route 10.65.90.0 255.255.255.0 192.168.1.3
!
end

hostname R2
!
interface GigabitEthernet1
 ip address 192.168.12.2 255.255.255.0
!
router bgp 12
 neighbor 192.168.12.1 remote-as 12
!
end

hostname R4
!
interface Loopback0
 ip address 4.128.0.4 255.240.0.0
!
interface Loopback1
 ip address 5.0.0.5 255.255.224.0
!
interface Loopback2
 ip address 44.44.44.44 255.255.255.255
!
interface GigabitEthernet1
 ip address 192.168.1.4 255.255.255.0
!
router bgp 8075
 network 4.128.0.0 mask 255.240.0.0
 network 5.0.0.0 mask 255.255.224.0
 network 44.44.44.44 mask 255.255.255.255
 neighbor 192.168.1.1 remote-as 12
!
end

hostname R5
!
interface Loopback0
 ip address 5.0.0.5 255.255.224.0
!
interface GigabitEthernet1
 ip address 192.168.1.5 255.255.255.0
!
router bgp 29256
 network 5.0.0.0 mask 255.255.224.0
 neighbor 192.168.1.1 remote-as 12
 neighbor 192.168.1.1 route-map AS_PATH_PREPEND out
!
route-map AS_PATH_PREPEND permit 10 
 set as-path prepend 29256 29256
!
end

version: '3'

services:
  routinator:
    image: nlnetlabs/routinator
    container_name: routinator
    restart: unless-stopped
    ports:
      - "3323:3323"
      - "8323:8323"

Routinator

My docker container with Routinator is up and running.

The GUI listens on HTTP TCP port 8323. Let’s have a look. Under Prefix Check, you can enter a prefix and AS number to see whether it’s valid, invalid, or not found:

This prefix checks out. Under Metrics, you can see how many Route Origin Authorizations (ROA) there are per Regional Internet Registry (RIR) and how many have been validated (VRPs):

Under Repositories, you can see all the RPKI repositories from the different RIRs:

Under Connections, you see how Routinator communicates with the repositories. This can be RPKI Repository Delta Protocol (RRDP) or Rsync:

Our Routinator docker instance is working well. There’s nothing we have to configure here.

Router

Let’s configure our router. Before I enable route validation, I’ll add this command on R1:

R1(config)#router bgp 12
R1(config-router)#bgp bestpath prefix-validate disable

This will ensure that once I connect to Routinator, R1 will check prefixes, but it won’t change the BGP table yet.

Next, I’ll connect to Routinator:

R1(config)#router bgp 12
R1(config-router)#bgp rpki server tcp 10.65.90.50 port 3323 refresh 300

Routinator uses TCP port 3323 for the RPKI-to-Router protocol (RTR).

Once connected, the router will now download a list of prefixes and the ASes they belong to. The server will send incremental updates, but the router can query the server for a refresh as well. I set the refresh time to 300 seconds, but on a production network, once every hour is probably more than enough.

Let’s check if R1 is connected to Routinator:

R1#show ip bgp rpki servers 
BGP SOVC neighbor is 10.65.90.50/3323 connected to port 3323
Flags 64, Refresh time is 300, Serial number is 135, Session ID is 11494
InQ has 0 messages, OutQ has 0 messages, formatted msg 2
Session IO flags 3, Session flags 4008
 Neighbor Statistics:
  Prefixes 500539
  Connection attempts: 20
  Connection failures: 18
  Errors sent: 0
  Errors received: 0
          
Connection state is ESTAB, I/O status: 1, unread input bytes: 0            
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: 192.168.1.1, Local port: 53929
Foreign host: 10.65.90.50, Foreign port: 3323
Connection tableid (VRF): 0
Maximum output segment queue size: 50

Enqueued packets for retransmit: 0, input: 0  mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x3F80132):
Timer          Starts    Wakeups            Next
Retrans             4          1             0x0
TimeWait            0          0             0x0
AckHold          7068          4             0x0
SendWnd             0          0             0x0
KeepAlive        8171          0       0x3F80628
GiveUp              0          0             0x0
PmtuAger            1          0       0x3FAACF2
DeadWait            0          0             0x0
Linger              0          0             0x0
ProcessQ            0          0             0x0

iss:  341721583  snduna:  341721604  sndnxt:  341721604
irs: 2065660931  rcvnxt: 2076787572

sndwnd:  64220  scale:      0  maxrcvwnd:  16384
rcvwnd:  15652  scale:      0  delrcvwnd:    732

SRTT: 234 ms, RTTO: 2984 ms, RTV: 2750 ms, KRTT: 0 ms
minRTT: 1 ms, maxRTT: 1000 ms, ACK hold: 200 ms
uptime: 427282 ms, Sent idletime: 14052 ms, Receive idletime: 14050 ms 
Status Flags: active open
Option Flags: keepalive running, nagle, path mtu capable
IP Precedence value : 6

Datagrams (max data segment is 1460 bytes):
Rcvd: 8151 (out of order: 0), with data: 8124, total data bytes: 11126640
Sent: 8215 (retransmit: 1, fastretransmit: 0, partialack: 0, Second Congestion: 0), with data: 2, total data bytes: 20

 Packets received in fast path: 0, fast processed: 0, slow path: 0
 fast lock acquisition failures: 0, slow path: 0
TCP Semaphore      0x7F272B14F018  FREE

We are connected. The number of prefixes (500539) is something to keep an eye on. It can take a few minutes for the router to download everything. We can see a list of all prefixes and AS numbers using this command:

R1#show ip bgp rpki table 
367569 BGP sovc network entries using 58811040 bytes of memory
407525 BGP sovc record entries using 13040800 bytes of memory

Network              Maxlen  Origin-AS  Source  Neighbor
1.0.0.0/24           24      13335      0       10.65.90.50/3323
1.0.4.0/24           24      38803      0       10.65.90.50/3323
1.0.4.0/22           22      38803      0       10.65.90.50/3323
1.0.5.0/24           24      38803      0       10.65.90.50/3323
1.0.6.0/24           24      38803      0       10.65.90.50/3323
1.0.7.0/24           24      38803      0       10.65.90.50/3323
1.0.64.0/18          18      18144      0       10.65.90.50/3323
1.1.1.0/24           24      13335      0       10.65.90.50/3323
1.1.4.0/22           22      4134       0       10.65.90.50/3323
1.1.16.0/20          20      4134       0       10.65.90.50/3323
1.2.9.0/24           24      4134       0       10.65.90.50/3323
1.2.10.0/24          24      4134       0       10.65.90.50/3323
1.2.11.0/24          24      4134       0       10.65.90.50/3323
1.2.12.0/22          22      4134       0       10.65.90.50/3323
1.3.0.0/16           16      4134       0       10.65.90.50/3323
1.6.0.0/22           24      9583       0       10.65.90.50/3323

This is looking good. Let’s find out how R1 uses this information by checking the BGP table:

R1#show ip bgp
BGP table version is 10, local router ID is 192.168.12.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>   4.128.0.0/12     192.168.1.4              0             0 8075 i
 *>   5.0.0.0/19       192.168.1.4              0             0 8075 i
 *                     192.168.1.5              0             0 29256 29256 29256 i
 *>   44.44.44.44/32   192.168.1.4              0             0 8075 i

Three prefixes have been selected as the best path. Take a closer look at 5.0.0.0/19. I advertise this prefix from R4 and R5, but I used AS path prepending on R5 so that R4 becomes the best path.

This is what the routing table looks like:

R1#show ip route bgp

      4.0.0.0/12 is subnetted, 1 subnets
B        4.128.0.0 [20/0] via 192.168.1.4, 00:07:29
      5.0.0.0/19 is subnetted, 1 subnets
B        5.0.0.0 [20/0] via 192.168.1.4, 00:03:03
      44.0.0.0/32 is subnetted, 1 subnets
B        44.44.44.44 [20/0] via 192.168.1.4, 00:14:35

Nothing new here. This is because of the bgp bestpath prefix-validate disable command we added earlier. Let’s get rid of it so you can see the difference:

R1(config)#router bgp 12
R1(config-router)#no bgp bestpath prefix-validate disable

This changes the BGP table:

R1#show ip bgp 
BGP table version is 11, local router ID is 192.168.12.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
V*>   4.128.0.0/12     192.168.1.4              0             0 8075 i
I*    5.0.0.0/19       192.168.1.4              0             0 8075 i
V*>                    192.168.1.5              0             0 29256 29256 29256 i
N*>   44.44.44.44/32   192.168.1.4              0             0 8075 i

This time, we see something different:

• 4.128.0.0/12 from AS 8075 is a valid prefix.
• 5.0.0.0/19 from AS 8075 is an invalid prefix.
• 5.0.0.0/19 from AS 29256 is a valid prefix.
• 44.44.44.44/32 from AS 8075 is a prefix that is not found.

This influences the BGP best path selection. R1 selects the longer path through R5 as the best path to get to 5.0.0.0/19. Here’s the routing table:

R1#show ip route bgp

      4.0.0.0/12 is subnetted, 1 subnets
B        4.128.0.0 [20/0] via 192.168.1.4, 00:08:50
      5.0.0.0/19 is subnetted, 1 subnets
B        5.0.0.0 [20/0] via 192.168.1.5, 00:00:32
      44.0.0.0/32 is subnetted, 1 subnets
B        44.44.44.44 [20/0] via 192.168.1.4, 00:15:56

Let’s check all prefixes in the RPKI table. Here’s the first one:

R1#show ip bgp 4.128.0.0/12 | include RPKI
      path 7F2732260020 RPKI State valid

The 4.128.0.0/12 from AS 8075 is a valid prefix. We have two entries for 5.0.0.0/19:

R1#show ip bgp 5.0.0.0/19
BGP routing table entry for 5.0.0.0/19, version 11
Paths: (2 available, best #2, table default)
  Advertised to update-groups:
     5          6         
  Refresh Epoch 1
  8075
    192.168.1.4 from 192.168.1.4 (44.44.44.44)
      Origin IGP, metric 0, localpref 100, valid, external
      path 7F27322601D0 RPKI State invalid
      rx pathid: 0, tx pathid: 0
      Updated on Jan 16 2024 10:08:41 UTC
  Refresh Epoch 1
  29256 29256 29256
    192.168.1.5 from 192.168.1.5 (5.0.0.5)
      Origin IGP, metric 0, localpref 100, valid, external, best
      path 7F2732260140 RPKI State valid
      rx pathid: 0, tx pathid: 0x0
      Updated on Jan 16 2024 10:08:40 UTC

Above, you can see that prefix 5.0.0.0/19 from AS 8075 is invalid and the one from AS 29256 is valid.

Let’s check the last prefix:

R1#show ip bgp 44.44.44.44/32 | include RPKI
      path 7F27322600B0 RPKI State not found

This prefix is not found because there is no ROA for it.

Invalid Prefixes

Let’s take a closer look at that invalid prefix. By default, a router refuses to use invalid prefixes. We can test this by shutting down R5 for now:

R5(config)#interface GigabitEthernet 1
R5(config-if)#shutdown

R1 now only has the invalid prefix 5.0.0.0/19 from AS 8075:

R1#show ip bgp 5.0.0.0/19
BGP routing table entry for 5.0.0.0/19, version 0
Paths: (1 available, no best path)
  Flag: 0x4100
  Not advertised to any peer
  Refresh Epoch 2
  8075
    192.168.1.4 from 192.168.1.4 (44.44.44.44)
      Origin IGP, metric 0, localpref 100, valid, external
      path 7F2732260140 RPKI State invalid
      rx pathid: 0, tx pathid: 0
      Updated on Jan 16 2024 10:47:03 UTC

R1 refuses to use this prefix:

R1#show ip route 5.0.0.0
% Network not in table

However, it is possible to tell R1 that you do want to use invalid prefixes. Here’s how:

R1(config)#router bgp 12
R1(config-router)#bgp bestpath prefix-validate allow-invalid

Because it’s the only option, R1 now uses this as the best path:

R1#show ip bgp 5.0.0.0/19
BGP routing table entry for 5.0.0.0/19, version 4
Paths: (1 available, best #1, table default)
  Advertised to update-groups:
     10        
  Refresh Epoch 2
  8075
    192.168.1.4 from 192.168.1.4 (44.44.44.44)
      Origin IGP, metric 0, localpref 100, valid, external, best
      path 7F2732260140 RPKI State invalid
      rx pathid: 0, tx pathid: 0x0
      Updated on Jan 16 2024 10:47:03 UTC

And installs it in the routing table:

R1#show ip route 5.0.0.0
Routing entry for 5.0.0.0/19, 1 known subnets
B        5.0.0.0 [20/0] via 192.168.1.4, 00:00:33

Let’s enable R5 again:

R5(config)#interface GigabitEthernet 1
R5(config-if)#no shutdown

Route-map

We can also use a route-map to match the different RPKI states and change one of the BGP attributes. For example, local preference. Here’s how to do it:

R1(config)#route-map RPKI permit 10
R1(config-route-map)#match rpki valid
R1(config-route-map)#set local-preference 200

R1(config)#route-map RPKI permit 20
R1(config-route-map)#match rpki not-found
R1(config-route-map)#set local-preference 100

R1(config)#route-map RPKI permit 30
R1(config-route-map)#match rpki invalid
R1(config-route-map)#set local-preference 50

R1(config)#route-map RPKI permit 40

We’ll give valid prefixes the highest local preference (200), not found the second best (100), and invalid prefixes the worst (50). Don’t forget to add an empty statement or your router will deny everything else. Let’s attach this route-map to R4 and R5:

R1(config)#router bgp 12
R1(config-router)#neighbor 192.168.1.4 route-map RPKI in
R1(config-router)#neighbor 192.168.1.5 route-map RPKI in

Let’s speed things up and reset the BGP neighbor adjacencies: