BGP Route Leaking

We use BGP on the Internet to exchange routing information between autonomous systems (AS). A route leak happens when one or more routes are advertised and accepted by ASes that shouldn’t have these routes. This can have nasty side effects like traffic delays, drops, and security issues. In this lesson, we’ll dive into BGP roles, what we should advertise, what a route leak is, why route leaks are bad, how to detect one, and how to deal with it. Let’s get started!

https://vimeo.com/901945801

Routing between ASes

When studying BGP, chances are that you only look at it from a technical perspective. The Internet might look like a huge spiderweb of ASes:

It looks like a spiderweb, but there is some hierarchy. It’s not like every AS connects with every other random AS they can connect to and let BGP figure out, using its path selection, how to get everywhere. These ASes are in some form one of the two:

• Customers requiring Internet access who pay a provider to get access to the Internet.
• ISPs that make money by selling Internet connectivity to customers.

Roles

An end user, such as an enterprise business requiring Internet access, is a customer. An ISP can be both a provider and a customer. ISPs can offer Internet to their customers but, depending on their size, require higher-tier providers to connect to the rest of the Internet.

In a nutshell, we could say that we have these roles:

• Customer
• Provider
• Peer

Here’s a visualization:

Let me explain this overview. We have ASes that pay for access and ASes that earn money. The black arrows tell who is paying whom. For example:

• AS 4, 5, 6, and 7 are customers requiring Internet access. They pay AS 2 and 3 for their services.
• AS 2 and AS 3 have two roles:
  • They are providers for customers AS 4, 5, 6, and 7.
  • They are customers of transit provider AS 1 for access to other parts of the Internet.
• AS 2 and AS 3 are also peers:
  • AS 2 can reach AS 6 and AS 7 directly through AS 3.
  • AS 3 can reach AS 4 and AS 5 directly through AS 2.
  • AS 2 and AS 3 peer with each other, so they don’t have to use transit provider AS 1 to reach each other’s networks.
  • AS 2 and AS 3 might not charge each other for traffic they send to each other.

Advertising between roles

Now we understand the roles, let’s talk about what we normally advertise.

Provider to Customer

A provider advertises Internet routes to customers. This can be a full or partial Internet routing table or a default route. The customer advertises its network and perhaps routes from its customers if it has any.

Peer to Peer

The ASes exchange routes to each other’s network and their customer’s networks but nothing else.

Route Leak Definition

You now know about the different roles and what each AS usually advertises to other ASes. When do we consider something a route leak?

RFC 7908 does a great job of explaining what a BGP route leak is. Their definition:

The scope is the BGP import and export policies of an AS that define what routes should be exchanged between the local AS and the remote AS. This is an agreement on paper between two ASes, and then you need to configure your router(s) to advertise and filter specific routes.

Route Leak Causes

We use a lot of network automation nowadays, but in the end, humans configure routers and advertise routes. A route leak can happen when someone advertises a route you are not supposed to. This can be an accident or malicious. Most of the time, it’s because of misconfiguration. You want to advertise something to one neighbor but forget to filter something outbound. It’s also possible you forgot to filter routes you receive from your neighbor.

The way BGP was designed, nothing checks whether our configuration matches the configuration of a remote router. Nothing checks our peering relationship (customer, provider, peer).

Route Leak Types

There are different types of route leaks, called classifications or types. We’ll walk through them. Type 1,2,3,4 are similar; a route leak is a violation of policy. The difference is what source AS leaks and to what destination AS (their roles).

Type 1: Hairpin Turn with Full Prefix

A multihomed AS learns a route from an upstream AS and propagates it to another upstream AS. The route hasn’t changed; it’s the same prefix, and the AS path did not change.

The multihomed AS shouldn’t advertise this prefix, and the second upstream AS should have filtered it, but it didn’t happen. This route leak is often successful because ASes are often configured to prefer customer routes over peer routes. It’s called a hairpin because the traffic goes to AS 4 and turns around the other way. This route leak often succeeds because ASes are often configured to prefer customer routes over peer routes. Traffic will be forwarded as long as AS 4 can keep up.

Type 2: Lateral ISP-ISP-ISP Leak

This route leak typically happens when you have three peers in a row. Lateral is the same thing as “peer-to-peer” or “non-transit”. Here’s an example:

This is what happens:

• AS 2 learns a prefix from its customer in AS 5.
• AS 2 forwards the prefix to:
  • AS 1 (transit provider)
  • AS 3 (peer)
• AS 3 forwards the prefix to AS 4, while it should only advertise its own prefixes.
• AS 4 might forward traffic destined for AS 5 through AS 3.

Traffic will get there, but it’s not using the intended path. Traffic from AS 4 to AS 2 should use transit provider AS 1.

Type 3: Leak of Transit-Provider Prefixes to Peer

This route leak happens when an AS leaks routes from a transit provider to a peer. Here’s an example:

AS 3 learns prefixes from its transit provider AS 1. These are forwarded to AS 2, which can now use its peer link to AS 3 to reach destinations behind AS 1. Traffic is forwarded, but this will cost AS 3 money and might overburden the link between AS 2 and AS 3.

Type 4: Leak of Peer Prefixes to Transit Provider

This route leak happens when an AS forwards routes from a peer to its transit provider. Here is an example:

AS 2 forwards a prefix from its customer in AS 4 to its transit provider AS 1 and to its peer AS 3. AS 3, however, leaks this prefix to AS 1.  AS 1 can now use AS 3 to get to AS 4.

Type 5: Prefix re-origination with data path to legitimate origin

This type is similar to type 1:

A multihomed AS learns a route from upstream AS 1 and advertises it to upstream AS 2. In this scenario, however, the route is advertised as if the AS originated the route. This means the AS path is removed. This is also known as re-origination.

Type 6: Accidental Leak of Internal Prefixes and More-Specific Prefixes

This route leak happens when an AS leaks internal prefixes to one or more transit provider ASes and/or ISP peers. The leaked route is more specific than an already advertised summary route. The specific route should not have been advertised, and the receiving AS failed to filter it. Here’s an example:

AS 2 advertises a summary route for the routes from AS 4 and AS 5. It also advertises a specific route to its peer, AS 3. Specific routes are preferred over summary routes.

Route Leak Consequences

The AS that performs the route leak attracts traffic to itself. There are several issues with route leaking:

• Violation of policies
• Traffic forwarding through unintended paths
  • Suboptimal routing
  • Increased latency
• Traffic blackholes
• Security issues:
  • Packet sniffing
  • Spoofing

ASes agree on what routes they advertise, so leaking routes violates policies. Because of a route leak, traffic is forwarded through unintended paths. This can cause suboptimal routing and additional delays. The leaking AS might not have enough capacity to forward the traffic.

If the leaking AS advertises something they don’t own, they may drop the traffic, causing black holes and denial of service (DoS).

There are also security issues. The leaking AS will receive traffic, which they can inspect. They can also set up servers using IP addresses that belong to the leaked route. For example, you can spoof DNS servers, making users resolve hostnames to IP addresses you own.

Route Leak Detection

Some companies detect global BGP route leaks by analyzing BGP messages. Two examples are:

• Cloudflare Radar
• Cisco BGPStream

On your local AS, the things you can monitor are:

• Route monitoring to see any changes in received routes.
• Spikes in traffic volume.
• Increase in round-trip time (RTT).
• Number of packets drops.

Route Leak Mitigation

What can we do to prevent route leaking? You need to configure your routers to have the correct outbound and inbound filters and not advertise anything you aren’t supposed to. There is no check between BGP neighbors to see whether your configuration matches the configuration on the other end. There is no enforcement of the relationship between the two neighbors.

There are some things you can do, however. I’ll give you a short overview.

Prefix List Filtering

You can configure prefix lists with ge or le operators to filter prefixes with a certain length. This prevents you from advertising or accepting specific prefixes.

Communities

Communities are also used. For example, no export tells a neighbor AS not to propagate a prefix and no advertise tells the remote router not to advertise it anywhere. ASes can use their own communities as well. The important thing with communities is that this is something you have to do both inbound and outbound.

AS-Set

An AS-Set is used by ASes to group AS numbers they provide transit for. This can be used to define the filters you need to configure. Don’t confuse this with the aggregate as-set.

BGP Roles and OTC

RFC 9234 describes a solution where eBGP neighbors must agree on a BGP role and peering relationship when they establish a neighbor adjacency. This is implemented with a new BGP role capability in the OPEN message. BGP roles can be:

• Provider
• Customer
• Peer
• Route Server
• Route Server Client

The following role pairs are permitted:

• Provider – Customer
• Customer – Provider
• Route Server – Route Server Client
• Route Server Client – Route Server
• Peer – Peer

When there is a mismatch, the BGP neighbor adjacency fails. Once two BGP routers agree on their BGP roles and the neighbor adjacency is established, they have to figure out whether to propagate a route or not. This is done with a new optional transitive BGP path attribute called only-to-customer (OTC).

When a router advertises a route, it sets the OTC attribute with the value of its own AS. When it is not set, the router on the receiving AS sets the OTC with the value of the originating AS. It doesn’t matter who sets it as long as the originating AS value is added.

Other routers in ASes use the OTC attribute and compare it with their peering relationship to decide whether to propagate a router to other ASes or not.

Resource Public Key Infrastructure (RPKI)

BGP assumes that the routes we exchange are genuine and trustworthy. That’s not the case on the Internet, though. Everyone can advertise whatever routes they want. RPKI is a digital certificate system where we associate BGP route announcements with the correct originating AS. This allows ASes to verify that the advertising AS is the legitimate owner of the routes they advertise. RPKI is defined in RFC 6480. We have an RPKI lesson and an example of implementing it on Cisco IOS XE: BGP Prefix Origin AS Validation with RPKI.

AS Provider Authorization (ASPA)

ASPA is a security mechanism that helps to validate the relationship between an AS and its upstream ASes. An ASPA record states which ASes are allowed to propagate routes.

Conclusion

You have now learned what BGP route leaking is, how it happens, the different types, and how to prevent it from happening. To learn more about BGP route leaks, look at RFC 7908. This RFC also has some real-life examples of global route leaks. There have been many global route leaks. Here are four major global BGP route leaks that are interesting to dive into:

• Vodafone India BGP Leak (2021): This leak occurred in Vodafone’s autonomous network (AS55410) based in India and significantly impacted U.S. companies, including Google. The network mistakenly advertised over 30,000 BGP prefixes or routes, leading to a 13 times spike in inbound traffic and effectively causing a self-inflicted DDoS attack. This incident affected over 20,000 prefixes from global autonomous networks and lasted for about 10 minutes​​.
• SafeHost BGP Leak (2019): In this incident, SafeHost (AS21217) announced over forty-thousand IPv4 routes learned from other peers and providers to its provider China Telecom (AS 4134). China Telecom then propagated these routes globally, affecting a vast number of networks. The leak was so extensive that almost every Full Routing Table (FRT) peer announced at least one leaked route to a route collector​​.
• Nigerian ISP BGP Leak: A Nigerian ISP, while peering with Google, accidentally leaked its route to its provider AS4809, creating a type 4 route leak. This overwhelmed the small ISP and led to Google services being down for over an hour​​.
• Cloudflare and AWS Route Leak (2019): This leak involved networks such as Cloudflare and AWS. It originated when ATI propagated thousands of routes received from its provider DQE communications to another provider, Verizon. This provider-to-provider leak significantly impacted global internet traffic, involving more than 4,000 different origins and over 65,000 subnets. Verizon was notably affected, with the leak lasting almost two hours​​.

If you want to see BGP route leaks in action, you can check out Cisco’s BGP Stream. It’s a free resource that analyses BGP messages to detect BGP hijacks and route leaks.

I hope you enjoyed this lesson. If you have any questions, feel free to leave a comment!