What is the internet?
The internet is a publicly accessible global information system made up of interconnected computer networks, through which people can reach information anywhere in the world quickly and conveniently. It works by transferring data between millions of computers belonging to governments, companies, individuals and organizations worldwide. Technically, the internet is made up of many smaller networks called autonomous systems (AS). An AS is operated by a single organization and holds one or more IP address ranges. These networks may belong to government agencies, internet service providers, financial institutions, universities and so on. The Border Gateway Protocol (BGP) is an inter-domain routing protocol designed to connect autonomous systems and determine the best path for packets crossing between them.
What is BGP?
BGP is the protocol designed to exchange routing information between autonomous systems on the internet. It scales remarkably well, supporting millions of IP prefixes and millions of paths in the global internet routing table. BGP comes in two forms: internal BGP (iBGP), deployed to exchange routing information between devices inside the same AS without sharing it with other autonomous systems; and external BGP (eBGP), deployed to connect different autonomous systems, allowing an AS to use multi-homing and thereby gain flexibility and redundancy. The current version of BGP is version 4 (BGP-4), standardized in RFC 4271 in 2006. BGP supports routing policy, which allows network traffic to be managed and controlled through path attributes.
BGP defines four attribute groups (attributes):
- Well-known mandatory: every BGP-speaking router must recognize and process this attribute. Attributes of this type are always sent with the route in an update.
- Well-known discretionary: a router running BGP must recognize and be able to process this attribute. Attributes of this type may or may not be sent alongside a route in an update.
- Optional nontransitive: a router running BGP may not need to process this attribute, and does not send it on to peers when advertising the route.
- Optional transitive: a BGP-speaking router need not process this attribute, but must pass it on if the route it received carries it.
The role of BGP in the Internet
The BGP protocol runs the Internet by routing data. BGP routing matters because the Internet is, at its core, made up of hundreds of thousands of ASNs. Each ASN is a small network allocated and managed by IANA and regional or national registries. Data moves between ASNs from source to destination. BGP supports ASNs in the following ways:
- Find the best route
- Discover network connectivity changes
- Manage routing policies
How does BGP work?
BGP belongs to the path vector family of protocols. The most important routing information it exchanges is IP address prefixes and the paths that must be traversed to reach them. This information is known as Network Layer Reachability Information (NLRI). For two routers to exchange NLRI they must first peer with each other by establishing a session over TCP. First, one of the two routers (the client) initiates the connection by sending a TCP SYN packet to the other router (the server) using a random source port and destination port 179. The server receives the TCP SYN and replies with a SYN-ACK from source port 179 to the destination port the client chose in the first step. Finally, on receiving the SYN-ACK, the client replies to the server with an ACK.
Figure 1 – Establishing a BGP session
Establishing a BGP session involves exchanging BGP control messages: OPEN, KEEPALIVE, NOTIFICATION and UPDATE. These messages are sent and received in the last three BGP states: OpenSent, OpenConfirm and Established. In OpenSent, the BGP router has sent an OPEN message to its peer and is waiting for an OPEN message in reply. When that arrives, the session moves to OpenConfirm, where the BGP router compares the attributes in the peer's OPEN message with its own. If the terms are acceptable, the BGP router replies with a KEEPALIVE message and the session moves to Established. UPDATE messages are then sent to exchange NLRI, and the session is maintained by sending 19-byte KEEPALIVE messages every 60 seconds (by default) to confirm the peer is still up. NOTIFICATION messages are sent in all three states to signal an error condition. For example, if an OPEN message contains an unacceptable parameter, a NOTIFICATION message is sent describing the exact nature of the error, and in most cases the session returns to Idle. For BGP session establishment, the OPEN message matters most. As noted, in OpenConfirm each BGP router sends an OPEN message containing its local attributes in order to build the BGP relationship. The OPEN message carries six main parameters: the BGP version in use, the autonomous system number (ASN), the router's hold timer, the router's BGP identifier, and the optional parameters the router supports.
Figure 2 - how a router processes routing
Once a BGP session is established, the router receives routing information from its BGP peers into the inbound routing table (Adj-RIB-In). That information passes through the inbound routing policy filter into the BGP routing table; the best routes are then installed in the device's local routing table and, after passing through the outbound routing policy filter, placed in the outbound routing table (Adj-RIB-Out) and advertised to BGP peers.
BGP routing security issues and real-world examples
As we have seen from how BGP works, BGP version 4 (RFC 4271) addresses connectivity and scalability but does not account for security: BGP was built on trust, with no security integrated into its design and no validation of IP addresses or AS numbers. The security issues fall into two categories: those relating to the BGP protocol itself, and those relating to the information BGP carries.
First, BGP has no built-in mechanism to protect the integrity, authenticity or confidentiality of BGP messages, which means attacks can be mounted by exploiting either TCP or BGP messages. As noted above, BGP runs over TCP port 179, so it is exposed to DoS/DDoS attacks: an attacker can flood a router with TCP SYN messages, or use TCP RESET or spoofed IP packets to tear down the BGP session between BGP speakers. This is also a weakness of the protocol — there is no authentication mechanism governing BGP session resets, so an attacker can send OPEN messages to reset sessions in the Active state, send out-of-sync KEEPALIVE messages, or send a NOTIFICATION message to tear down a session as a form of DoS/DDoS. An attacker can also send spoofed packets inside UPDATE or Route Refresh messages to BGP speakers, carrying incorrect routing information or overloading their processing capacity. Second, BGP has no mechanism to verify that an AS is authorized to advertise a given IP address range, nor any way to verify the authenticity of the attributes in a BGP update message. As a result, any AS can advertise any IP address, and any AS can insert any ASN into the AS path. This is a common problem in global internet routing, known as BGP hijacking and BGP route leaks.
BGP hijacking falls into two categories: blackhole attacks and interception attacks. The difference lies in what happens to the affected traffic. In a blackhole attack, traffic destined for the targeted service is drawn toward the advertising source and dropped. In an interception attack, packets are not discarded but forwarded on to their original destination, with the attacker sitting in the middle (man in the middle) and able to read and alter their contents. A well-known example of a blackhole attack is the YouTube outage caused in Pakistan — also a textbook case of how even accidental misconfiguration can cause widespread damage. In 2008, the Pakistani government planned to block YouTube nationwide and instructed Pakistan Telecom to implement the block. Pakistan Telecom advertised a more specific IP prefix than the one YouTube was advertising, and in doing so drew most of the traffic destined for YouTube from Pakistan into a black hole on its own network.
Figure 3: Illustration of Pakistan Telecom's blackhole attack on YouTube. Source: NANOG
Pakistan Telecom advertised YouTube's 208.65.153.0/24 range with origin AS 17557. As noted above, because routers prefer the longest matching prefix, the /24 was selected over the /22. Traffic destined for YouTube was therefore redirected into a black hole on Pakistan Telecom's network. The impact should have been confined to Pakistan, but a misconfigured session between Pakistan Telecom and PCCW — a tier-1 upstream — caused 208.65.153.0/24 to be advertised widely across the global internet. The incident took YouTube offline for roughly two hours and was resolved only when YouTube re-advertised an equally specific IP range and PCCW blocked the incorrect advertisement from Pakistan Telecom. Another example of BGP hijacking, this time an interception attack, is the April 2018 attack on AWS DNS infrastructure to steal cryptocurrency. The attacker hijacked the Amazon IP addresses hosting DNS resolution for MyEtherWallet.com. As a result, many users were redirected to a fake website hosted in Russia and had USD 150,000 worth of Ethereum stolen.
A BGP route leak occurs when BGP route advertisements go beyond the scope of the AS concerned — in other words, when advertisements do not match routing policy. RFC 7908 describes the problem. Causes include human error, misconfiguration and route filtering errors between BGP peers, all of which lead to incorrect or suboptimal routing. The scope of a route is normally defined by inter-domain routing policy based on the relationships between autonomous systems. Those relationships fall into three categories: customer, transit provider and peer.
Figure 4: Illustration of routing relationships. Source: Cloudflare
In a customer–provider relationship, the customer's AS has an agreement with the provider's network to forward its traffic to the global Internet. In a peering relationship, two ASes agree to exchange traffic bilaterally between the two networks and, in some cases, between their respective customer networks as well. RFC 7908 defines the following types of BGP route leak:
– Type 1: "provider – customer – provider" is, in simple terms, where a multihomed AS learns a prefix from one provider and re-advertises it to another provider. This is the best-known type of route leak and its impact is severe. In many cases, a customer route is preferred over a peer route or a provider route
Figure 5: Type 1 route leak
In the illustration above, ISP-01 may prefer traffic to ISP-02 via the Customer rather than over its peering or other provider links, unintentionally turning the Customer into a transit network. This can significantly degrade performance for traffic associated with the leaked prefixes, or cause an outage when the traffic volume exceeds what the Customer's network can handle. As a real-world example, on 1 August 2022 Akamai — a US content delivery and cloud computing provider — leaked 660 routes belonging to 203 autonomous systems (ASNs) across 58 countries, in an incident that lasted 3 hours and 9 minutes.
– Type 2: "provider – provider – provider" is the case where a provider leaks routes between BGP peers to another provider, unintentionally turning a "peer" relationship into a "customer" relationship. This rarely causes congestion, but it can increase network latency and degrade the service quality of the other operator.
Figure 6: Illustration of a type 2 route leak
– Type 3: "Provider–Peer" is where a service provider, after receiving a prefix from an upstream direction, re-advertises it to a partner in a peering relationship, causing transit traffic to be rerouted through the peering link. This case has an impact similar to the case above.
Figure 7: Type 3 route leak
– Type 4: “Peer–Provider” is the reverse of Type 3: after receiving a prefix from a peering direction, the service provider re-advertises it to its upstream, causing return traffic from the Internet to route back through the peering link. This can increase network latency and affect the carrier's service quality.
Figure 8: Type 4 route leak
Some approaches to reducing routing risk
Securing BGP starts with protecting the BGP routers (BGP speakers) and the BGP sessions themselves. Taking control of a router is difficult when its management IP address sits in a hardened network segment behind firewalls. eBGP sessions between autonomous systems should be established over directly connected links, with access control policies applied on the routers. This limits both the interception of control packets within the same broadcast domain and spoofed packets arriving from unwanted peers. BGP authentication is a further measure used to secure the exchange of routing information between BGP routers, typically between autonomous systems (AS). It ensures that BGP updates exchanged between routers come from a trusted source and are not altered in transit. BGP authentication is usually implemented with a simple mechanism: a shared password, known as the BGP authentication key, configured on both peer routers. When routers establish a BGP session they exchange this password, and subsequent BGP messages are authenticated using the agreed key. Routers using BGP authentication hold a hash of the shared password. The receiving router computes a hash from the message it received and its own copy of the shared password. If the computed hash matches the value carried in the message, the message is considered authenticated. If a router receives a BGP message with an incorrect authentication hash, or with authentication disabled, it can discard the message or take whatever action the administrator has configured.
Next, validating the routing information in BGP advertisements requires routing information databases, together with additional cryptographic measures in BGP, so that appropriate route filtering policies can be applied to reduce BGP hijacking and BGP route leaks. The IRR is a globally distributed routing information database. Established in 1995, its purpose is to keep routing across the internet stable and consistent by sharing information between network operators. The IRR is in fact a set of databases in which network operators publish their routing policies and route announcements so that other operators can use the data. The IRR allows internet operators to:
- Debug routing issues
- Automatically configures backbone routers
- Carry out network planning
An IRR holds routes and routing policies published in a common format that network operators can use to configure their backbone routers. This supports network management in several ways:
- Route filtering: traffic can be filtered on the basis of registered routes, preventing network incidents caused by accidental or malicious route announcements. Route filtering can be applied between: peer networks, where the peers agree to filter routing solely on registered routes — if a route is not registered it is discarded; and ISP and customer networks, where the ISP protects its own network from a customer's accidental announcements, with the customer required to register its routes with the ISP first.
- Network troubleshooting: makes it easier to identify routing incidents outside your own network. Use the directory entry of the ASN associated with the problematic route to resolve traffic routing issues.
- Router configuration: tools such as IRRToolset can generate router configurations, check ASN objects and the routes they register, and run RPSL syntax checks against the ASN registered in the IRR.
The main weakness of the IRR is that it is not centrally governed and is fragmented across many organizations — APNIC, RIPE NCC, LACNIC, AFRINIC, RADB, Level3, NTT and others — so over time older records go un-updated and information becomes inaccurate. As a result, among all the published routing records it is very hard to tell which data is legitimate and verified and which is not. To address the problems with the IRR, a new approach was introduced and standardized in RFC 6486 under the name “RPKI”. RPKI encodes internet resources — IP addresses and autonomous system numbers (ASNs) — into a cryptographic form based on PKI (X.509) certificate standards, so they can be used to validate routing information on the internet. RPKI consists of two main processes:
- Creating the record data: a Route Origin Authorization (ROA) is a cryptographically signed record indicating which Autonomous System is authorized to hold a given IP address range. A ROA record provides three information fields:
+ Prefix: the IP address range used by an autonomous network.
+ Origin ASN: the autonomous system number advertising that IP range.
+ Max length: the largest IP prefix that may be advertised
- Validating the origin of routing information: Route Origin Validation (ROV) is the process of checking whether network routing information is valid against the data held in ROA records).
RPKI plays several important roles in strengthening the safety and trustworthiness of internet infrastructure:
- Lower network security risk: by preventing BGP hijacking, RPKI reduces security risk and improves the stability of Internet infrastructure.
- Strengthening routing trust: RPKI provides a mechanism for validating network routing, allowing Internet service providers and other organizations to determine whether the routing information they receive is legitimate.
Figure 9: How RPKI works
RPKI works by cryptographically binding internet resource information (IP addresses, ASNs) to the resource holder using PKI (X.509) certificates — in other words, by creating ROA records. This establishes ownership and other status information relating to a specific internet resource. With an RPKI certificate, an internet number resource is associated with a digital signature. RPKI operates as a hierarchy of mutually signed certificates: a 'root' certificate signs child certificates, and those child certificates sign their own children. Regional Internet Registries (RIRs) sign certificates for their direct members and for National Internet Registries (NIRs). ROA records and their digital signatures are then stored and distributed across the internet through the RIRs' repositories.
Figure 10: Delegation in RPKI
ROV (Route Origin Validation) works by checking the validity of network routing information against the ROA records held in the RPKI system. The specific steps and operating principles of ROV are as follows:
- ROA collection: first, the ROV system must collect ROA records from the RPKI database. These ROA records contain information on authorized IP addresses and their corresponding digital signatures, retrieved via the RSYNC protocol (TCP/873) or RRDP, using a uniform resource identifier (URI) as defined by RFC 5781.
- Digital signature verification: the ROV system uses digital certificates from Certificate Authorities (CAs) to verify the validity of the digital signature attached to ROA records. If the digital signature verifies successfully, the information in the ROA record is treated as valid.
- Send digitally signed Internet resource information to routers over the RTR protocol (TCP/8282)
- Route validity check: when a routing device receives a route from a peer (via the BGP protocol), it checks it against digitally signed Internet resource data to determine the route's status, expressed as: Valid, Invalid, or Unknown (no data found).
However, this approach is easily circumvented when an attacker uses AS path prepending during route advertisement — a technique originally intended to steer traffic by changing the AS path length in an advertisement. Solving this requires path validation, which BGPsec — an extension to the BGP protocol — addresses. In BGPsec, the AS_PATH attribute is replaced by a BGPsec_PATH attribute that carries additional information in the form of signatures. Each signature corresponds to one AS number in the path, so the longer the path, the more signatures it contains. The router advertising the IP prefix generates a signature over the advertised prefix, the advertising AS number and the AS number the UPDATE message is forwarded to (the AS number of the BGP peering session). A router receiving an UPDATE message reuses the preceding signature rather than creating a new one, and each signature is accompanied by a Subject Key Identifier (SKI).
Figure 11: How BGPsec works
The figure above shows a routing process using BGPsec, in which AS1 advertises the IP prefix 203.119.0.0/24 and its router generates a signature corresponding to its own AS. When AS2 receives the UPDATE message sent by AS1, it validates every signature in the signature block, then appends its own AS number to the BGPsec path along with a new signature in the signature block. It then forwards the UPDATE message to AS3. The router at AS3 validates the information again. Because the signature block now contains two signatures, the router at AS3 must validate both. It starts with the most recently added signature — sig2 in this example. To validate it, the router at AS3 looks up the certificate matching SKI2 and AS2. If a matching certificate is found, it uses the public key to verify the signature. If verification fails, or no certificate is found, the UPDATE message is treated as invalid. The router then validates the next signature in the same way. Based on the validated signatures, the router receiving the UPDATE message can be confident that every AS in the path has routers that generated a signature. It can also confirm that the next hop in each signature corresponds to the next AS number in the path. This is how the route advertising the IP prefix is authenticated. Fundamentally, BGPsec is only effective if every AS adopts it, and every AS must perform cryptographic operations — so at the scale of today's global routing table (1.2 million IPv4 prefixes and 270,000 IPv6 prefixes), device capacity and capital investment become real concerns. This is a significant obstacle to wide adoption, although many argue that BGPsec can still deliver value even if only the top-tier upstreams agree to use the protocol between themselves.
A more practical alternative is Autonomous System Provider Authorization (ASPA). ASPA validates which provider AS numbers are permitted to advertise and forward routes. Unlike the earlier mechanism, which validates the origin of a route, ASPA validates the AS_PATH and allows other parties to reject a route when the AS_PATH does not appear in the record. ASPA records are created and distributed in the same way as ROAs in RPKI. With ASPA in place, an AS can be confident that no AS on the path is advertising the IP prefix without authorization. The example below shows an ASPA record created by AS945, listing five providers permitted to advertise its IP prefixes: AS1299, AS6939, AS32097, AS50058 and AS61138.
Figure 12: Illustration of an ASPA record. Source: MANRS
Summary
Securing our networks therefore means protecting routing information on the internet. Understanding how the BGP routing protocol works helps us anticipate the risks the internet poses to our own networks. RPKI, BGPsec and ASPA are proposed extensions to BGP that add path validation and help prevent certain attacks and configuration errors. That security typically comes at the cost of additional equipment capacity, since generating and validating signatures takes more time. BGP is also an old protocol, and carries the same class of problems as its contemporaries, such as DNS. Finally, these security measures still need time for research, optimization and standardization in RFCs, as well as global adoption, before they can deliver their full effect.
Phan Xuân Dũng – Nguyễn Xuân Trường
References
Routing Security: BGP and AS Path Validation, Geoff Huston (APNIC)
Best Practices for Enhancing BGP Security, Networkphil
BGPsec in the context of routing system security, CSNOG
