How to Secure Routing in the IXP Route Servers Infrastructure
Because Internet Exchange Points (IXPs) play key roles in contributing to a more resilient and secure Internet, they should take steps to protect it. This post introduces ways to improve routing security on Route Servers, a key feature of IXPs.
The MANRS IXP Programme has a specific set of actions designed for IXPs. In particular, Action 1 requires IXPs to implement filtering of route announcements at the Route Server, based on routing information data (IRR and/or RPKI). This helps prevent propagation of incorrect routing information.
56 IXPs around the world participate in the MANRS initiative.
What are Route Servers?
Route Servers (RS) are typically used on shared access media networks, such as IXPs, to facilitate simplified interconnection between multiple Internet routers. They are a value-added service offered by IXPs and their popularity has made them critical infrastructure for large IXPs.
The basic function of an RS is to replicate the behavior of a full mesh of eBGP peerings in a centralized way. Each member at an IXP can establish a BGP session with the RS on which it can announce its own prefixes, optionally specifying a selective export policy. The Route Server performs the best path selection for each prefix announced, according to the export policy specified by each IXP member. The RS is also responsible for the propagation of best BGP paths to each IXP member who peers with it.
A RS is never involved in packet forwarding, but only performs control-plane operations. This implies that the Autonomous System (AS) Number of the RS does not appear in the BGP announcements that the RS propagates to the IXP members. Moreover, the next-hop attribute of the BGP announcements propagated by an RS must be identical to that contained in the BGP announcements received by the RS.
Route Servers are currently a critical piece of infrastructure in all the most important IXPs worldwide. This is especially true for those environments where a huge number of peers exists (often more than a hundred) and an RS is a quick shortcut to full operational status for new members.
Moreover, Route Server performs an initial filtering on routes received from its peers, thus guaranteeing greater reliability of announced routes (protection against hijacks, announcement of bogons/martians, and default routes). RS takes care of applying proper input filtering to all the BGP announcements.
Route Server and the Internet Routing Registry system
Routes received from each peer in the RS infrastructure are filtered according to data registered in the Internet Routing Registries. Each member must provide an AS-SET macro containing all ASes that are advertised by it on the IXP.
Support for the automatic generation of import filters is provided: import filtering ensures that prefixes coming from a certain client are put in the routing information base (RIB) only after having been compared with IRR registered policies for that peer. Thus, only routes originated from the client AS or from one of its registered customers are accepted into the RIB.
In order to support input filtering, each client has to specify, along with its own AS number, an AS-SET macro containing all its direct customers that are going to be announced on the exchange. Then a set of filter rules is generated by looking at registered routes for the aforementioned AS-SET (client + customers). Each route is filtered by matching the exact netmask and origin AS, as in this example (OpenBGPD syntax):
# AS24796
allow from 193.201.28.100 inet prefix 193.201.40.0/24 { source-as 24796 }
allow from 193.201.28.100 inet prefix 193.201.28.0/23 { source-as 24796 }
allow from 2001:7f8:10::2:4796 inet6 prefix 2001:7F8:10::/48 { source-as 24796 }
It is of topmost importance that each RS client properly registers its routes into the IRR. For clients that also have a set of customer ASes, it is also important to create a proper AS-SET to contain all customers. This enables the RS to generate its filters from fresh, up-to-date IRR data. Additional filtering is performed in order to block default routes and martians.
Route Servers and RPKI architecture
The Secure Inter Domain Routing (or SIDR) group of the Internet Engineering Task Force (IETF) in 2012 developed a standard architecture, RFC 6481 – A Profile for Resource Certificate Repository Structure based on a public structure (Resource Public Key Infrastructure, or RPKI) with distributed databases (RPKI repositories) containing the associations <prefix; authorised AS>.
Each association is linked to a Digital Certificate that allows anyone consulting the repository to check that this association is correct. Each of these declarations (prefixes, AS number, and digital certificates) is called Route Origin Authorization, or ROA.
RPKI uses the format of X.509 digital certificates with extensions for IP addresses and AS numbers (see RFC 3779 – X.509 Extensions for IP Addresses and AS Identifiers). The certificates do not include identification information since their scope is simply to transfer usage rights to Internet resources. The RIRs have the role of Trust Anchor and are tasked to issue the digital certificates to their members. The main use of these digital certificates is to validate public keys and the legitimacy of an AS to inject a particular block of prefixes into the BGP, and to use a particular AS number.
Routes received from each peer in the RS infrastructure are also filtered according to their RPKI ROA status: currently, prefixes that have an INVALID ROA state are dropped, prefixes with VALID or UNKNOWN ROA state are exported to peers.
It is worth noting that on the network operators side, the handbook “BGP RPKI: Instructions for use” illustrates the main components of the BGP RPKI architecture and how these interact to create a system that will check whether or not an AS is authorized to originate prefixes to the Internet.
Sami Salih
As the RS is very critical to the operation of the IXP, I wondering if its KPIs include Always-On feature. In other words, if the RS is down for a while can the IXP tolerate its absence by return back to less mesh onfiguration.
Flavio Luciani
Dear Sami, thank you for your question.
Typically a RS infrastructure hosts multiple instances, independent platforms for redundancy and diversification (for example at Rome IXP we have two instances and we use a different software for each, Bird / Openbgpd). To provide high redundancy, it is therefore advisable to enable both BGP sessions, in the case that one of the instances goes down, both for issues and for software / hardware maintenance. This guarantees high reliability of the system and increases its availability.
In any case, nobody forbids operators from establishing bilateral sessions and using the RS as a backup element. It is also true that in some cases the RS is the only possible solution for peering with some operators (precisely to contain the number of BGP sessions, some OTTs allow peering to small operators only through the RS). In this case, however, the multiple RS instances guarantee reliability of the service, as mentioned previously.
Hope this answers your question.