Seven actions REANNZ takes to secure routing of New Zealand’s Research and Education sectors

REANNZ, New Zealand’s National Research and Education Network (NREN), is proud to be the first network operator in New Zealand to join MANRS. Security best practice is at the forefront of what we do, from the protocols and systems that we implement internally to the partnerships we form externally. In this post, we would like to share our thoughts outlining the top seven efforts that we’ve undertaken to ensure the security and stability of our membership’s routing on a global scale.

Route Hijacking/Leaking

Route hijackers advertise address space belonging to an unsuspecting network in an attempt to draw that network’s traffic towards themselves. This type of attack can be used to impersonate services (banking, communications, etc.) as a man-in-the-middle (MITM) attack or can be used to simply disrupt connectivity to that network as a denial-of-service (DOS) attack.

Although malicious hijacking of traffic is of the utmost concern, as it can end in theft of funds or intellectual property, the majority of incidents tend to be accidental. Configuration typos and poor routing policy logic can lead to advertisement of bad routing information. These mistakes are incredibly easy to make, and if they’re being honest, many network engineers will admit to having made such blunders.

In order for a route hijack to be effective, an attacker must ensure their route is preferred over the real one. This can be as simple as “being closer” to most of the Internet. The number of networks (autonomous systems) on the path back to the originating network is one basic metric that is used to decide between multiple versions of the “same” route. The route with the shortest AS (Autonomous System) path is preferred.

The Internet uses more specific routes as the most basic metric for making routing decisions. This functionality is extremely useful for balancing traffic across multiple handovers or service providers. An attacker can split a route into multiple smaller routes and advertise them from anywhere. They will likely become preferred across most or all of the Internet.

REANNZ has put a number of systems in place to protect the network and our membership from foul play in global routing and to protect our membership and the wider Internet from REANNZ’s human error:

1. Exact route filters on member Internet handovers

REANNZ actively maintains a close working relationship with the operators of our membership networks. This includes collecting an exact list of routes that each member expects to advertise. After being checked for legitimacy, this information is stored in an IP address management (IPAM) system. Route policy is then built from this information to ensure only the expected routes are accepted. This means should a mistake be made by a REANNZ member, it will not be seen by the rest of the world.

2. Limit number of routes that will be accepted across external peering connections

If a peer network advertises a larger number of routes than the configured maximum, then the BGP session to that network will be immediately dropped. This is to protect against such events as the “BGP Optimiser” incident where a network, due to misconfiguration, suddenly begins to advertise a large number of erroneous routes. It is a very coarse protection and should be used with care as it can cause disruption on legitimate network behaviour, such as when Hurricane Electric, one of the largest network providers in the world, arrived on the Auckland peering exchanges adding their 108k routes to the existing 55k.

3. Additional route filters on external peering connections

REANNZ adds route policy to external peering connections to ensure we can only export member routes that have originated from their Internet handovers. This is belts-and-braces protection against propagation of erroneous route information. It also ensures REANNZ cannot propagate routes between external peers which would in effect make REANNZ an accidental transit provider for those networks.

4. Automation of all Internet routing policy

The vast majority of REANNZ Internet routing policy is generated by automation (~37k lines of config). As well as making day-to-day operations simpler and quicker, automation reduces the substantial risk of human-error in our routing policy.

5. Management of Internet Routing Registry (IRR) objects for members

IRRs are a series of databases used to communicate routing intention. This intention can be used by peers, Internet service providers, and Internet exchanges to generate routing policy to ensure propagation of accurate routing information.

As incorrect IRR information can cause parts of the Internet to become inaccessible, REANNZ maintains IRR information on behalf of our members. All IRR objects are generated by automation using data from our IPAM system. We also have alerting in place to notify us when member IRR information is changed by another party.

6. Resource Public Key Infrastructure (RPKI)

RPKI is a technique for cryptographically matching a route to its origin network (AS). When importing routes from peers, REANNZ BGP routers can ensure routes have been originated by the appropriate network before accepting them. RPKI does have some failings and so is not a full solution to the problem of erroneous routing information, but it is a step in the right direction.

For more information on RPKI, REANNZ gave an operational presentation of our deployment at the APRICOT Conference 2020 in Melbourne. You can watch it at https://youtu.be/ZXLPzIgkRbM?t=2486.

IP Address Spoofing

IP address spoofing is a technique where an attacker pretends to be another Internet user by forging their IP address when sending data. It can be used in conjunction with a route hijack to perform a man-in-the-middle attack. However, on its own this technique can be used in denial of service (DoS) attacks.

An attacker can send a request to a service on the Internet with a faked IP address, but the response will be sent back to the legitimate user. This behaviour can be further exploited by sending specially crafted requests to certain types of services (DNS, NTP, etc.) which solicit much larger (potentially >100x) responses. A single attacker sending 10Mb/s of requests, can generate a 1Gb/s of responses. 1,000 concurrent attackers (e.g. a botnet) can generate 1Tb/s of attack traffic against a single target! This is an amplification (or amplified reflection) distributed denial-of-service (DDoS) attack.

REANNZ’s membership occasionally see incidents of IP address spoofing, and there are mitigations we can put into place while they’re occurring to keep the network and our membership functioning, but these are just stop-gaps. The only reliable method for stopping these types of attacks before they can occur is to ensure that packets with faked IP addresses are not forwarded onto the wider Internet in the first place.

7. ACLs on member Internet handovers

As a good citizen of the Internet, REANNZ places stateless access control lists (ACLs) on all of our member Internet handovers to limit IP addresses with which they’re allowed to send traffic. These ACLs are generated by automation (~21k lines of config) using the same IPAM database that is used to generate our route filters. Although, these measures will not protect our members’ networks specifically, it does serve to protect against reputational damage as they are unable to be used by nefarious parties for reflection-type attacks.

REANNZ is early in the process of enabling these ACLS on member handovers. This type of change has the potential to be very disruptive if not done carefully, so close collaboration with our members is of utmost importance. Expect to hear more from us on this in the future.

Conclusion

At REANNZ we take pride in our position as gatekeeper to New Zealand’s research and tertiary education sectors and are committed to protecting the networks and reputations of our member institutions. That’s why we’ve invested significant time implementing and maintaining the capabilities outlined above.

Alone we’re only able to accomplish so much, but as participation of collaborative routing security initiatives, such as MANRS, increases, so does the security and reliability of the Internet as a whole. Therefore, REANNZ urges every citizen of the Internet to do their part.

Aaron Murrihy is Senior Network Engineer at REANNZ.