For 12 Hours, Was Part of Apple Engineering’s Network Hijacked by Russia’s Rostelecom?

For a little over 12 hours on 26-27 July, a network operated by Russia’s Rostelecom started announcing routes for part of Apple’s network. The effect was that Internet users in parts of the Internet trying to connect to Apple’s services may have been redirected to the Rostelecom network. Apple Engineering appears to have been successful in reducing the impact, and eventually Rostelecom stopped sending the false route announcements. This event demonstrated, though, how Apple could further protect its networks by using Route Origin Authorizations (ROAs). 

We are not aware of any information yet from Apple that indicates what, if any, Apple services were affected. We also have not seen any information from Rostelecom about whether this was a configuration mistake or a deliberate action. 

Let’s dig into what we know so far about what happened, and how Route Origin Authorization (ROA) can help prevent these kinds of events. 

Around 21:25 UTC On 26 July 2022, Rostelecom’s AS12389 network started announcing 17.70.96.0/19. This prefix is part of Apple’s 17.0.0.0/8 block; usually, Apple only announces the larger 17.0.0.0/9 block and not this shorter prefix length. 

When the routes a network is announcing are not covered by valid Route Origin Authorization (ROA), the only option during a route hijack is to announce more specific routes. This is exactly what Apple Engineering did today; upon learning about the hijack, it started announcing 17.70.96.0/21 to direct traffic toward AS714.

It is not clear what AS12389 was doing, as it announced the same prefix at the same time with AS prepend as well. 

In the absence of any credible data to filter out any possible hijack attempts, the route announced by AS12389 was propagated across the globe. The incident was picked up by BGPstream.com (Cisco Works) and GRIP Internet Intel (GA Tech). 

Our route collectors in Sydney and Singapore also picked up these routes originated from AS12389: 

BGP4MP_ET|07/26/22 21:25:10.065207|A|169.254.169.254|64515|17.70.96.0/19|64515 65534 20473 3257 1273 12389 12389 12389 12389|IGP 

BGP4MP_ET|07/26/22 21:25:11.211901|A|169.254.169.254|64515|17.70.96.0/19|64515 65534 20473 17819 7474 7473 12389|IGP 

BGP4MP_ET|07/26/22 21:25:12.022767|A|169.254.169.254|64515|17.70.96.0/19|64515 65534 20473 17819 4826 12389|IGP 

BGP4MP_ET|07/26/22 21:29:06.885842|A|169.254.169.254|64515|17.70.96.0/19|64515 65534 20473 3491 1273 12389|IGP 

Apple must have received the alert too. Whatever mitigation techniques they tried didn’t stop the Rostelecom announcement and so Apple announced the more specific route. As per the BGP path selection process, the longest-matching route is preferred first. Prefix length supersedes all other route attributes. Apple started announcing 17.70.96.0/21 to direct traffic toward AS714. 

The first announcement for 17.70.96.0/21 appeared around 02:41 UTC on 27 July – 5 hours and 16 minutes after the first appearance of 17.70.96.0/19 from AS12389. 

In the meantime, the possible hijacked route continued to be announced by AS12389 for many hours. Finally, around 09:39 UTC on 27 July we started seeing withdraws of the 17.70.96.0/19 route, 12 hours and 14 minutes after it began. 

It is not clear which services were impacted by this incident. Unless we get more details from Apple or other researchers, we can only guess. 

It is extremely important for network operators to implement effective route filtering based on verifiable information about which networks are legitimately authorized to originate which number resources (AS numbers and IP prefixes).  

This is what RPKI was designed for, having a valid Route/Route6 Object is important, but these days when many major network operators are doing Route Origin Validation (ROV) and dropping any routes with Invalid origins, it is even more important to have a valid Route Object Authorization (ROA) for all your resources. 

Unfortunately, AS12389 (Rostelecom) has been part of multiple incidents in the past, which we have covered in older blog posts in detail. 

This can happen again at any time. Network operators have a responsibility to ensure a globally robust and secure routing infrastructure. Your network’s safety depends on a routing infrastructure that stops bad actors and mitigate accidental misconfigurations that wreak havoc on the Internet. The more network operators work together, the fewer incidents there will be, and the less damage they can do. 

If you are network operator, implement MANRS Actions and to join the MANRS community. If you are already a MANRS participant (as some ASNs are who forwarded these bad announcements) then learn from this and make sure it doesn’t happen again – have valid ROAs for all your resources! You have an obligation to keep the global routing table clean.