Large European Routing Leak Sends Traffic Through China Telecom

Beginning at 09:43 UTC today (6 June 2019), Swiss data center colocation company Safe Host (AS21217) leaked over 70,000 routes to China Telecom (AS4134) in Frankfurt, Germany.  China Telecom then announced these routes on to the global internet redirecting large amounts of internet traffic destined for some of the largest European mobile networks through China Telecom’s network.   Some of the most impacted European networks included Swisscom (AS3303) of Switzerland, KPN (AS1136) of The Netherlands, and Bouygues Telecom (AS5410) and Numericable-SFR (AS21502) of France.

Often routing incidents like this only last for a few minutes, but in this case many of the leaked routes in this incident were in circulation for over two hours.  In addition, numerous leaked routes were more-specifics of routed prefixes, suggesting the use of route optimizers or similar technology.

At 09:57 UTC, over 1,300 Dutch prefixes were announced in this leak.  For 470 routes of KPN (AS1136), the leak took the form:

… 4134 21217 21217 21217 21217 21217 21217 13237 1136          

If someone thought the prepending of AS21217 would keep these routes from leaking out, they were mistaken.  Below is a visualization of the propagation profile of a prefix (46.145.0.0/16) announced by Dutch carrier KPN.

Beginning at 09:44 UTC, over 200 Swiss prefixes were announced by this leak.  For 64 routes of Swisscom (AS3303), the leak took the form:

… 4134 21217 21217 21217 21217 21217 21217 6830 3303

Below is a visualization of the propagation profile of a prefix (46.14.128.0/17) announced by Swisscom.

Each of these plots depicts peaks and valleys in the spread of propagation over time.  These “features” are caused by changes in how China Telecom exported these routes to Tier-1 telecoms during the leak. Below we can see the preceding graphics side-by-side with equivalent views from the upstream perspective of AS4134.  

Beginning at 10:10 UTC, 150 Bouygues Telecom (AS5410) prefixes were in this leak – 127 of which were more-specifics of existing routes.  For example, 176.171.75.0/24 is not normally in the global routing table and is a more-specific of 176.128.0.0/10, announced by Bouygues Telecom (AS5410). These leaked routes took the form below:

… 4134 21217 21217 21217 21217 21217 21217 25091 5410

Users of these services began noticing problems and reporting seeing traceroutes to their networks traversing China:

Bouygues Telecom:

KPN:

Swisscom:

Many of our traceroute measurements were also sucked into this routing leak. The traceroute below begins in Google in Ashburn, Virginia and is destined for Vienna, Austria, but is detoured through China Telecom (hops 5-8) en-route to its destination.

Below is another traceroute from an Oracle datacenter is Toronto to Numericable-SFR in France that gets diverted through China Telecom (hops 8-10).

Conclusion

Today’s incident shows that the internet has not yet eradicated the problem of BGP route leaks. It also reveals that China Telecom, a major international carrier, has still implemented neither the basic routing safeguards necessary to both prevent propagation of routing leaks nor the processes and procedures necessary to detect and remediate them in a timely manner when they inevitably occur.  Two hours is a long time for a routing leak of this magnitude to stay in circulation, degrading global communications.

A great place for any telecom to start improving their routing hygiene is to join the Internet Society’s Mutually Agreed Norms for Routing Security (MANRS) project.

This article was originally published on Oracle’s blog. The views expressed in this article are those of the author alone and not the Internet Society/MANRS.