Micro BGP Suite – Shell tools for MRT dumps and where to find them

By Lorenzo Cogotti, founder and lead of The DoubleFourteen Code Forge, a non-profit community promoting independent research and good software practices.

As an admirer of the Unix philosophy, I’ve always marvelled at the simplicity of its shell design: basic tools cooperating to achieve complex results.

Most network operators and researchers might share this kind of fascination in their everyday work, but may also feel the lack of such well-behaving tools capable of performing trivial steps to a bigger combined result.

The network community has no shortage of amazing collaborative efforts that provide endless amount of data useful for analysis.

Specifically, Multi-threaded Routing Toolkit (MRT) dumps are a standardized format for compressed files archiving precious information on Border Gateway Protocol (BGP) routing tables at a point in time, and their subsequent evolution, as observed by route collectors.

Several of such route-collecting projects publish their MRT dumps freely, including Route Views at the University of Oregon and the RIPE NCC Routing Information Service (RIS), being a crucial source of data to assess network health and investigate routing security.

Whenever you run into an Internet reachability issue, BGP looking glasses are the usual place you go to start troubleshooting. Since we cannot travel back in time to identify the problem, the use of historical data and the subsequent flow of events, such as BGP updates, can help inform whether an incident is caused by a mis-origination or a missing route. Thanks to projects such as RouteViews and RIPE RIS you can have access to daily BGP snapshots from which you can extract routing entries to your network.

Despite the abundance of well-understood data, oftentimes only fairly complex pieces of software are available for their analysis, hardly interoperable with familiar tools. Possibly even network services with a strict set of features whose only means of tailoring to one’s needs is a client API.

Thus requiring:

• basic knowledge of scripting in a specific language;
• knowledge of the specific server web API exposed by the service.

In short, a whole dedicated knowledge base for the sake of a single tool. Wouldn’t it be possible to reuse all our hard-earned knowledge in the Unix shell, and our playful hacking skills to conjure scripts quickly, to work on publicly available MRT dumps and produce useful data sets for our purposes the way we want?

Wouldn’t it be nice to bring these useful new tools to our familiar grounds in awk, sed, grep, perl or python?

This is the gap Micro BGP Suite (µbgpsuite for short) attempts to fill.

The µbgpsuite project comes with three objectives:

• performance – good tools should not come with a prohibitive requirement set, but produce interesting results quickly even on modest machines;
• provide powerful analysis and research tools on BGP to make previously challenging tasks plausible, without alienating the user;
• provide an extensive library to developers, good tools should be extensible in face of unanticipated needs.

…and of course, be free and open source, because we like to ponder on the implementation of our tools and customize them to our needs.

I won’t elaborate on the developer library, as it’s targeted mostly at developers with specific needs. Instead, I will concentrate on its tools for operators and analysts:

• bgpgrep – the main star of the show;
• peerindex – an inspection tool for MRT Routing Information Base (RIB) dumps, to print out relevant entries of the Peer Index Table;
• bgpscanner – a backwards compatibility script to Isolario bgpscanner, for users familiar with the old bgpscanner utility by the same author, its purpose is to ease the migration process and learning of its replacement tools.

Let us focus briefly on bgpgrep, by far the most flexible among the tools mentioned.

In a way similar to the excellent bgpdump by RIPE NCC, bgpgrep is a friendly shell utility to format MRT dumps in a human readable way.

It inherits the same output format as bgpscanner, carefully defined to be parsed easily by scripting languages (like perl or python) or other shell utilities, making it a nice Unix citizen that can easily interoperate with the rest of the system.

Other defining traits are its performance (about 20% faster than the already outstanding bgpscanner), and its extremely flexible BGP filtering and matching capabilities.

For simple uses, bgpgrep can directly decode and format a number of MRT dumps efficiently, in such a way its output can be conveniently fed to more utilities.

In more advanced uses, bgpgrep can extrapolate only data of interest from a vast set of dumps, by applying rules defined with an intuitive syntax.

Extraneous data is discarded from output with minimal processing, making the use of rules a convenient time-saver, often making the difference between a few hours and a few minutes when several gigabytes of input are involved. Data can even be piped directly to bgpgrep, there is no actual need for it to come in an MRT dump file.

Examples

• Print any BGP message with supernets of 8.8.8.8 originated after July 10th:

$ bgpgrep ./rib.20210701.0000.gz ./updates.*.gz -supernet 8.8.8.8/32 -timestamp ">=2021-07-10"

• Print any BGP message whose AS_PATH contains loops or bogon ASN:

$ bgpgrep ./rib.20210701.0000.gz -loops -or -bogon-asn

• Print messages with subnets of 2001:67c:1b08::/48 not originated by AS2598 (trivial mis-origin/hijack detection):

$ bgpgrep ./rib.20210701.0000.gz ./updates.*.gz -subnet 2001:67c:1b08::/48 -not -aspath "2598$"

• Print messages whose AS_PATH crosses link AS174 AS3356 AS2914, with possible prepending inbetween (useful for leak detection):

$ bgpgrep ./rib.20210701.0000.gz ./updates.*.gz -aspath "174 3356+ 2914"

Conclusion

If you are looking for some nice tools to inspect MRT dumps, integrating well with other familiar utilities the way you want, be sure to check out the Micro BGP Suite at DoubleFourteen Code Forge and have a look its sources at the official DoubleFourteen Code Forge Git Repository.

Hoping this post can be at least as useful to you as it was fun for me to write.