RPKI Deployment Case Study: NYSERNet

NYSERNet recently implemented peer-facing RPKI Route Origin Validation (ROV) to protect its members’ routing infrastructure. In this post, I want to share the progress we’ve made and the lessons we’ve learned thus far.

Serving the New York Research and Education Community

For those who aren’t aware, NYSERNet operates colocation and data center facilities, provides dark fiber, offers security and education, and holds conferences. However, our core has always been the network.

We connect around 60 member institutions to our statewide 100-gigabit network, which runs throughout New York state on our fiber facilities and extends on the Internet2 optical backbone down to Ashburn, Virginia. There’s a wealth of educational institutions in New York, and we’ve always connected with those who are most heavily research-oriented.

In the last few years, we’ve dramatically increased our ability to serve schools for whom education is the primary driver, as well as healthcare and cultural institutions such as museums. And we connect many of the K-12 schools in the state through their own networks.

As our emphasis has grown from providing high bandwidth for research to encompassing educational and administrative uses of the network, we’ve seen a big change in our campuses’ reliance on our network. It still needs the performance they have come to expect but it has to be solid and secure. A disruption to their NYSERNet connection will affect cloud and content services, as well as research data transfers; it’s amazing how quickly people become concerned about the network when their video calls are interrupted!

Why Routing Security Matters

We know most routing ‘hijacks’ are accidental and not malicious, but they still cause major problems. Anything we can do to make the network more stable and robust is worth looking at, and the effort to implement RPKI is small in comparison to the benefits of making the infrastructure resistant to routing problems. There’s always the possibility that someone will decide to launch an attack as well, and we owe it to our members to provide them with the best defenses.

Also, as a good net citizen and member of MANRS, we should not allow routes to leave NYSERNet if they’re incorrect. So we are doing our part to keep the global network clean.

Splitting our RPKI Deployment Into Two Activities 

Our first task was to enable RPKI route signing for all the NYSERNet IP space. As part of this, we talked with our members about the process, pointed out hurdles they might face, and provided a realistic idea of what they’ll be required to do.

We know they will need support in the long term, as well, since this isn’t something most engineers will do every day. We will be there to help them remember how it works in a year or two when changes are required.

Our second task was to validate routes on our backbone, which we did during the holiday break in 2022. We don’t expect many of our campuses to do their own validation since their providers (including NYSERNet) will do it for them. But we’re documenting the process as we go—as per MANRS Action #3—and will be happy to consult with any of our members who want to take that step.

So far, we haven’t had too many issues: some extra paperwork to get the RPKI process started with ARIN and a couple of stumbles with the extremely precise syntax required for the signatures.

One thing we discovered is the old lint and dust that gathers in the corners of the network, which is suddenly quite obvious when you are trying to validate routes. There have been a few things we always meant to clean up but never got around to. The new configuration has forced us to get that done, or else we would break our routes.

A lot of lab time was needed to get to the point where we were comfortable putting the configuration into the production network, and so far it’s been fine.

Everyone Can Sign Their Routes Now, and That Will Have a Real Impact

Once you have signed your space, it is much more difficult for someone to accidentally break your routing. Also, it encourages other networks to sign and validate their routes. I expect to see major network providers requiring signed routes from their customers in the next couple of years.

Enabling validation is a more considerable step, in some respects, but has significant rewards as well and should be on everyone’s roadmap if you’re providing transit connections.

The Internet2 community has helped us a great deal, as has the excellent documentation at ARIN and other online resources, including MANRS where you can learn what you can do beyond RPKI to complete your routing security picture.

With the openness that is a hallmark of the research and education community, I hope we can help each other as RPKI adoption spreads.

This is an adaption of an original post that appeared on the Internet2 Blog.

Bill Owens is the chief network architect at NYSERNet.