My new BGP book: 'Internet Routing with BGP' by Iljitsch van Beijnum BGPexpert My BGP book from 2002: 'BGP' by Iljitsch van Beijnum

Home · BGP Expert Test · What is BGP? · BGP Vendors · Links · Archives · Books · My New BGP Book

BGP (advertisement)

Table of contents (for this page):

inet⁶ consult

If you could use some help with BGP, have a look at my business web site:

BGP routing courses

There are currently no training courses planned.

Interdomain Routing & IPv6 News

  • Search for:
  • Should the datacenter be in the middle? (posted 2023-09-07)

    The other day, I landed on this article: In Focus: Subsea Network Architecture: IXPs. The article takes some time to arrive at the point that undersea internet exchanges would be a good idea. The most eyecatching part is a variation on this image:

    As the article starts out discussing how datacenters have been moving away from large cities to take advantage of opportunities such as space, cheap energy and easier cooling, this image seems to suggest that these blue dots in are good locations for datacenters and/or internet exchanges in general. And that's definitely not the point of the paper that the image is from.

    That paper is very specifically about the best locations to place servers for high speed algorithmic trading on multiple markets some distance away from each other. This immediately explains why there is nothing around the western US: there are simply no stock exchanges / markets there (the red dots in the image).

    The math looks more complicated, but presumably, in these cases it helps when the servers executing the trading algorithms are in the middle between the "users", rather than close to one and further from the other(s).

    If you need data from two places far away from each other, then it's better when each is 25 milliseconds away, as you can then complete your action in 25 ms plus however long it takes to do your own processing. If you're close to one so it's 0 ms for one data source and 50 ms for the other, then the entire action takes at least 50 ms.

    But is that a common situation?

    In general, you can just copy the data beforehand. So this only applies if you're using "live" data from two or more locations. Videoconferencing with a number of participants could be an example, where a server receives the video from all the participants, mixes it into a single feed and then sends that single feed out to all the participants. If the server is in the middle, this limits the maximum delay. I guess that could be somewhat helpful. But to the degree that it makes sense to have datacenters in the middle of the ocean? I'm not convinced.

  • Looking back on 50 years of Ethernet (posted 2023-06-29)

    On Geoff Huston wishes Happy 50th Birthday Ethernet.

    Back in 2011 I wrote an Ars Technica feature about the history of Ethernet: Speed matters: How Ethernet went from 3Mbps to 100Gbps… and beyond.

    Interesting to compare our different takes!

    And of course Ethernet is still going strong. My oldest computers have the original 10 Mbps Ethernet adapters that I got almost 30 years ago, while my newest computer has 10 gigabit Ethernet, 1000 x faster.

    Read the whole article

  • Has BGP routing security failed (yet)? (posted 2022-12-13)

    In the thorough style we've come to expect from him, Geoff Huston tries to answer the question Is Secured Routing a Market Failure? Please read about the market aspect (and the limitations imposed on the IETF by big router vendors) in that article. His final conclusion is broader, through:

    But mostly it's a failure because it does not deliver. Security solutions that offer only a thin veneer of the appearance of improvement while offering little in the way of improved defence against determined attack are perhaps worse than a placebo.

    20 years ago I went to my first IETF meeting and there I learned about S-BGP and soBGP. It would take another almost ten years for RPKI to be standardized and fifteen years for the BGPSEC specification (which is largely S-BGP with the RPKI functionality removed) to be published. RPKI is gaining some traction but hasn't been able to stop at least some determined attackers. There are no production level implementations of BGPSEC after five years.

    So why is that? True, if people would be prepared to pay extra for secure routing, we'd have it by now.

    But I think the bigger issue is failure to understand the problem.

    Looking at the valley-free model, we can observe that the right hand part of the network hierarchy is trusted by the sender of the routing updates / receiver of the packets, while the left hand side is trusted by the receiver of the routing updates / sender of the packets.

    This hierarchy is pretty flat, so there's usually only a handful of networks on each side. So telling the world that those are networks you trust should be something we can do reasonably efficiently. Then, the only thing is to make sure that there are no network hops in the path that aren't trusted by either side.

    Earlier this year, RFC 9234 "Route Leak Prevention and Detection Using Roles in UPDATE and OPEN Messages" was published. This adds some information to BGP that lets it detect paths that aren't valley-free. The nice thing about this mechanism is that it protects two networks that implement the mechanism against mistakes made by networks that don't implement the mechanism.

    We still need better mechanisms to thwart determined, purposeful attackers, but the good news is that we've made life a lot harder for those already. So even though we're not there yet when it comes to secured routing, it's not so much that we've failed, but that we are taking too much time to succeed.

  • → What can be learned from BGP hijacks targeting cryptocurrency services? (posted 2022-11-24)

    Interesting blog post on the APNIC blog by Doug Madory:

    On 17 August 2022, an attacker was able to steal approximately USD 235,000 in cryptocurrency by employing a BGP hijack against the Celer Bridge, a service that allows users to convert between cryptocurrencies.

    In this blog post, I discuss this and previous infrastructure attacks against cryptocurrency services. While these episodes revolve around the theft of cryptocurrency, the underlying attacks hold lessons for securing the BGP routing of any organization that conducts business on the Internet.

    Using BGP to steal cryptocurrency is happening with some regularity now...

    The important lesson comes at the end: Amazon shouldn't have RPKI ROAs for a /10 and a /11 with a maximum prefix limit of /24.

    This way, the attacker, thanks to an ISP that didn't properly filter its customer's BGP announcements, was able to advertise a /24 out of Amazon's address space and have that announcement be labeled "valid" by RPKI route origin validation.

    Amazon advertises a /11, and if the maximum prefix length in the ROA for that /11 had been just /11, the attacker wouldn't have been able to "shoplift" just that /24, but they'd have to go head-to-head against Amazon for that entire /11. That would have had a much lower chance of success and much higher chance of being noticed quickly.

    (Shameless plug: if all that RPKI and ROA talk is gibberish to you, my new BGP e-book has a section on what RPKI is and how it works.)

    Read the whole article

Archive of all articles - RSS feed

My Books: "BGP" and "Running IPv6"

On this page you can find more information about my book "BGP". Or you can jump immediately to chapter 6, "Traffic Engineering", (approx. 150kB) that O'Reilly has put online as a sample chapter. Information about the Japanese translation can be found here.

More information about my second book, "Running IPv6", is available here.

BGP Security

BGP has some security holes. This sounds very bad, and of course it isn't good, but don't be overly alarmed. There are basically two problems: sessions can be hijacked, and it is possible to inject incorrect information into the BGP tables for someone who can either hijack a session or someone who has a legitimate BGP session.

Session hijacking is hard to do for someone who can't see the TCP sequence number for the TCP session the BGP protocol runs over, and if there are good anti-spoofing filters it is even impossible. And of course using the TCP MD5 password option (RFC 2385) makes all of this nearly impossible even for someone who can sniff the BGP traffic.

Nearly all ISPs filter BGP information from customers, so in most cases it isn't possible to successfully inject false information. However, filtering on peering sessions between ISPs isn't as widespread, although some networks do this. A rogue ISP could do some real damage here.

There are now two efforts underway to better secure BGP:

  • Secure BGP (S-BGP) is developed by Bolt, Beranek and Newman (BBN). It has been around for several years and there is a proof-of-concept implementation. S-BGP tries to secure all aspects of the BGP protocol, and subsequently needs several signature checks for each BGP update, making the protocol relatively heavy-weight. You can see my earlier rants on S-BGP at the top of this page. Note that I'm not as anti-S-BGP as I used to be any more, although I still think implementing the protocol will be expensive because routers will need lots of extra memory (up to four times as much) and CPU power (possibly dedicated crypto hardware) and this aspect deserves some serious attention.

    Secure BGP (S-BGP) index at BBN.

  • Secure Origin BGP (soBGP) has surfaced fairly recently and hails from Cisco. There are no implementations so far. soBGP mainly focusses on securing the relationship between prefixes and the source AS number, and doesn't need as many computationally expensive checks as S-BGP. However, the protocol can easily be expanded to perform more checks.

    draft-ng-sobgp-bgp-extensions-00.txt (main soBGP draft)
    draft-white-sobgp-bgp-extensions-00.txt (deployment considerations)

    (If the links don't work, the drafts have expired; you'll have to use a search engine to find them.)

There is now also a different approach to increasing BGP security using an "Interdomain Routing Validation" service that works independent from the BGP protocol itself. See what I wrote about this in interdomain routing news on this site, or jump immediately to the Working Around BGP: An Incremental Approach to Improving Security and Accuracy of Interdomain Routing paper.

The IETF RPSEC (routing protocol security) working group is active in this area.

What is is a website dedicated to Internet routing issues. What we want is for packets to find their way from one end of the globe to another, and make the jobs of the people that make this happen a little easier.

Your host is Iljitsch van Beijnum. Feedback, comments, link requests... everything is welcome. You can read more about me here or email me at iljitsch@bgpexpert. or follow iljitsch on Twitter.

Ok, but what is BGP?

Have a look at the "what is BGP" page. There is also a list of BGP and interdomain routing terms on this page.

BGP and Multihoming

If you are not an ISP, your main reason to be interested in BGP will probably be to multihome. By connecting to two or more ISPs at the same time, you are "multihomed" and you no longer have to depend on a single ISP for your network connectivity.

This sounds simple enough, but as always, there is a catch. For regular customers, it's the Internet Service Provider who makes sure the rest of the Internet knows where packets have to be sent to reach their customer. If you are multihomed, you can't let your ISP do this, because then you would have to depend on a single ISP again. This is where the BGP protocol comes in: this is the protocol used to carry this information from ISP to ISP. By announcing reachability information for your network to two ISPs, you can make sure everybody still knows how to reach you if one of those ISPs has an outage.

Want to know more? Read A Look at Multihoming and BGP, an article about multihoming I wrote for the O'Reilly Network.

For those of you interested in multihoming in IPv6 (which is pretty much impossible at the moment), have a look at the "IPv6 multihoming solutions" page.

Are you a BGP expert? Take the test to find out!

These questions are somewhat Cisco-centric. We now also have another set of questions and answers for self-study purposes.

You are visiting over IPv4. Your address is