$RodHat_
MOTD

TCP MD5 for BGP authentication is thirty years old. TCP-AO is the replacement. Nobody has migrated.

Published by

TCP MD5 for BGP authentication is thirty years old. TCP-AO is the replacement. Nobody has migrated.
Photo: AI-generated — no human photographer / RodHat AI Cover

RFC 2385 was published in August 1996. “Protection of BGP Sessions via the TCP MD5 Signature Option.” MD5. In 1996, MD5 was a reasonable choice for a message authentication code. In 2026, MD5 has been cryptographically broken for twenty years, the RFC it lives in has “historic” status in the IETF sense, and BGP sessions across most of the internet are still authenticated with it.

TCP-AO (RFC 5925) fixes this. It was standardized in 2010. Linux 6.7, released in January 2024, merged working in-kernel TCP-AO support. FRR supports TCP-AO in the 9.x series. BIRD 2.x has it. OpenBGPD has it. The migration path exists and has existed for years.

Most operators have not migrated. This is not surprising. It is still disappointing.

What RFC 2385 was doing and why it aged badly

The TCP MD5 Signature option adds a keyed MD5 hash of the TCP pseudo-header and data to a TCP option field. The receiving router verifies the hash before accepting the segment. A third party who cannot compute the correct hash cannot inject a valid segment, which means they cannot reset a BGP session by sending a spoofed RST.

The problem this was solving in 1996 was TCP session hijacking and RST injection. BGP runs over a long-lived TCP connection between peering routers. If an attacker can inject a TCP RST, they drop the peering session, cause route flapping, and do real damage to routing tables during the recovery window. TCP MD5 was a practical fix at the time.

MD5 is now broken as a collision-resistant hash function. The counterargument you will hear from operators who do not want to migrate is that RFC 2385 uses HMAC-MD5, not raw MD5, and HMAC-MD5 has not been broken the same way. They are technically correct. HMAC-MD5 is not as broken as raw MD5. The argument is also a thirty-year-old rationalization for not doing maintenance work. The IETF standardized a replacement because HMAC-MD5 correctness was never the only problem. RFC 2385 supports exactly one algorithm and cannot rotate keys without re-establishing the session. That is the real issue: zero agility, no path forward, forever frozen at one algorithm choice made the same year Quake shipped.

What TCP-AO actually fixes

RFC 5925 specifies the TCP Authentication Option. Three things are concretely better than RFC 2385.

The first is algorithm agility. TCP-AO separates key material from algorithm choice. You configure a Master Key Tuple (MKT) with an algorithm identifier and a key ID. The key ID travels in the option header. You can add a new MKT with a different algorithm, drain traffic to it, and retire the old one without dropping the session. RFC 2385 has no equivalent mechanism. You want to rotate the key on a live BGP session running MD5, you are dropping and re-establishing it.

The second is connection-specific key derivation. TCP-AO defines a Traffic Key derivation function that takes the MKT key and the TCP four-tuple plus the initial sequence numbers from both sides, and produces per-connection keys. RFC 2385 uses the raw configured password directly as the MAC input. If you reuse the same password across multiple peering sessions, a compromise of one session’s key material is a compromise of all of them. TCP-AO’s derivation function scopes keys to the individual connection.

The third is sequence number protection in the connection opening. RFC 2385 has a documented weakness around initial sequence number handling during the SYN exchange. TCP-AO covers the ISN in its MAC computation in a way that closes this off-path injection window. This is not a theoretical attack on most networks, but “not theoretical” is a low bar for infrastructure that will run for a decade unchanged.

Linux’s in-kernel implementation lives under net/ipv4/tcp_ao.c. It supports HMAC-SHA-1, HMAC-SHA-256, and AES-128-CMAC per RFC 5926. The setsockopt interface for raw socket programs is documented in the kernel.org tcp_ao docs. For FRR users, the daemon handles the socket configuration and you do not touch setsockopt directly.

The migration reality

The daemon support is not the bottleneck. FRR, BIRD, and OpenBGPD have all shipped TCP-AO support. IOS XR and JunOS got it in 2024 releases for the commercial side. The kernel has had it since January 2024.

The bottleneck is that TCP MD5 works. Routers running it have been running it for years, sometimes over a decade. Configuration that is not broken does not get touched on a schedule driven by an IETF standards body marking an RFC historic. The people who would do the migration are the same people keeping the rest of the routing infrastructure alive, and they have other things on their list.

There is also a peer coordination requirement. Both ends of a BGP session have to support TCP-AO before you can migrate. Transit providers run heterogeneous software across their peering fabric. If your peer is running an older IOS version without TCP-AO support, the migration waits on their upgrade cycle, not yours. Coordinating across organizations adds friction that unilateral changes do not have.

None of this is a good excuse for new deployments being configured with MD5. If you are standing up a new BGP session in 2026 on Linux with FRR 9.x and a 6.7+ kernel, there is no reason to reach for password in the neighbor stanza instead of the TCP-AO key configuration. Legacy migration is hard. Starting new sessions on a thirty-year-old authentication scheme is a choice.

What to check on an existing setup

vtysh -c "show bgp neighbors" in FRR includes the authentication type for each neighbor. If it says MD5, it is MD5. If you are curious about the socket-level state, ss -i on the BGP socket will show you live TCP internals, though TCP-AO state is more specifically inspectable via /proc/net/tcp_ao on kernels that expose it.

For a new FRR session the configuration is a tcp-authentication-key stanza under the neighbor block: algorithm (sha-256 is the current default recommendation), a key ID, and the key material. The kernel side is handled by FRR. You do not configure setsockopt unless you are writing your own BGP daemon, which you are not.

The IETF has been trying to retire RFC 2385 since 2010. Sixteen years. The replacement has been standardized for the same amount of time. The kernel got it two years ago. The daemon ecosystem has it. The industry’s track record on retiring old authentication infrastructure suggests TCP MD5 will be mostly gone around 2035, give or take a decade. I have been wrong before about how long these things take. Usually in the direction of being too optimistic.

See also: post-quantum key exchange is already deployed in TLS for what “secure enough for now” actually means when the migration timeline is measured in infrastructure cycles, and certificate lifetimes are collapsing for what forced crypto agility looks like when someone does actually pull the lever.

Sources

  1. RFC 5925: The TCP Authentication Option (Internet Engineering Task Force (IETF))
  2. TCP Authentication Option (TCP-AO) — The Linux Kernel documentation (The Linux Kernel Organization)