IPSec vs MASQUE

For decades, private traffic hid inside a special protocol. IPsec, OpenVPN, and WireGuard still do that: they punch an encrypted L2/L3 tunnel across the public path, then stuff TCP, UDP, SCTP, or RTP inside it. Middleboxes learned the silhouette. ESP is protocol 50. IKE is UDP/500. If those ports are blocked, the VPN is gone.

HTTP/3 broke that bargain. QUIC runs over UDP, in user space, multiplexes many streams on one connection, and looks like HTTPS on 443. The tunnel no longer has to advertise itself as a tunnel.

That is the opening MASQUE took. A MASQUE proxy is not another IPsec daemon. It is an HTTP server. The client sends Extended CONNECT. Inner UDP or inner IP rides as HTTP Datagrams, preferably in QUIC DATAGRAM frames. To the path, it is a website. To the endpoints, it is still a private channel.

In 2022 I called MASQUE nascent in Encapsulting Protocols. It is not. MASQUE has RFCs for datagrams, UDP proxying, and IP proxying. IPSECME did not yield the field: IPsec grew TCP encapsulation, IP-TFS, per-resource Child SAs, and post-quantum PSK mixing. Same job, two stacks. This post contrasts IPsec vs MASQUE: tunnel mode and IKEv2 on one side, CONNECT-IP and CONNECT-UDP on the other.

But why this comparison ?

Both IPsec tunnel mode and MASQUE CONNECT-IP can carry an entire inner IP packet across an untrusted path. Both can be used as remote-access VPN, site-to-site VPN, or a general packet tunnel. Both rewrite outer addressing so that the path sees a tunnel endpoint rather than the original source and destination.

The difference is where the tunnel lives, and therefore how it is built.

IPsec is a network-layer security architecture. ESP (and optionally AH) protect IP packets. IKEv2 negotiates keys, algorithms, and traffic selectors. The outer packet is still IP, typically protocol 50 (ESP) or UDP/4500 after NAT traversal. The tunnel is identified by a Security Parameter Index (SPI) bound to a Security Association (SA). User-space VPN clients exist, but the data plane is still an IP-layer SA, often terminated in the kernel.

MASQUE is an HTTP extension family. An HTTP client opens an Extended CONNECT request to an HTTP server that acts as a proxy. Inner UDP datagrams or inner IP packets then travel as HTTP Datagrams, preferably inside QUIC DATAGRAM frames on HTTP/3. The outer packet looks like ordinary HTTPS. The tunnel is identified by an HTTP request stream, plus a Context ID inside the datagram payload. Multiplexing is native: several CONNECT-UDP and CONNECT-IP tunnels, plus ordinary web requests, can share one QUIC connection.

For VoIP and telecom this matters greatly because carrier and enterprise VPNs, IMS interconnect, and many CPE devices already speak IPsec. And browser, mobile paths increasingly speak HTTP/3 but QUIC pkts are often blocked unless they look like web traffic.

IPsec as a traditional tunneling suite

RFC 4301 defines IPsec as a security architecture for IP, not a single packet format. The unit of state is the Security Association. An SA binds:

• cryptographic algorithms and keys
• traffic selectors (what inner traffic is allowed)
• mode (transport or tunnel)
• sequence-number and anti-replay state
• encapsulation parameters (UDP, TCP, DSCP handling, and so on)

Inbound packets are demultiplexed with the SPI, a 32-bit identifier in the AH or ESP header. As in the encapsulating-protocols post, SPI is the tunnel identifier analogous to a GRE key, an MPLS label, or a QUIC connection ID. It attaches SA keys to each packet so the receiver can process the packet without first parsing the inner header.

Keying can be manual, but operational IPsec uses IKEv2 (RFC 7296, Internet Standard / STD 79). IKEv1 is deprecated (RFC 9395). IKEv2 runs over UDP/500, or UDP/4500 when NAT is detected. A typical exchange:

• INFORMATIONAL for liveness, deletes, configuration payloads
• IKE_SA_INIT: cryptographic suites, nonces, Diffie-Hellman (or later a KEM)
• IKE_AUTH: identities, certificates or EAP, and the first Child SA
• CREATE_CHILD_SA: additional Child SAs, rekeys

Transport mode protects the payload of an existing IP packet. Tunnel mode encapsulates the original packet and gives it a new outer IP header addressed to the IPsec peer. Site-to-site VPN, remote-access VPN, and most overlay use cases are tunnel mode.

Transport mode (host-to-host protection of the payload):
  [ IP header ][ ESP hdr ][ TCP/UDP/... ][ ESP trailer + ICV ]
Tunnel mode (gateway-to-gateway or remote-access VPN):
  [ Outer IP ][ ESP hdr ][ Inner IP ][ Inner payload ][ ESP trailer + ICV ]

MASQUE CONNECT-IP is conceptually closer to tunnel mode: the inner packet is a full IP packet, and the outer addressing belongs to the HTTP/QUIC connection.

MASQUE as HTTP/QUIC proxying

MASQUE( Multiplexed Application Substrate over QUIC Encryption) client is an HTTP client. A MASQUE proxy is an HTTP server. Between them, the path only needs to allow HTTPS. That is the architectural bet: if the tunnel looks like web traffic, it inherits web reachability.

HTTP Datagrams and the Capsule Protocol

RFC 9297 is the substrate. It defines two things:

1. HTTP Datagrams which is potentially unreliable datagrams associated with an HTTP request, not with HTTP message content.
2. Capsule Protocol which is a TLV( type-length-value) sequence on the request’s data stream, used for reliable control messages and as a fallback when QUIC DATAGRAM is unavailable.

On HTTP/3, an HTTP Datagram rides in a QUIC DATAGRAM frame (RFC 9221). Here the Quarter Stream ID binds the datagram to the CONNECT request that opened the tunnel.

QUIC DATAGRAM {
  Quarter Stream ID (i),   // request stream ID / 4
  HTTP Datagram Payload (..)
}

CONNECT-UDP

RFC 9298 defines the connect-udp upgrade token. It is the UDP analogue of HTTP CONNECT.

:method = CONNECT
:protocol = connect-udp
:scheme = https
:authority = proxy.example.org
:path = /.well-known/masque/udp/target.example.com/443/

After a 2xx response, each inner UDP datagram is an HTTP Datagram. This is the MASQUE mode that matters for WebRTC, QUIC, DNS-over-UDP, and games: one UDP 4-tuple through an HTTP proxy without converting the inner protocol to TCP.

CONNECT-IP

RFC 9484 defines connect-ip, the MASQUE equivalent of an IPsec tunnel: arbitrary inner IP packets, including protocols HTTP CONNECT and CONNECT-UDP cannot carry.

Outer encryption is mandatory: IP proxying MUST run over TLS, QUIC, or equivalent. MASQUE does not replace ESP’s cryptographic services with a new IPsec transform. It borrows TLS/QUIC for the outer hop.

HTTP Datagram payload for CONNECT-IP:

IP Proxying HTTP Datagram Payload {
  Context ID (i),
  Payload (..)
}

When forwarding, CONNECT-IP also decrements TTL / Hop Limit on encapsulation, matching IPsec, so routing loops do not live forever.

Security model

IPsec authenticates IP-layer peers and protects inner packets with algorithms negotiated per SA. It does not require DNS, PKI for web PKI, or an HTTP stack. It can run between routers that have no notion of URLs. Confidentiality, integrity, and replay protection are properties of ESP itself.

MASQUE authenticates the HTTP server with TLS (typically Web PKI) and optionally the client with TLS client certificates or HTTP authentication. Confidentiality of the outer tunnel is TLS/QUIC. Inner IP packets on Context ID 0 are not additionally ESP-protected unless the application stacked IPsec inside. Replay of inner packets is not an ESP anti-replay window; it is whatever the inner protocol and QUIC packet protection provide.

• CONNECT-UDP proxy sees the target host and port, not the inner QUIC or RTP payload.
• CONNECT-IP proxy sees full inner IP headers unless the inner packet is itself encrypted (inner IPsec, inner QUIC, inner TLS). 

So on the face of it the diff are quite stark, for unique identification  IPsec puts it in SPI. MASQUE puts it in the HTTP request + context Id. One HTTP/3 connection can carry many CONNECT-UDP and CONNECT-IP tunnels plus ordinary web requests. That is the “multiplexed application substrate” in the name. IPsec can also carry many Child SAs under one IKE SA, but each Child SA is still an IPsec SA, not an HTTP stream, and outer packets do not share HTTP/3’s stream scheduler.

But the diff lies in operation. The firewalls, NATs and middleboxes. IPsec native ESP (protocol 50) fails on many NATs. UDP/4500 tries to fixes most of them but remaining failure modes: UDP blocked, UDP timeouts shorter than IKE liveness, and enterprise firewalls so still exist and TCP encap attempts to address them( somewhat).

MASQUE’s outer flow is HTTPS. On HTTP/3 it is UDP/443 and on HTTP/2 it is TCP/443. Middleboxes that already allow web traffic will allow the tunnel. The proxy, not the client, opens the inner UDP socket or forwards the inner IP packet, so the client’s NAT only needs to keep the HTTPS mapping.

Mobility: QUIC connection migration is an advantage MASQUE gets for free on HTTP/3. IPsec needs MOBIKE or a new IKE SA when the outer address changes.

Telecom and real-time use cases

Real-time stacks (WebRTC, RTP, MoQ) care more about latency. HTTP/3 DATAGRAM gives MASQUE a native datagram path. IPsec ESP is already a datagram path, but it is not HTTP and is easier for middleboxes to classify and block.

Site-to-site and carrier VPN

IPsec tunnel mode remains the default(well so far). Hardware offload, IKEv2, traffic selectors, and operational practice (including LI and routing integration) are decades deep. CONNECT-IP can do the same overlay but the peers are HTTP clients and servers, not IKE-speaking gateways.

Remote access / SASE / Zero Trust

This is where MASQUE is already deployed in consumer and enterprise proxies: the client already speaks HTTPS, the policy point is an HTTP proxy, and CONNECT-UDP lets QUIC and DNS-over-HTTPS/3 through without forcing TCP. IPsec remote access (IKEv2 with EAP, certificates, or PSK) remains dominant on managed devices (laptops, mobile VPNs), and anything that must assign a stable inner address into an enterprise routing domain.

WebRTC, RTP, and MoQ. 

TURN (RFC 8656) is the media relay for WebRTC. But because CONNECT-UDP is the HTTP-proxy equivalent of a UDP relay, it becomes useful when the only allowed egress is an HTTP proxy rather than a TURN server.

Inner RTP/RTCP stay UDP. MoQ objects flow on QUIC datagrams or streams through the same CONNECT-UDP or CONNECT-IP path.

IMS / 3GPP

IPsec (often with IKEv2 and, in some interfaces, specific profiles) is baked into 3GPP security architecture. MASQUE is not. Do not expect CONNECT-IP to appear on N2/N3/N4. Do expect it on the user-facing proxy path and OTT apps

Nested tunnels. RFC 9484 notes CONNECT-IP can forward ESP itself. That is a way to get IPsec through an HTTP-only network: IKE and ESP become inner IP protocols on a CONNECT-IP tunnel (or IKE/ESP can be CONNECT-UDP’d if they have been UDP-encapsulated). Overhead and MTU become severe; treat this as a reachability hack, not a design goal.

Confused on when to use which ?

They are not substitutes in every deployment. CONNECT-IP can even carry inner ESP, and IPsec can protect a MASQUE outer hop. But in my opinion use IPSec when both endpoints are IKE-capable gateways or managed VPN clients especially if you need strict policy control like group keying, IP-TFS, or router-based VPN. If you are reading this article and have reached this far that you know you need IPSec if you are in a 3GPP, CPE, or existing IPsec operational domain.

Use MASQUE CONNECT-UDP if the inner protocol is UDP (QUIC, DNS, WebRTC, games) and the path has HTTP proxy or a network that only allows HTTPS.

Use MASQUE CONNECT-IP when you need a full or split IP tunnel, and the client and proxy are HTTP endpoints. You can also use it if a new remote-access or site-to-site VPN is being built on HTTP/3 infrastructure rather than on IKE.

Prefer HTTP/3 DATAGRAM for either MASQUE mode. Fall back to HTTP/2 capsules or IPsec TCP encapsulation only when UDP is blocked.

Do not treat MASQUE as “IPsec over QUIC.” IPsec is still the IP-layer SA architecture. MASQUE is HTTP proxying of UDP and IP. The overlapping VPN use cases are real; the control planes, identifiers, and trust models are not the same.

At the time of writing this article these things are still being standardized

Published MASQUE RFCs as of this writing: RFC 9297, RFC 9298, RFC 9484. RFC 9298 is updated by RFC 9484 and RFC 9931.

Active MASQUE work (not RFCs): Ethernet frame proxying, bound UDP listen, QUIC-aware proxying, CONNECT-IP DNS and PREF64, ECN/DSCP for CONNECT-UDP, HTTP Datagram field compression.

On the IPsec side, the architecture RFCs remain RFC 4301 / 4302 / 4303 and IKEv2 RFC 7296. IPSECME’s recent published extensions are operational rather than a new tunnel mode: RFC 9329, RFC 9347, RFC 9611, RFC 9838, RFC 9867. Diet-ESP, EESP, and ML-KEM in IKEv2 are still drafts.

The 2022 note that MASQUE was nascent can be retired for UDP and IP proxying. Ethernet, QUIC-aware proxying, and IPsec header compression are the parts that are still moving.

References

Do not start at RFC 4301 and read until your eyes glaze over. Pick a path.

If you only have an afternoon, open these four:

1. RFC 4303 ESP — the IPsec tunnel you already know
2. RFC 7296 IKEv2 — how that tunnel is keyed
3. RFC 9298 CONNECT-UDP — MASQUE for one UDP flow
4. RFC 9484 CONNECT-IP — MASQUE as a VPN-shaped IP tunnel

MASQUE

Start here if the question is “what did the IETF actually ship?”

• RFC 9297 HTTP Datagrams and the Capsule Protocol — the substrate. One blob, two encodings.
• RFC 9298 Proxying UDP in HTTP — CONNECT-UDP. Updated by RFC 9484 and RFC 9931.
• RFC 9484 Proxying IP in HTTP — CONNECT-IP, addresses, routes, full inner packets.
• RFC 9931 Security Considerations for Optimistic Protocol Transitions in HTTP/1.1 — do not send UDP on HTTP/1.1 before the upgrade sticks.

The CONNECT request itself is ordinary HTTP. These are the docs underneath it:

• RFC 9110 HTTP Semantics — classic CONNECT for TCP
• RFC 8441 / RFC 9220 Extended CONNECT on HTTP/2 and HTTP/3
• RFC 9114 HTTP/3
• RFC 9000 QUIC
• RFC 9221 QUIC DATAGRAM — the unreliable frame MASQUE wants for media

IPsec

Start here if the question is “why does ESP still win on gateways?”

• RFC 4301 /02/03 : Security Architecture for the Internet Protocol
• RFC 7296 IKEv2 (STD 79) — the control plane. RFC 9395 retired IKEv1.
• RFC 8221 / RFC 8247 which algorithms you are supposed to implement
• RFC 8750 Implicit IV — a few bytes less per packet

When the path is hostile, these are the ones operators actually reach for:

• RFC 3948 UDP encapsulation of ESP — NAT-T, UDP/4500
• RFC 4555 MOBIKE — the outer address moved
• RFC 9329 TCP encapsulation of IKE and IPsec — UDP is blocked, 443/TCP is not
• RFC 9347 IP-TFS — hide inner packet sizes

IKEv2 did not freeze in 2014. Open these when the problem is keys, Child SAs, or quantum:

• RFC 8784 and RFC 9867 mixing PSKs for post-quantum security
• RFC 9242 / RFC 9370 extra key-exchange rounds
• RFC 9464 encrypted DNS via IKEv2
• RFC 9611 per-resource Child SAs -> one replay window per resource, not one giant SA
• RFC 9827 ESN transform rename
• RFC 9838 group key management

Also not RFCs yet:

• draft-ietf-ipsecme-diet-esp Diet-ESP header compression
• draft-ietf-ipsecme-eesp Enhanced ESP
• draft-ietf-ipsecme-ikev2-mlkem ML-KEM in IKEv2

If you came from the Encapsulting Protocols post then draft-altanai-tsv-multipath-nested-tunnels Congestion-Aware Multipath Tunnel Selection which describes after you nest MASQUE and IPsec, how to score GRE → IPsec → MASQUE vs IPsec → CONNECT-UDP vs direct. Path selection, not protocol interworking.