IPsec Packet Construction: Understanding AH vs ESP and Transport vs Tunnel Modes

What Are IPsec Headers?

IPsec (Internet Protocol Security) is a suite of protocols designed to provide security services for internet protocols. It operates at the network layer of the OSI model, ensuring secure communication between hosts over an insecure network like the Internet. Two primary IPsec headers are used: Authentication Header (AH) and Encapsulating Security Payload (ESP).

AH provides data integrity and authentication but does not encrypt the payload. ESP, on the other hand, can provide both encryption and integrity checks for the entire packet.

AH vs ESP: Key Differences

The Authentication Header (AH) is used to ensure data integrity and authenticity of a packet by generating an authentication tag. AH operates in transport mode, which means it only protects the payload, or tunnel mode, where it protects both the original IP header and the payload. However, AH's operation can be problematic with Network Address Translation (NAT), as NAT changes the source IP address, leading to failed integrity checks.

Encapsulating Security Payload (ESP) is more versatile because it can operate in transport or tunnel mode. ESP not only provides data integrity but also encrypts the payload and optionally the entire packet header. Importantly, ESP uses sequence numbers and other techniques that allow it to function correctly even when NAT changes the IP addresses.

Transport vs Tunnel Modes

In transport mode, AH or ESP only protects the payload of a packet, leaving the original IP header unaltered. This is suitable for scenarios where you want to secure data without modifying the network path. However, this can lead to issues with NAT because the source and destination IP addresses in the IP header are changed by NAT, leading to failed integrity checks.

In tunnel mode, AH or ESP protects both the original IP header and the payload. This is particularly useful for scenarios where you need to secure data over a network that uses NAT, as it ensures that all parts of the packet remain consistent even after NAT modifications.

NAT Traversal: Ensuring Security Across Networks

Network Address Translation (NAT) is widely used to manage IP addresses and provide security by hiding internal network details. However, this can interfere with the operation of AH because NAT changes the source IP address in the original IP header, leading to failed integrity checks. ESP, on the other hand, uses techniques like sequence numbers that allow it to function correctly even when NAT modifies the IP addresses.

NAT Traversal mechanisms for ESP include techniques such as NAT-PMP (NAT Port Mapping Protocol) and UPnP (Universal Plug and Play), which help devices communicate through NAT routers by dynamically configuring port mappings.

Frequently asked questions

Try it live

Everything above runs in your browser — open IPsec Packet Builder 2D: AH vs ESP, Transport vs Tunnel, NAT Traversal and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.