M3 / Key encapsulation

KEM vs classical key agreement

FoundationsPractitionerAdvisor

After this lesson you can

  • Define the mechanism of classical Diffie-Hellman (DH/ECDH) key agreement in one sentence
  • Explain step by step how a KEM (encapsulate/decapsulate) works mechanically differently from DH
  • Explain what this mechanical difference changes in TLS in practice (message contents, which side produces what)

Before thisML-KEM and FIPS 203

Mental model

Classical key agreement (Diffie-Hellman, DH) is a mechanism in which the two sides perform a symmetric computation: each side produces its own private value, combines it with the other side’s public value, and both reach mathematically the same result (the shared secret). A KEM (Key Encapsulation Mechanism) breaks this symmetry: the two sides perform different operations. As you will see below, this difference is not a necessary consequence of lattice mathematics but a deliberate security-proof choice by ML-KEM’s designers.

DH: symmetric, both sides do the same operation

In classical ECDH (for example X25519, RFC 7748), each side takes the same three steps: produce a private key, compute the matching public key, send the public key to the other side. Then both sides reach the shared secret using the same formula (their own private key × the other side’s public key). It is mathematically symmetric: A’s computation and B’s computation have exactly the same structure, only with the inputs reversed.

KEM: asymmetric, the sides do different operations

A KEM consists of three separate algorithms, and these three are run by different sides, in a particular order:

  1. KeyGen: one side produces a key pair (an encapsulation key, which is public, and a decapsulation key, which is private) and sends the encapsulation key to the other side.
  2. Encapsulate: the other side uses that encapsulation key to produce both a shared secret and a ciphertext; it sends the ciphertext back and keeps the shared secret.
  3. Decapsulate: the first side uses its private key and the received ciphertext to reach the same shared secret.

The difference: in DH both sides “produce a public key and send it” and arrive at the result with a symmetric formula; in a KEM only one side produces a key pair (KeyGen), the other side uses that public key to produce both the secret and the ciphertext in one step (Encapsulate), and the first side decapsulates the ciphertext to reach the same secret (Decapsulate). The two sides run different algorithms; it is not symmetric.

Why the difference exists: a design choice, not a mathematical necessity (extra depth for ADVISOR/PRACTITIONER; FOUNDATIONS may skip)

This section is not needed for any lesson outcome; it is for those who want to be ready when a hostile architect asks “why a different name, isn’t a KEM just DH rebranded”.

The accurate answer: a lattice-based key agreement working symmetrically like DH is not mathematically impossible. The design called NewHope, a direct ancestor of Kyber/ML-KEM, had exactly such a DH-like, symmetric structure: both sides produced and sent their own lattice-based values, and a technique called “reconciliation” brought the two sides’ close-but-not-equal values down to the same shared secret. ML-KEM’s designers chose the KEM (encapsulate/decapsulate) path instead, not out of mathematical necessity but as a security-proof and engineering choice. The KEM structure reaches a strong security guarantee (CCA2 security, roughly: resistance to an attacker leaking information by tampering with ciphertexts) more cleanly through a standard technique called the “Fujisaki-Okamoto transform”; reconciliation-based designs needed more fragile, error-prone mechanisms to provide that guarantee and ran into side-channel security problems. In short: a DH-like lattice design was possible and was tried; ML-KEM choosing the KEM structure was a deliberate engineering decision for a sturdier security proof, not “lattice math allows nothing else”.

When answering this question for an architect, saying “it is mathematically required” is a checkable and wrong claim; saying “a design choice, for the security proof”, as above, is both correct and a stronger answer, because it actually explains why NIST took this path.

What changes in practice: the TLS example

In a TLS 1.3 handshake this mechanical difference becomes directly visible. In a classical (X25519) handshake, both client and server send their own ephemeral public key (in the key_share extension) and both reach the shared secret with the same formula. In a hybrid (X25519MLKEM768) handshake, the client runs ML-KEM KeyGen and sends its ML-KEM encapsulation key in the ClientHello key_share, alongside its X25519 public key. The server encapsulates against that key and returns the ML-KEM ciphertext in the ServerHello key_share, alongside its own X25519 public key; the client then decapsulates. Both secrets are combined into the handshake secret.

This is the mechanical explanation for the real measurements you will see in M8 (the ClientHello growing from 308 BMEASURED to 1484 BMEASURED): the client now carries an ML-KEM-768 encapsulation key of 1184 BSOURCED, and the server’s reply grows by an ML-KEM-768 ciphertext of 1088 BSOURCED. Unlike DH, where each side sends one small public key, a KEM puts a large public key in one direction and a large ciphertext in the other.

Sources

Checkpoint

Answer first, then compare with the model answer and score yourself against the rubric. Saved in this browser only.

  1. 01Recall

    What are the three algorithms of a KEM (name them)?

  2. 02Recall

    In DH both sides compute something. In a KEM is this symmetric, or do the sides do different operations?

  3. 03Scenario

    In a TLS 1.3 hybrid handshake, which side runs KeyGen, which side encapsulates and which decapsulates? Describe it in a client-server scenario.

  4. 04Hostile

    An architect says 'Isn't a KEM just DH renamed? Why a different name?' Answer by showing the mechanical difference.

Project linkHelps you recognize which messages carry ML-KEM's key and ciphertext when examining a TLS handshake with packet capture in Project 1 (measurement lab).