Envelope encryption is the practice of encrypting data with a locally generated data encryption key, then encrypting that key with a key encryption key held in a central key management service. The encrypted data and its wrapped key are stored and transmitted together, so decryption requires access to the KMS rather than to a shared secret.
The core mechanical fit for Kafka
The pattern has three parts, and the definitions come straight from the KMS providers that popularised it. Google Cloud defines envelope encryption as the process of encrypting a key with another key. The data encryption key, the DEK, encrypts the data itself and is generated locally. The key encryption key, the KEK, wraps the DEK and is stored centrally in the KMS. Google’s guidance for the DEK: use a strong algorithm such as 256-bit AES in Galois Counter Mode, and when stored, always ensure DEKs are encrypted at rest.
A KMS key detail page, and an encrypted record as the broker stores it beside the decrypted value.
AWS documents the same flow through its data key API. A GenerateDataKey call returns a plaintext data key for immediate use and an encrypted copy that can be safely stored with the data. The application encrypts locally, discards the plaintext key from memory as soon as possible, and later passes the encrypted key back to the KMS to decrypt it.
Applied to a Kafka record, the producer encrypts the payload with a DEK, and publishes the ciphertext together with the wrapped DEK, carried in the message envelope or a record header. The broker stores opaque bytes end to end. A consumer authorised against the KMS unwraps the DEK and decrypts locally. This matters on Kafka specifically because the broker does not encrypt data at rest itself, as our Kafka security architecture guide lays out. TLS protects data in transit, and disk-level encryption protects the volume, but neither protects a record from anyone with read access to the topic, which is the gap envelope encryption closes.
Definitions in this section come from Google Cloud KMS and the AWS KMS data key documentation.
Solving the Kafka constraints
Three properties of Kafka make envelope encryption the workable pattern rather than a nice option.
Throughput versus KMS round-trips. A producer pushing tens of thousands of messages per second, the working scale the complete Kafka guide assumes throughout, cannot call the KMS once per message. Under envelope encryption the KMS call happens per key, not per record: a producer generates or unwraps a DEK once, caches it locally, and reuses it across many records. KMS traffic then scales with key rotation frequency rather than message rate, which keeps both latency and KMS API cost flat as volume grows.
Immutable logs versus deletion requirements. A Kafka topic cannot delete one record on demand, which collides with erasure obligations under regimes such as GDPR. Retention settings, covered in the governance-policy context on what is a data governance policy, delete by age or size, never by subject. Crypto-shredding resolves the collision: encrypt each tenant’s or subject’s records under their own key hierarchy, and destroy the key when the data must die. The ciphertext remains in the log but is permanently unreadable. Central key storage is what makes this auditable. Google notes that a central key service gives a singular point to more easily audit and restrict data access.
Key rotation without rewriting history. Rotating a KEK means re-wrapping DEKs, not re-encrypting log segments. Old records stay readable through their existing wrapped DEKs while new writes use fresh keys, which is the only rotation model an append-only log can afford.
One interaction to plan for: Kafka compresses at the batch level, and well-encrypted bytes do not compress. Producers encrypting full payloads should expect batch compression to stop paying for itself, which shifts the size budget conversation to the payload layer.
Implementation trade-offs and tooling
The implementation decision is where the encryption happens, and each option trades transparency against operational surface.
In the client. Producers and consumers encrypt through their serializers or interceptors. Nothing new sits in the data path, and field-level encryption is possible, encrypting only the sensitive fields so the record’s structure stays visible to schema tooling. The cost is a cryptography dependency in every producing and consuming application, in every language the platform supports. At least one large North American bank took this route and built a custom message-level encryption library for exactly this purpose, which illustrates both that the pattern works and that the build is substantial.
In a proxy. A layer-7 Kafka proxy encrypts and decrypts records transparently, so applications stay unchanged. The open-source option is Kroxylicious, an Apache-2.0 licensed Kafka proxy whose record encryption filter performs the envelope wrapping against a pluggable KMS. Commercial gateways occupy the same position. The trade is a new component in the hot path: every byte now flows through the proxy, which adds latency, an availability dependency and real operational overhead. Tencent’s published account puts proxy-layer overhead at roughly 30 percent in one large deployment.
Schema registry interaction. Whole-payload encryption turns records into opaque bytes, which blinds schema validation, compatibility checks and any tooling that inspects topic data. Field-level encryption preserves the record structure and keeps the registry useful, at the cost of a more complex encryption contract. This choice, more than performance, is what usually decides client-side field-level encryption over whole-payload approaches for schema-governed platforms, and it is the same trade the stream governance enforcement layer weighs for masking.
On the proxy route, I have been watching Kroxylicious closely. When they announced multi-topic and multi-cluster routing I called it big news in our own Slack: it will make all sorts of things useful and massively simplify cluster migrations, and I said at the time I would look at contributing myself because the project looks fun. A proxy that can route and encrypt at the record level without touching client code is the direction this tooling is heading.