Cryptography

Hybrid Encryption: How RSA and AES Work Together

8 min read
By
Hybrid Encryption: How RSA and AES Work Together

Photo by Tima Miroshnichenko from Pexels

RSA is the most famous public key algorithm, but it has a problem: it is slow and can only encrypt small messages. AES is fast and can encrypt unlimited data, but it needs a shared key that both parties must already have.

Hybrid encryption solves this by using both together. RSA handles the key exchange, AES handles the data. This is how virtually all real-world encryption works, from HTTPS to encrypted email to cloud storage.

Let's break down why this is necessary and how it works.

Why You Cannot Just Use RSA for Everything

RSA has two practical limitations that prevent it from being used alone:

**1. Message size limit.** RSA can only encrypt messages smaller than the modulus. For a 2048-bit key with OAEP padding, the maximum message size is about 245 bytes. That is enough for a short text message or a symmetric key, but not for a file, an image, or a web page.

Even a 4096-bit key only gives you about 445 bytes. You cannot encrypt a 1 MB file with RSA directly. You would need to split the file into tiny chunks, encrypt each one separately, and decrypt each one separately. This is slow, complex, and unnecessary.

**2. Performance.** RSA operations are computationally expensive. Encrypting 245 bytes with a 2048-bit RSA key takes about 1 millisecond. AES can encrypt 245 bytes in about 1 microsecond, a thousand times faster.

For a 1 MB file split into 4096 chunks of 245 bytes each, RSA encryption would take about 4 seconds. AES encryption of the same 1 MB file takes about 1 millisecond. The difference is enormous.

Why You Cannot Just Use AES for Everything

AES is fast and has no practical message size limit, but it has a different problem: it is symmetric. Both the sender and receiver need the same key.

If Alice wants to send an encrypted message to Bob for the first time, she has no way to share an AES key with him securely. She could send it in a separate message, but that message would also need to be encrypted, creating a chicken-and-egg problem.

This is the key distribution problem that public key cryptography was invented to solve.

The Hybrid Solution

Hybrid encryption combines the best of both:

1. **RSA solves the key distribution problem.** Alice can use Bob's public key to securely send an AES key. No pre-shared secret is needed. 2. **AES solves the performance and size problems.** AES encrypts the actual data quickly, with no size limit.

Here is the step-by-step flow:

**Step 1: Generate a random AES key.** Alice generates a fresh, random AES key (typically 128 or 256 bits). This key is for this one message or session only. It is called the session key or content encryption key.

**Step 2: Encrypt the data with AES.** Alice encrypts the actual message using AES and the session key. This can be any size, from a few bytes to terabytes.

**Step 3: Encrypt the AES key with RSA.** Alice encrypts the session key using Bob's RSA public key. The session key is small (16 or 32 bytes), well within RSA's size limit.

**Step 4: Send both.** Alice sends the RSA-encrypted session key and the AES-encrypted data to Bob.

**Step 5: Decrypt the AES key with RSA.** Bob uses his RSA private key to decrypt the session key.

**Step 6: Decrypt the data with AES.** Bob uses the recovered session key to decrypt the actual data with AES.

The result: Alice and Bob have securely exchanged a large encrypted message without ever meeting or sharing a key in advance. RSA was used only for the small key exchange, AES was used for the bulk data.

TLS: The Real-World Example

The most common use of hybrid encryption is TLS, the protocol that powers HTTPS. When you visit a website, here is what happens:

1. Your browser connects to the server. 2. The server sends its certificate, which contains its RSA public key. 3. Your browser and the server perform a key exchange (using RSA or, more commonly, ECDHE for forward secrecy) to establish a shared symmetric key. 4. All subsequent communication is encrypted with AES using that shared key.

The RSA (or ECDHE) part takes a few milliseconds and happens once per connection. The AES part takes microseconds per message and happens for every byte of data transferred. This is why HTTPS is fast despite using "slow" public key cryptography.

Envelope Encryption in Cloud Systems

Cloud providers use a specific pattern of hybrid encryption called envelope encryption. Here is how it works:

**Data Encryption Key (DEK):** A unique AES key generated for each piece of data (each file, each database row, each object in storage). The DEK encrypts the data.

**Key Encryption Key (KEK):** A master key that encrypts the DEK. The KEK is typically an RSA key or an AES key stored in a key management service (like AWS KMS or Google Cloud KMS).

The flow: 1. Generate a random DEK. 2. Encrypt the data with the DEK using AES. 3. Encrypt the DEK with the KEK using RSA (or AES). 4. Store the encrypted data and the encrypted DEK together. 5. The KEK never leaves the key management service.

The advantage of envelope encryption is key rotation. To rotate the KEK, you only need to re-encrypt the DEKs, not all the data. Since DEKs are small (32 bytes), this is fast even if you have millions of encrypted objects.

Forward Secrecy

Modern TLS implementations use a variant of hybrid encryption called ECDHE-RSA that provides forward secrecy. With plain RSA key exchange, if the server's RSA private key is compromised in the future, all past recorded traffic can be decrypted.

With ECDHE (Elliptic Curve Diffie-Hellman Ephemeral), a new ephemeral key pair is generated for each connection. The RSA key is used only to sign the handshake (proving the server's identity), not to encrypt the session key. The session key is derived from the ECDHE exchange, and the ephemeral keys are discarded after the connection. Even if the RSA key is compromised later, past sessions cannot be decrypted because the ephemeral keys no longer exist.

This is why modern TLS prefers ECDHE over plain RSA for key exchange. But the hybrid principle is the same: asymmetric cryptography for key establishment, symmetric cryptography for data.

Why Hybrid Encryption Is More Secure

Counterintuitively, hybrid encryption is more secure than using RSA alone, not less:

- The AES key is generated randomly for each session, so it has full entropy. - The AES key is never stored or reused, so compromise of one session does not affect others. - Breaking the encryption requires either breaking RSA (to recover the AES key) or breaking AES (to recover the data directly). - AES-256, used for the bulk data, provides 256 bits of security, which is stronger than RSA 2048's 112 bits.

The weakest link is the RSA key exchange, which is why choosing an appropriate RSA key size (2048 or 4096) matters. See our RSA 2048 vs 4096 guide for details.

The Bottom Line

Hybrid encryption is not a workaround or a compromise. It is the correct way to use public key cryptography for real-world data. RSA is for key exchange, AES is for data. Every secure communication system you use, from HTTPS to Signal to encrypted cloud storage, uses this pattern.

Understanding hybrid encryption helps you understand why RSA key size matters (it protects the key exchange), why AES key size matters (it protects the data), and why both are needed. For more on the individual components, read our RSA encryption explained guide and our AES-256 GCM explained guide. You can also experiment with RSA operations using our RSA Encrypt/Decrypt tool.

Frequently Asked Questions

Why can RSA not encrypt large files directly?

RSA can only encrypt messages smaller than the key modulus. For a 2048-bit key with OAEP padding, the maximum is about 245 bytes. Even a 4096-bit key can only handle about 445 bytes. To encrypt larger data, you need hybrid encryption: RSA encrypts a symmetric key, and the symmetric key encrypts the data.

Is hybrid encryption used in TLS?

Yes. TLS is the most common example of hybrid encryption. During the TLS handshake, RSA (or another public key algorithm like ECDHE) is used to establish a shared symmetric key. All subsequent data is encrypted with AES using that shared key. This combines the key distribution advantage of public key cryptography with the speed of symmetric encryption.

What is envelope encryption?

Envelope encryption is a specific pattern of hybrid encryption used in cloud systems. A data encryption key (DEK) encrypts the data using AES. A key encryption key (KEK) encrypts the DEK using RSA or another algorithm. The encrypted data and encrypted DEK are stored together. This allows you to rotate the KEK without re-encrypting all the data, only the DEK.

Is hybrid encryption as secure as using RSA alone?

Hybrid encryption is at least as secure as RSA alone, and in practice more secure. The symmetric key is generated randomly for each session, so it has full entropy. The RSA encryption protects the key during exchange. The AES encryption protects the data with a strong, well-studied algorithm. Breaking the system requires either breaking RSA (to recover the key) or breaking AES (to recover the data directly).

Try NovelCrypt Tools

Experience military-grade encryption for your sensitive data. Create self-destructing messages, encrypt files, or explore our experimental lab tools.

Explore NovelCrypt