Skip to content

ADR-004: Hybrid Post-Quantum Encryption (X25519 + ML-KEM-768)

Status

Accepted | 2025-12-29 (Fully Implemented)

Context

AEGIS stores sensitive data at rest including governance private keys, telemetry PII, and audit trails. While current symmetric encryption (AES-256) remains secure against quantum attacks (Grover's algorithm only halves effective security to 128-bit), the key exchange mechanisms used to establish encryption keys are vulnerable to Shor's algorithm.

Threat Model

Current Risk: LOW (no CRQCs exist today)
Future Risk: HIGH (15-30 year horizon per NIST estimates)
Attack Vector: "Harvest now, decrypt later"
Impact: Encryption key recovery → confidentiality breach

Data at Risk

Data Type Sensitivity Retention PQ Priority
Governance private keys CRITICAL Years IMMEDIATE
Override audit trail HIGH 7+ years (compliance) HIGH
Actor identities HIGH Years HIGH
Telemetry PII MEDIUM 90 days MEDIUM

NIST Standardization (August 2024)

NIST finalized ML-KEM in FIPS 203: - ML-KEM-512: NIST Level 1 (128-bit) - ML-KEM-768: NIST Level 3 (192-bit) - ML-KEM-1024: NIST Level 5 (256-bit)

Decision

Implement hybrid X25519 + ML-KEM-768 + AES-256-GCM encryption with HKDF key derivation.

Architecture

┌─────────────────────────────────────────────┐
│         HybridKEMProvider                   │
├─────────────────────────────────────────────┤
│  algorithm_name: "x25519+ml-kem-768+aes-256-gcm"│
│                                              │
│  PUBLIC_KEY_SIZE  = 1,216 bytes              │
│    └─ X25519:      32 bytes                  │
│    └─ ML-KEM-768: 1,184 bytes                │
│                                              │
│  PRIVATE_KEY_SIZE = 2,432 bytes              │
│    └─ X25519:      32 bytes                  │
│    └─ ML-KEM-768: 2,400 bytes                │
│                                              │
│  ENCRYPTED_OVERHEAD = 1,148 bytes            │
│    └─ Ephemeral X25519: 32 bytes             │
│    └─ ML-KEM ciphertext: 1,088 bytes         │
│    └─ AES-GCM nonce: 12 bytes                │
│    └─ AES-GCM tag: 16 bytes                  │
└─────────────────────────────────────────────┘

Key Technical Decisions

Aspect Decision Rationale
Approach Hybrid (classical + PQ) Defense in depth, mitigates algorithm risk
Classical KEM X25519 Fast, well-understood, cryptography library
Post-Quantum KEM ML-KEM-768 (FIPS 203) NIST Level 3, balanced size/security
Symmetric Cipher AES-256-GCM Authenticated encryption, quantum-safe
Key Derivation HKDF-SHA256 RFC 5869, combines secrets securely
Library liboqs-python OQS consortium, reference implementation

Why ML-KEM-768 (NIST Level 3)?

Level Security Public Key Ciphertext Decision
ML-KEM-512 NIST Level 1 (128-bit) 800 B 768 B Too small
ML-KEM-768 NIST Level 3 (192-bit) 1,184 B 1,088 B Selected
ML-KEM-1024 NIST Level 5 (256-bit) 1,568 B 1,568 B Overkill

Level 3 exceeds current threat models and pairs well with AES-256.

Encryption Flow

Encrypt(plaintext, recipient_public_key):
1. Generate ephemeral X25519 keypair
2. X25519 DH → classical_secret (32 bytes)
3. ML-KEM encapsulate → (ciphertext, pq_secret) (32 bytes)
4. combined_secret = HKDF(classical_secret || pq_secret, info="AEGIS-HybridKEM-v1")
5. AES-256-GCM encrypt with random 12-byte nonce
6. Return HybridEncryptedBlob

Decrypt(blob, recipient_private_key):
1. X25519 DH with ephemeral key → classical_secret
2. ML-KEM decapsulate → pq_secret
3. combined_secret = HKDF(classical_secret || pq_secret, info="AEGIS-HybridKEM-v1")
4. AES-256-GCM decrypt and verify tag
5. Return plaintext

Implementation

Phase 1: Core Primitives (2025-12-28)

Files Created

File Purpose
src/crypto/mlkem.py ML-KEM-768 wrapper using liboqs-python
src/crypto/hybrid_kem.py HybridKEMProvider implementation
tests/crypto/test_mlkem.py ML-KEM unit tests (30 tests)
tests/crypto/test_hybrid_kem.py Hybrid KEM provider tests (40 tests)

Files Modified

File Change
src/crypto/__init__.py Export MLKEM_AVAILABLE, HYBRID_KEM_AVAILABLE, get_hybrid_kem_provider()

Phase 2: Key Store & PII Encryption (2025-12-28)

Files Created

File Purpose
src/crypto/kek_provider.py KEK provider abstraction (EnvironmentKEKProvider, InMemoryKEKProvider)
src/workflows/persistence/key_store.py KeyStoreRepository with hash-chained audit trail
src/telemetry/encryption.py PIIEncryptionEnricher, DEKCache, DEKRotator
src/telemetry/decryption.py PIIDecryptor with integrity verification
scripts/generate_master_kek.py KEK bootstrap script for deployment
tests/crypto/test_kek_provider.py KEK provider tests (27 tests)
tests/telemetry/test_pii_encryption.py PII encryption/decryption tests (31 tests)

Files Modified

File Change
src/workflows/persistence/models.py Added GovernanceKey, KeyUsageAudit ORM models
src/workflows/persistence/__init__.py Export new models
src/workflows/override.py Added key_store param, sign_with_stored_key()
src/telemetry/pipeline.py Added PII encryption stage
src/telemetry/__init__.py Export encryption/decryption classes

Provider Usage

# src/crypto/__init__.py
def get_hybrid_kem_provider() -> HybridKEMProvider:
    """
    Get the hybrid post-quantum encryption provider.

    Returns HybridKEMProvider for X25519 + ML-KEM-768 + AES-256-GCM encryption.
    """
    if HYBRID_KEM_AVAILABLE and HybridKEMProvider is not None:
        return HybridKEMProvider()
    else:
        raise RuntimeError(
            "HybridKEMProvider not available. "
            "Install liboqs-python>=0.10.0 for post-quantum support."
        )

Example Usage

from crypto import get_hybrid_kem_provider

provider = get_hybrid_kem_provider()

# Generate keypair
keypair = provider.generate_keypair()

# Encrypt sensitive data
plaintext = b"governance private key material"
blob = provider.encrypt(plaintext, keypair.public)

# Decrypt
decrypted = provider.decrypt(blob, keypair.private)
assert decrypted == plaintext

Consequences

Positive

  1. Quantum Resistance: Protection against future CRQCs
  2. Defense in Depth: If one algorithm fails, the other provides security
  3. NIST Compliance: Uses standardized FIPS 203 algorithm
  4. Synergy with GAP-Q1: Shares liboqs infrastructure with post-quantum signatures
  5. Graceful Degradation: System works without liboqs installed

Negative

  1. Key Size Increase: 1,216 byte public keys vs 32 bytes (38x larger)
  2. Ciphertext Overhead: 1,148 byte overhead per encrypted blob
  3. Optional Dependency: Requires liboqs-python (pip) and liboqs (native)
  4. Encryption Time: ~0.2ms vs ~0.05ms (4x slower)

Neutral

  1. AES-256 Remains Core: Only key exchange is quantum-vulnerable

Security Properties

Property Status Evidence
Defense in depth Both X25519 and ML-KEM must succeed
Harvest-now-decrypt-later ML-KEM-768 quantum resistant
Authenticated encryption AES-256-GCM with 16-byte tag
Key derivation HKDF-SHA256 per RFC 5869
Algorithm agility Provider pattern enables swaps

Test Coverage

Phase 1 (Core Primitives):
  tests/crypto/test_mlkem.py: 30 tests
  tests/crypto/test_hybrid_kem.py: 40 tests

Phase 2 (Key Store & PII):
  tests/crypto/test_kek_provider.py: 27 tests
  tests/telemetry/test_pii_encryption.py: 31 tests
  tests/workflows/persistence/test_key_store.py: 52 tests

Coverage Expansion (2025-12-29):
  tests/test_override_coverage.py: 99 tests (override.py: 61.27%→95.19%)
  tests/test_persistence_coverage.py: 36 tests (durable.py: 73.49%→100%)
  tests/telemetry/test_coverage.py: 64 tests (decryption.py: 74.10%→94.58%)

Total: 846 tests | 93.60% overall coverage
KeyStoreRepository: 96.52% coverage
All quality gates pass (mypy --strict, ruff, bandit)

Phase 2: Key Store & PII Encryption

KEK Provider Usage

from crypto.kek_provider import get_kek_provider, InMemoryKEKProvider

# Auto-detect: uses environment if AEGIS_MASTER_KEK_PUBLIC/PRIVATE set
provider = get_kek_provider("auto")

# For testing
provider = InMemoryKEKProvider()

# Encrypt/decrypt
encrypted = provider.encrypt(b"sensitive data")
decrypted = provider.decrypt(encrypted)

KeyStoreRepository Usage

from workflows.persistence import KeyStoreRepository

repo = KeyStoreRepository(session, kek_provider)

# Store governance key (private key encrypted with KEK)
key_id = await repo.store_key(
    role="risk_lead",
    algorithm="bip322-simple",
    private_key=private_key_bytes,
    public_key=public_key_bytes,
    created_by="system"
)

# Retrieve for signing (decrypted)
private_key = await repo.get_private_key("risk_lead", "bip322-simple")

# Audit trail
await repo.record_usage("risk_lead", "bip322-simple", "sign", actor_id, context)
audit_log = await repo.get_audit_trail("risk_lead", "bip322-simple")

PII Encryption Usage

from telemetry.encryption import PIIEncryptionEnricher, PIIManifest, PIIPriority
from telemetry.decryption import PIIDecryptor

# Create encryptor
encryptor = PIIEncryptionEnricher(
    public_key=kek_provider.get_public_key(),
    dek_version="dek-v1-20251228",
    min_priority=PIIPriority.HIGH  # Encrypt HIGH and CRITICAL fields
)

# Encrypt PII in telemetry event
event = {
    "event_id": "evt-123",
    "actor_id": "user@acme-corp.test",  # HIGH priority - encrypted
    "decision_actor": "lead@acme-corp.test",  # CRITICAL - encrypted
    "proposal_id": "prop-001"  # Not PII - remains plaintext
}
encrypted_event = encryptor.enrich(event)

# Decrypt
decryptor = PIIDecryptor(
    private_key=kek_provider.get_private_key(),
    verify_integrity=True
)
decrypted_event = decryptor.decrypt_event(encrypted_event)

Override Workflow with Key Store

from workflows.override import OverrideWorkflow
from workflows.persistence import KeyStoreRepository

workflow = OverrideWorkflow(proposal_id="prop-001", reason="Critical fix")
workflow.set_key_store(key_store_repo)

# Sign with stored key (retrieves, signs, records audit)
success, message = await workflow.sign_with_stored_key(
    role="risk_lead",
    actor_id="alice@acme-corp.test"
)

PII Fields Encrypted

Priority Fields
CRITICAL actor_assignments, override_justification, risk_lead_signature, security_lead_signature, decision_actor, decision_rationale
HIGH workflow_id, author_id, actor_id, combined_hash
MEDIUM requested_by, approved_by

Dependencies

Package Version Purpose
liboqs-python >=0.10.0 NIST PQ algorithm bindings
liboqs (native) 0.14.0 OQS library (brew or source)
cryptography >=41.0.0 X25519, HKDF, AES-GCM

References

External Standards

Internal Documents

Changelog

Date Change
2025-12-29 Zero-defect deployment: 846 tests, 93.60% coverage; 261 new tests added across override, persistence, telemetry modules
2025-12-29 Phase 2 complete: KeyStoreRepository tests added (52 tests), 96.52% coverage
2025-12-29 Phase 2 verification: All 533 tests passing, quality gates clean
2025-12-28 Phase 2: Key store integration, PII encryption, override workflow integration
2025-12-28 ADR created, Phase 1 implementation complete