DynamoDB + KMS Key Management

The DynamoDB key source stores BLS validator private keys in AWS DynamoDB, encrypted with a master key that is itself protected by AWS KMS using Shamir’s Secret Sharing. No single KMS key can decrypt the master key alone — you configure a threshold of M-of-N KMS keys that must cooperate to reconstruct it.

This is the right choice if you run validators in AWS and want durable, cloud-native key storage with hardware-backed encryption, multi-account custody, and automatic key refresh without signer restarts.

Overview

Use DynamoDB when:

• You run validators in AWS and want keys in a managed, durable service
• You need multi-account key custody (e.g. 2-of-3 Shamir across separate AWS accounts)
• You want to generate new validator keys without ever touching a filesystem
• You need to import existing keystores and have them encrypted at rest by KMS
• You want automatic key refresh without restarting the signer

Stick with filesystem when:

• You already have a working filesystem setup and don’t need cloud-native key management
• You run outside AWS or want to avoid AWS dependencies
• You need the simplest possible deployment

The two sources are additive. You can run filesystem and dynamodb simultaneously, loading keys from both.

Architecture

Each validator private key is encrypted with AES-256-GCM using a 256-bit master key. The master key itself is never stored in plaintext — it’s protected by Shamir secret sharing across multiple AWS KMS keys.

Master Key Lifecycle

On first boot, the signer generates a random master key, splits it using Shamir’s secret sharing, encrypts each share with a different KMS key, and stores the ciphertexts in DynamoDB. On subsequent boots, it fetches and decrypts enough shares to reconstruct the key.

Caution

If fewer than the configured KMS threshold of keys are accessible, the signer cannot reconstruct the master key and will refuse to unseal. Keep your age backup mnemonics in offline cold storage.

Key Operations

Validator keys can be loaded from DynamoDB at startup, imported via the Chamber API, or generated with BIP-39 mnemonics. All keys are encrypted with AES-256-GCM using the master key before storage.

Configuration

All key sources live under the key_sources section. The DynamoDB source can run alongside the filesystem source.

Note

KMS key ARNs and the Shamir threshold are no longer set in the config file. They’re provided once during the init ceremony via the API and stored in DynamoDB. See Seal & Unseal for the init ceremony procedure.

key_sources:
  # Filesystem key source (optional, runs alongside DynamoDB)
  filesystem:
    paths:
      - /data/keystores/raw
      - /data/keystores/encrypted
    keystore_load_concurrency: 128

  # DynamoDB key source
  dynamodb:
    table: containment-keys
    key_refresh_interval_minutes: 10
    unseal_timeout_minutes: 30
    keygen:
      enabled: false
      max_items_per_request: 100
    writable:
      enabled: false
      max_concurrent_writes: 16
      request_timeout_seconds: 600

Field reference

Field	Type	Default	Description
dynamodb.table	string	required	DynamoDB table name
dynamodb.key_refresh_interval_minutes	integer	10	How often to reload keys from DynamoDB in the background
dynamodb.unseal_timeout_minutes	integer	30	How long a partial unseal remains valid before expiring (0 = no timeout)
dynamodb.max_concurrent_reads	integer	16	Parallel DynamoDB read workers for key loading
dynamodb.status_filter	list	["active"]	Only load keys with these statuses
dynamodb.keygen.enabled	boolean	false	Enable POST /api/v1/chamber/keys/generate
dynamodb.keygen.max_items_per_request	integer	100	Max keys per keygen request
dynamodb.keygen.max_concurrent_keygen	integer	16	Parallel keygen workers
dynamodb.keygen.request_timeout_seconds	integer	600	Keygen request timeout
dynamodb.writable.enabled	boolean	false	Enable POST /api/v1/chamber/keys
dynamodb.writable.max_concurrent_writes	integer	16	Parallel keystore decryption workers
dynamodb.writable.request_timeout_seconds	integer	600	Import request timeout
dynamodb.writable.max_items_per_request	integer	100	Max keys per import request
dynamodb.keygen.backup.enabled	bool	false	Enable age-encrypted mnemonic backup during key generation
dynamodb.keygen.backup.recipients	list	—	age public keys for mnemonic backup encryption

For IAM setup, see AWS IAM Permissions.

Key Generation

The keygen endpoint generates fresh BLS validator keys, stores them in DynamoDB as pending, and returns deposit data ready for submission to the Ethereum deposit contract. Keys are derived via BIP-39 mnemonic and EIP-2333 path.

Enable it in config:

key_sources:
  dynamodb:
    table: containment-keys
    keygen:
      enabled: true
      max_items_per_request: 100

Then generate keys:

curl -s -X POST http://localhost:9000/api/v1/chamber/keys/generate \
  -H "Content-Type: application/json" \
  -u "x:your-key-manager-token" \
  -d '{
    "count": 5,
    "withdrawal_credentials": "0x020000000000000000000000abcdef1234567890abcdef1234567890abcdef12",
    "amount": 32000000000
  }'

Response:

{
  "keys": [
    {
      "pubkey": "0xabc123...",
      "withdrawal_credentials": "0x020000000000000000000000abcdef...",
      "amount": 32000000000,
      "signature": "0xdef456...",
      "deposit_message_root": "0x789abc...",
      "deposit_data_root": "0x012345...",
      "fork_version": "00000000",
      "network_name": "mainnet"
    }
  ]
}

Note

withdrawal_credentials must start with 0x02 (Pectra withdrawal prefix), with bytes 1-11 set to zero. The amount must be between 32,000,000,000 and 2,048,000,000,000 gwei.

Pending to active workflow: Generated keys start as pending so you can submit deposit data and wait for the validator to appear on-chain before activating signing. Once the validator is active on the beacon chain, update the key’s status to active in DynamoDB directly, or restart the signer with the key already active.

The deposit data signature is self-verified before the key is stored. If verification fails, the request returns an error and nothing is stored.

Key Import

The import endpoint accepts existing EIP-2335 keystores or raw hex keys, decrypts them, and re-encrypts them with the DynamoDB master key. Use this to migrate keys from a filesystem setup or another signer.

Enable it in config:

key_sources:
  dynamodb:
    table: containment-keys
    writable:
      enabled: true
      max_concurrent_writes: 16
      request_timeout_seconds: 600

EIP-2335 Keystores

Import encrypted keystores with their passwords:

curl -s -X POST http://localhost:9000/api/v1/chamber/keys \
  -H "Content-Type: application/json" \
  -u "x:your-key-manager-token" \
  -d '{
    "keystores": [
      "{\"crypto\":{\"kdf\":{\"function\":\"pbkdf2\",...},...}}"
    ],
    "passwords": [
      "my-keystore-password"
    ],
    "status": "active"
  }'

Raw Hex Keys

Import raw BLS private keys directly:

curl -s -X POST http://localhost:9000/api/v1/chamber/keys \
  -H "Content-Type: application/json" \
  -u "x:your-key-manager-token" \
  -d '{
    "raw_keys": [
      "5d79146c27a33cea86a97080f377c2f5eef0e3fdbdc4436b82f8d23abc7a373c"
    ],
    "status": "active"
  }'

Response:

{
  "data": [
    { "status": "imported", "pubkey": "0xabc123..." },
    { "status": "duplicate", "pubkey": "0xdef456...", "message": "key already exists in DynamoDB" },
    { "status": "error", "pubkey": "0x789abc...", "message": "wrong password" }
  ],
  "summary": {
    "imported": 1,
    "duplicate": 1,
    "error": 1,
    "total": 3
  }
}

The storage.status field controls whether imported keys are immediately active or held as inactive. Use "inactive" if you want to stage keys before activating them via PATCH /api/v1/chamber/keys.

Disaster Recovery

age-encrypted mnemonic backup

When configured, the signer encrypts each generated key’s BIP-39 mnemonic with an age public key before storing it in DynamoDB. This gives you an offline recovery path that doesn’t depend on AWS at all.

key_sources:
  dynamodb:
    table: containment-keys
    keygen:
      enabled: true
      backup:
        enabled: true
        recipients:
          - "age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p"
          - "age1second..."  # optional — multiple recipients supported

The encrypted mnemonic is stored alongside the key in DynamoDB. To recover a key:

# Fetch the encrypted mnemonic from DynamoDB and decrypt it
aws dynamodb get-item \
  --table-name containment-keys \
  --key '{"pubkey": {"S": "0xabc123..."}}' \
  --query 'Item.encrypted_mnemonic.S' \
  --output text | age --decrypt -i ~/.age/key.txt

This gives you the BIP-39 mnemonic. From it, you can re-derive the validator key using standard EIP-2333 derivation and import it into any compatible signer.

Recovery without AWS

If KMS becomes unavailable and you have the age-encrypted mnemonics:

1. Decrypt each mnemonic with your age private key (stored offline)
2. Re-derive the BLS key from the mnemonic using EIP-2333 path m/12381/3600/0/0/0
3. Import the raw key into a new signer instance (filesystem or a new DynamoDB table)

Tip

Keep your age private key in offline cold storage: a hardware security module, paper backup in a safe, or a geographically separate secure location. It’s your last line of defense if all KMS keys become inaccessible.

What happens when KMS is lost

If fewer than the configured threshold of KMS keys are accessible, the signer cannot reconstruct the master key and will refuse to unseal. It logs a clear error indicating which KMS keys failed. If you cannot restore access to enough KMS keys, use the age backup to recover the mnemonics and re-import the keys.

Multi-Account KMS

For maximum security, split the master key across KMS keys in separate AWS accounts. A 2-of-3 setup means any two accounts can reconstruct the master key, but no single account can compromise it alone.

Reference architecture

Account A (primary, runs signer)
  KMS key: arn:aws:kms:us-east-1:111111111111:key/aaa...
  DynamoDB table: containment-keys
  IAM role: containment-chamber-signer

Account B (secondary custody)
  KMS key: arn:aws:kms:us-east-1:222222222222:key/bbb...
  Cross-account grant: allows Account A signer role to Decrypt

Account C (tertiary custody)
  KMS key: arn:aws:kms:us-east-1:333333333333:key/ccc...
  Cross-account grant: allows Account A signer role to Decrypt

Config:

key_sources:
  dynamodb:
    table: containment-keys
    key_refresh_interval_minutes: 10

The KMS key ARNs and the 2-of-3 threshold are provided during the init ceremony via the API — not in the config file. See Seal & Unseal for the init ceremony procedure.

The signer needs kms:Decrypt and kms:DescribeKey on all three keys, and kms:Encrypt on at least one (for the init ceremony). Cross-account grants are set up in the secondary and tertiary accounts to allow the primary signer role.

Ready-to-use Terraform examples are in terraform/examples/single-account/ (all resources in one account) and terraform/examples/multi-account/ (2-of-3 Shamir across accounts).

Troubleshooting

KMS access denied

Error: failed to decrypt KMS share: AccessDeniedException

The signer’s IAM role lacks kms:Decrypt on one or more KMS keys. Check:

1. The IAM role attached to the instance or pod has the correct policy
2. The KMS key policy allows the role’s ARN
3. For cross-account keys, the key policy in the remote account grants access to the primary role

# Test KMS access manually
aws kms describe-key --key-id arn:aws:kms:us-east-1:111111111111:key/aaa...

See AWS IAM Permissions for the full policy reference.

Insufficient shares

Error: only 1 of 2 required KMS shares could be decrypted

The signer needs at least as many KMS keys as the threshold configured during the init ceremony. Check:

1. Network connectivity to the KMS endpoints in each region
2. IAM permissions for each key
3. Whether any KMS keys have been disabled or scheduled for deletion

If you can’t restore access to enough KMS keys, use the age backup to recover the mnemonics and re-import the keys.

DynamoDB throttling

Error: ProvisionedThroughputExceededException

The DynamoDB table is throttled. Options:

• Switch to PAY_PER_REQUEST billing mode (recommended for variable workloads)
• Increase provisioned capacity
• Increase key_refresh_interval_minutes to spread reads over time

HMAC mismatch

Error: HMAC verification failed for key 0xabc123...

The key’s integrity check failed. This means either the master key changed (different KMS shares reconstructed a different master key) or the stored key was tampered with. Don’t use this key for signing.

Check that the KMS keys accessible to the signer match those used during the init ceremony. If you rotated KMS keys without re-encrypting the Shamir shares, the master key reconstruction will produce a different result.