Production Hardening

A remote signer holds your validator private keys. Every layer of your deployment should reflect that responsibility. This guide covers practical hardening steps you can apply today.

Network Exposure

Your signing port should never be reachable from the public internet. Bind it to a private interface and restrict access at the firewall level.

server:
  listen_address: "127.0.0.1"  # or your private network interface
  listen_port: 9000

metrics:
  listen_address: "127.0.0.1"
  listen_port: 3000

Firewall rules to consider:

Port 9000 (signing API): allow only your validator client IPs
Port 3000 (metrics): allow only your monitoring system (Prometheus, Grafana, etc.)
Block all other inbound traffic to these ports

Token Security

If you’re running multiple clients against one signer, configure auth policies with per-client tokens.

Key rules for tokens:

Minimum 16 characters, alphanumeric and dashes only
Load tokens from environment variables using the env: prefix. Never hardcode them in your config file.
Tokens are hashed with HMAC-SHA256 at startup and never stored in plaintext

auth_policies:
  vc1:
    token: "env:VC1_TOKEN"  # resolved from $VC1_TOKEN at startup
    allowed_scopes:
      - sign
      - public_keys

See the full Auth Policies guide for policy configuration, scope restrictions, and client integration.

File Permissions

Tight file permissions prevent other users on the system from reading your keys or config.

# Config file: owner read/write only
sudo chmod 600 /etc/containment-chamber/config.yaml

# Keystores directory: owner only
sudo chmod 700 /var/lib/containment-chamber/keystores

# Individual keystore files
sudo chmod 600 /var/lib/containment-chamber/keystores/*.json

# Ensure correct ownership
sudo chown -R containment-chamber:containment-chamber \
  /etc/containment-chamber \
  /var/lib/containment-chamber

The SQLite slashing protection database is created with 0600 permissions automatically.

systemd Hardening

On Linux with systemd, the service unit can enforce additional isolation. These directives are already included in the bare metal guide:

[Service]
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadOnlyPaths=/
ReadWritePaths=/var/lib/containment-chamber

This prevents the process from gaining new privileges, restricts filesystem access to what it actually needs, and isolates its /tmp.

Docker Security

When running in Docker, apply the principle of least privilege:

docker run \
  --user 1000:1000 \
  --read-only \
  --tmpfs /tmp \
  --cap-drop ALL \
  -v ./config.yaml:/config.yaml:ro \
  -v ./keystores:/keystores:ro \
  -v ./data:/data \
  ghcr.io/unforeseen-consequences/containment-chamber:latest \
  -c /config.yaml

What each flag does:

--user 1000:1000 runs as a non-root user
--read-only makes the container filesystem immutable
--tmpfs /tmp provides a writable scratch space
--cap-drop ALL removes all Linux capabilities (none are needed)
:ro mounts config and keystores as read-only

Seccomp Syscall Filter

On Linux, you can restrict the signer to a minimal syscall allowlist using the kernel’s seccomp BPF filter. This limits what a code execution vulnerability can do — even if an attacker achieves arbitrary code execution, they can’t call execve, ptrace, or other dangerous syscalls.

server:
  seccomp: true  # opt-in, Linux only. Default: false

If the filter fails to apply (e.g., the kernel doesn’t support it or the process lacks CAP_SYS_ADMIN), the signer logs a warning and continues without the filter rather than refusing to start.

Canary Keys

Canary keys are designated validator public keys that should never sign in normal operation. When a canary key signs, the signer logs a warning and increments the containment_canary_signing_total metric. Signing proceeds normally — canary keys don’t block requests.

canary_keys:
  - "0x1234..."
  - "0x5678..."

Use canary keys to detect unauthorized access. If an attacker can submit signing requests, they’ll likely try to sign with whatever keys are loaded. A canary key that suddenly appears in your metrics is a strong signal that something is wrong.

Audit Logging

All security-relevant events are logged with target: "audit". This lets you route audit events to a separate sink — a SIEM, a write-once log store, or a separate file — without changing the rest of your logging configuration.

Events logged to the audit target include:

State transitions — seal machine state changes (e.g., Sealed → AwaitingUnseal → Unsealed)
Signing requests — every signing attempt, including the key and operation type
Unseal share submissions — when an operator submits a share, including the share index
Seal operations — when the signer is sealed, and by whom

To capture audit events separately, configure your tracing subscriber to route the audit target:

# Include audit events at info level alongside normal logs
RUST_LOG=containment_chamber=info,audit=info

# Audit-only (suppress all other logs)
RUST_LOG=off,audit=info

In production, pipe JSON logs to a log aggregator and filter on "target":"audit" to build an audit trail.

Built-in Protections

Containment Chamber includes several protections that activate automatically:

Memory zeroization: private keys are zeroed from memory when they’re no longer needed
Core dump protection (Linux): the process marks itself as non-dumpable at startup, preventing key material from leaking into core dumps
Token hashing: authentication tokens are HMAC-SHA256 hashed at startup and never held in plaintext
Constant-time comparison: token validation uses constant-time comparison to prevent timing attacks

These require no configuration. They’re always on.

Checklist

A quick reference for production deployments: