Trust distribution

AWS Private Certificate Authority Amazon S3 AWS Systems Manager State Manager

Issuing a Private CA leaf is easy. Making a Linux fleet trust your CA — safely and repeatedly — is the operational work this lab centres on.

On this page


What you are distributing

Only the CA certificate (trust anchor). Never the CA private key, and never the server leaf private key, onto clients.

One source · Private CA (ROOT) · two materials, two destinations

Trust path → clients · CA certificate (PEM) only
CA PEM public anchor
→
S3 object trust/ca.pem
→
SSM assoc tag TrustCA=intranet
→
Managed client system trust store
Leaf path → edge (default alb_acm) · ACM holds the key
Leaf via ACM private cert
→
ACM private certificate
→
internal ALB HTTPS :443
→
Clients never receive the leaf key

Optional nginx_export: leaf + key → Secrets Manager → State Manager → nginx on private-web.

One CA, two materials with opposite rules — the trust anchor fans out to the fleet; the private key stays in ACM (default) or on the server (export mode).
Material Destination Clients
CA PEM S3 → SSM → system trust store Managed hosts only
Leaf (alb_acm, default) ACM → internal ALB Not on clients
Leaf + key (nginx_export) Secrets Manager → State Manager → private-web Not on clients

Client trust is the same for both server modes. ACM holding a private cert does not install the CA on clients. An ALB trust store is for mTLS (ALB verifying client certs) — the opposite direction.

That split is also drawn on Architecture.

Pipeline

Pipeline · issue → publish → target → distribute → verify → rotate
1 · Issue Private CA anchor
→
2 · Publish PEM + script → S3
→
3 · Target tag TrustCA=intranet
→
4 · Distribute SSM desired state
→
5 · Verify curl / openssl, no --cacert
→
6 · Rotate overlap old + new CA

The lab wires this end to end: s3://…/trust/ca.pem plus install-ca-trust.sh. The SSM document copies the script, then runs it with the CA S3 URI; the association targets tag TrustCA=intranet.

Six ordered steps from issuing the anchor to rotating it — State Manager keeps the install as desired state, not a one-shot push.
  1. Issue — Private CA becomes the trust anchor.
  2. Publish — Put the CA PEM (and the install script) in a hardened S3 bucket.
  3. Target — Tag hosts that should trust it (TrustCA=intranet).
  4. Distribute — SSM State Manager runs the install as desired state.
  5. Verify — curl / OpenSSL against the private hostname without --cacert.
  6. Rotate — Overlap old and new CA PEMs before removing the old anchor.

Do and don’t

Do CA PEM, system store, tags

Private, encrypted, versioned S3 (account Block Public Access). Least-privilege s3:GetObject on that object. Install into the system trust store. Idempotent script. Separate leaf renewal from CA trust.

Don't Bypass or hand-paste

No SSH PEM pastes. No forever CA baked into an AMI with no update path. No CA or leaf private keys on clients. Do not call the managed path “proven” with curl --cacert — that skips the trust store.

Success for the managed client is TLS that works because the CA is in the system store. --cacert proves the PEM file, not fleet trust.

Lab mechanism

Terraform always wires CA trust on clients (stage 04). Server leaf handling depends on private_tls_mode.

Default leaf: ACM on internal ALB (alb_acm)

Piece What it does
ACM private certificate Issued from the lab Private CA; ARN on the internal ALB HTTPS listener
Internal ALB VPC-only HTTPS; HTTP target group → private-web
private-web HTTP nginx only — no leaf material on the instance

Optional leaf: nginx export (nginx_export)

State Manager association on LabRole=private-web (not used in alb_acm mode). No EC2 user data for certificate material.

Piece What it does
Secrets Manager JSON leaf (certificate, chain, private_key)
scripts/configure-private-web.sh Idempotent: install nginx, pull secret, write /etc/nginx/ssl/, restart
SSM document Decode script, set env, run configure
SSM association Targets tag LabRole=private-web; LeafVersion forces re-apply on rotation
private-web Instance profile may secretsmanager:GetSecretValue on that secret only

CA trust on managed clients (stage 04)

Piece What it does
S3 bucket SSE-S3, versioning, deny insecure transport (account BPA)
trust/ca.pem Root CA certificate from ACM Private CA
trust/install-ca-trust.sh Idempotent install script (same file as scripts/install-ca-trust.sh)
SSM document aws s3 cp the script, then run it with the CA S3 URI
SSM association Targets tag TrustCA=intranet; schedule_expression = rate(30 minutes)
client-managed Tagged TrustCA=intranet; instance profile may s3:GetObject on the trust prefix
client-unmanaged Same AMI, VPC, and SG; only difference is no trust tag — association never targets it

When vs what: State Manager owns the cadence (create, tag change, and the 30-minute schedule). The script does not sleep or cron — it only installs when the PEM SHA-256 changed.

Install script behaviour

scripts/install-ca-trust.sh:

  1. Fetch PEM from s3://… (or a local path).
  2. Require a BEGIN CERTIFICATE block.
  3. Compare SHA-256 to /etc/pki/ca-trust/source/anchors/intranet-private-ca.pem.
  4. If unchanged, exit 0; otherwise install and run update-ca-trust extract.

Prove contrast

Client System trust Private HTTPS
Managed CA installed by SSM Succeeds without --cacert
Unmanaged Default Amazon Linux store Fails (untrusted issuer)

Fair contrast: same AMI and security group; only the TrustCA tag differs. Run both from the walkthrough prove scripts.

Distro commands

This lab uses Amazon Linux 2023. Same idea elsewhere:

Family Anchor location Refresh
Amazon Linux / RHEL /etc/pki/ca-trust/source/anchors/ update-ca-trust extract
Debian / Ubuntu /usr/local/share/ca-certificates/*.crt update-ca-certificates

Rotation (overlap)

  1. Publish the new CA PEM (new object version, or a second anchor file for overlap).
  2. Wait for the scheduled association (or force a re-run) — hosts with a changed SHA-256 install the new anchor; unchanged hosts no-op.
  3. Re-issue leaves from the new CA; cut traffic over.
  4. Remove the old CA from anchors after the overlap window.

Leaf renewal (new server cert, same CA) is a different pipeline from CA replacement. Keep them separate so a leaf rollover does not require touching every client.

Where next

Topic Page
Topology and host roles Architecture
Apply and prove Walkthrough
When to use Private CA Usage guide

Next: Walkthrough.


Back to top

Lab repository. Tear down the Private CA when finished — it bills monthly.

ACM + Private CA lab docs