Trust distribution
Issuing a Private CA leaf is easy. Making a Linux fleet trust your CA — safely and repeatedly — is the operational work this lab centres on.
On this page
- What you are distributing
- Pipeline
- Do and don’t
- Lab mechanism
- Distro commands
- Rotation (overlap)
- Where next
What you are distributing
Only the CA certificate (trust anchor). Never the CA private key, and never the server leaf private key, onto clients.
One source · Private CA (ROOT) · two materials, two destinations
Optional nginx_export: leaf + key → Secrets Manager → State Manager → nginx on private-web.
| Material | Destination | Clients |
|---|---|---|
| CA PEM | S3 → SSM → system trust store | Managed hosts only |
Leaf (alb_acm, default) |
ACM → internal ALB | Not on clients |
Leaf + key (nginx_export) |
Secrets Manager → State Manager → private-web |
Not on clients |
Client trust is the same for both server modes. ACM holding a private cert does not install the CA on clients. An ALB trust store is for mTLS (ALB verifying client certs) — the opposite direction.
That split is also drawn on Architecture.
Pipeline
The lab wires this end to end: s3://…/trust/ca.pem plus
install-ca-trust.sh. The SSM document copies the script, then runs it
with the CA S3 URI; the association targets tag TrustCA=intranet.
- Issue — Private CA becomes the trust anchor.
- Publish — Put the CA PEM (and the install script) in a hardened S3 bucket.
- Target — Tag hosts that should trust it (
TrustCA=intranet). - Distribute — SSM State Manager runs the install as desired state.
- Verify —
curl/ OpenSSL against the private hostname without--cacert. - Rotate — Overlap old and new CA PEMs before removing the old anchor.
Do and don’t
Private, encrypted, versioned S3 (account Block Public Access). Least-privilege s3:GetObject on that object. Install into the system trust store. Idempotent script. Separate leaf renewal from CA trust.
No SSH PEM pastes. No forever CA baked into an AMI with no update path. No CA or leaf private keys on clients. Do not call the managed path “proven” with curl --cacert — that skips the trust store.
Success for the managed client is TLS that works because the CA is in the system store.
--cacertproves the PEM file, not fleet trust.
Lab mechanism
Terraform always wires CA trust on clients (stage 04). Server leaf handling
depends on private_tls_mode.
Default leaf: ACM on internal ALB (alb_acm)
| Piece | What it does |
|---|---|
| ACM private certificate | Issued from the lab Private CA; ARN on the internal ALB HTTPS listener |
| Internal ALB | VPC-only HTTPS; HTTP target group → private-web |
private-web |
HTTP nginx only — no leaf material on the instance |
Optional leaf: nginx export (nginx_export)
State Manager association on LabRole=private-web (not used in alb_acm mode).
No EC2 user data for certificate material.
| Piece | What it does |
|---|---|
| Secrets Manager | JSON leaf (certificate, chain, private_key) |
scripts/configure-private-web.sh |
Idempotent: install nginx, pull secret, write /etc/nginx/ssl/, restart |
| SSM document | Decode script, set env, run configure |
| SSM association | Targets tag LabRole=private-web; LeafVersion forces re-apply on rotation |
private-web |
Instance profile may secretsmanager:GetSecretValue on that secret only |
CA trust on managed clients (stage 04)
| Piece | What it does |
|---|---|
| S3 bucket | SSE-S3, versioning, deny insecure transport (account BPA) |
trust/ca.pem |
Root CA certificate from ACM Private CA |
trust/install-ca-trust.sh |
Idempotent install script (same file as scripts/install-ca-trust.sh) |
| SSM document | aws s3 cp the script, then run it with the CA S3 URI |
| SSM association | Targets tag TrustCA=intranet; schedule_expression = rate(30 minutes) |
client-managed |
Tagged TrustCA=intranet; instance profile may s3:GetObject on the trust prefix |
client-unmanaged |
Same AMI, VPC, and SG; only difference is no trust tag — association never targets it |
When vs what: State Manager owns the cadence (create, tag change, and the 30-minute schedule). The script does not sleep or cron — it only installs when the PEM SHA-256 changed.
Install script behaviour
scripts/install-ca-trust.sh:
- Fetch PEM from
s3://…(or a local path). - Require a
BEGIN CERTIFICATEblock. - Compare SHA-256 to
/etc/pki/ca-trust/source/anchors/intranet-private-ca.pem. - If unchanged, exit 0; otherwise install and run
update-ca-trust extract.
Prove contrast
| Client | System trust | Private HTTPS |
|---|---|---|
| Managed | CA installed by SSM | Succeeds without --cacert |
| Unmanaged | Default Amazon Linux store | Fails (untrusted issuer) |
Fair contrast: same AMI and security group; only the TrustCA tag differs.
Run both from the walkthrough prove scripts.
Distro commands
This lab uses Amazon Linux 2023. Same idea elsewhere:
| Family | Anchor location | Refresh |
|---|---|---|
| Amazon Linux / RHEL | /etc/pki/ca-trust/source/anchors/ |
update-ca-trust extract |
| Debian / Ubuntu | /usr/local/share/ca-certificates/*.crt |
update-ca-certificates |
Rotation (overlap)
- Publish the new CA PEM (new object version, or a second anchor file for overlap).
- Wait for the scheduled association (or force a re-run) — hosts with a changed SHA-256 install the new anchor; unchanged hosts no-op.
- Re-issue leaves from the new CA; cut traffic over.
- Remove the old CA from anchors after the overlap window.
Leaf renewal (new server cert, same CA) is a different pipeline from CA replacement. Keep them separate so a leaf rollover does not require touching every client.
Where next
| Topic | Page |
|---|---|
| Topology and host roles | Architecture |
| Apply and prove | Walkthrough |
| When to use Private CA | Usage guide |
Next: Walkthrough.