Key Rotation and Compromise Response Runbook

Purpose: - Standardize rotation and emergency response for JWT/JWKS, terminal token signing, and provisioning control-key material.

Scope

1. Keycloak/JWKS signing keys
2. Terminal token signing/encryption material
3. Provisioning control keys (KMS/env/file sources)
4. Envelope encryption keys (ENVELOPE_KEY_ID/ENVELOPE_KEY_B64)

Rotation Cadence (Baseline)

• JWKS signing keys: every 90 days or provider policy, whichever is stricter.
• Terminal token/envelope keys: every 90 days.
• Provisioning control keys: every 60-90 days with overlap window.

Planned Rotation Procedure

1. Prepare new key material and register with provider/secret manager.
2. Deploy overlap configuration:
3. keep old + new keys active.
4. set preferred key id to new value.
5. Validate:
6. auth/token verification succeeds on new key.
7. terminal token mint/consume works.
8. provisioning SSH paths work with new control key id.
9. Remove old key after stable window and verification checks.
10. Record execution evidence in doc/operations/evidence/secrets_key_ops.md.

Emergency Compromise Response

1. Declare incident severity (SEV-1 for active compromise risk).
2. Rotate compromised key immediately and revoke impacted sessions/tokens.
3. Execute JWKS force refresh endpoint per break-glass runbook:
4. POST /internal/auth/jwks/refresh
5. Confirm deny-list and revocation paths are active for admin tokens.
6. Validate service health and auth/provisioning/terminal flows.
7. Publish stakeholder update + timeline and complete post-incident actions.

Verification Checklist

• All API instances refreshed/rotated and healthy
• Auth rejects compromised/retired key material
• Terminal and provisioning key paths operational after rotation
• Evidence captured with timestamps and owners

References: - doc/operations/runbooks/JWKS_Compromise_Breakglass_Runbook.md - doc/operations/KMS_Control_Key_Source_Guardrails.md - doc/operations/evidence/secrets_key_ops.md