Skip to content

JWKS Compromise Break-Glass Runbook

Purpose: - Provide a concrete emergency procedure when JWT verification trust is suspected to be compromised.

Severity: - SEV-1 if active token forgery or compromised signing key is suspected.

Trigger Conditions

  • Keycloak/JWKS signing key compromise suspected or confirmed.
  • Unexpected token acceptance patterns indicate signature trust drift.
  • Security incident response requests immediate auth trust reset.

Immediate Actions

  1. Freeze high-risk admin operations if possible.
  2. Rotate signing keys at IdP (Keycloak) per security policy.
  3. Revoke known-compromised admin sessions via deny-list workflow.
  4. Force JWKS cache refresh on API instances.

JWKS Refresh Options

Preferred (when internal control path is available): - Invoke internal admin-controlled force-refresh operation that calls JWKSAuth.ForceRefresh on all API instances. - API exposes POST /internal/auth/jwks/refresh when INTERNAL_JWKS_REFRESH_TOKEN is configured. - Request auth: Authorization: Bearer <INTERNAL_JWKS_REFRESH_TOKEN>. - Execute once per API pod/instance (or through instance-targeted automation) to avoid stale caches.

Example: 

curl -sS -X POST \
  -H "Authorization: Bearer ${INTERNAL_JWKS_REFRESH_TOKEN}" \
  "http://<api-host>/internal/auth/jwks/refresh"

Current fallback (always available): - Perform rolling restart of API pods/processes so JWKS cache re-initializes from issuer. - Validate startup logs show successful JWKS bootstrap.

Verification

  • Confirm API rejects tokens signed by retired key versions.
  • Confirm valid fresh tokens continue to authenticate.
  • Confirm admin deny-list checks are functioning.
  • Record timestamps for key rotation, cache refresh/restart, and verification checks.

Evidence to Capture

  • Incident ticket ID and timeline.
  • Key rotation evidence from IdP/security team.
  • API restart/refresh execution logs.
  • Post-change auth verification results.

References: - packages/shared/middleware/auth.go (JWKSAuth.ForceRefresh) - doc/operations/evidence/secrets_key_ops.md - doc/operations/Scalability_Security_Watchlist.md