--- path: onboarding/devops-onboarding/secret-rotation.mdx title: Secret rotation sidebar_position: 7 slug: secret-rotation pagination_next: null pagination_prev: null description: >- How to automate secret rotation with Passwork: rotation workflow, CLI-based rotation scripts for databases and API keys, Python SDK rotation patterns, scheduling via cron, and handling downstream consumers. keywords: - Passwork - secret rotation - passwork-cli - Python - automation - rotation script - PostgreSQL - MySQL - API key - cron - scheduled rotation --- import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; ## Why rotate secrets | Risk | How rotation mitigates it | |------|--------------------------| | Credential leak | A leaked password becomes invalid after the next rotation cycle | | Long-lived tokens | Shorter lifetimes reduce the window of opportunity for an attacker | | Former employee access | Credentials rotated after offboarding are useless even if memorized | | Compliance requirements | PCI-DSS, SOC 2, and ISO 27001 mandate regular credential changes | --- ## Rotation workflow :::warning Order of operations **Always update the target system first, then save to Passwork.** If you save to Passwork first and the target system update fails, Passwork holds the new password but the system still uses the old one — your systems are out of sync. ::: ``` 1. Generate new secret ↓ 2. Apply to target system (database, API provider, etc.) ↓ 3. Verify the new credential works ↓ 4. Update Passwork (passwork-cli update / Python SDK) ↓ 5. Notify / update downstream consumers ``` --- ## Rotation via CLI ### PostgreSQL ```bash #!/bin/bash set -euo pipefail PASSWORK_ITEM_ID="" DB_USER="order_service" DB_HOST="pg.prod.internal" # 1. Generate a new password NEW_PASS=$(openssl rand -base64 32 | tr -d '=+/' | cut -c1-32) # 2. Apply to PostgreSQL psql -h "$DB_HOST" -U postgres -d postgres \ -c "ALTER ROLE ${DB_USER} WITH PASSWORD '${NEW_PASS}';" # 3. Update in Passwork passwork-cli update --password-id "$PASSWORK_ITEM_ID" --password "$NEW_PASS" echo "Rotated password for ${DB_USER}" ``` ### MySQL ```bash #!/bin/bash set -euo pipefail PASSWORK_ITEM_ID="" DB_USER="inventory_svc" NEW_PASS=$(openssl rand -base64 32 | tr -d '=+/' | cut -c1-32) mysql -h mysql.prod.internal -u root \ -p"$(passwork-cli get --password-id '')" \ -e "ALTER USER '${DB_USER}'@'%' IDENTIFIED BY '${NEW_PASS}';" passwork-cli update --password-id "$PASSWORK_ITEM_ID" --password "$NEW_PASS" echo "Rotated password for ${DB_USER}" ``` ### External API key ```bash #!/bin/bash set -euo pipefail PASSWORK_ITEM_ID="" OLD_KEY=$(passwork-cli get --password-id "$PASSWORK_ITEM_ID") # 1. Request new key from external service (provider-specific) NEW_KEY=$(curl -s -X POST "https://api.service.example.com/keys/rotate" \ -H "Authorization: Bearer ${OLD_KEY}" \ -H "Content-Type: application/json" | jq -r '.new_key') # 2. Verify the new key works curl -s "https://api.service.example.com/verify" \ -H "Authorization: Bearer ${NEW_KEY}" | grep -q '"status":"ok"' # 3. Save to Passwork passwork-cli update --password-id "$PASSWORK_ITEM_ID" --password "$NEW_KEY" echo "API key rotated" ``` --- ## Rotation via Python SDK The Python connector gives you full control over the rotation workflow — error handling, logging, transaction semantics, and downstream notifications. ```python #!/usr/bin/env python3 """PostgreSQL password rotation with full error handling.""" import os import secrets import sys import psycopg2 from passwork import Client from passwork.exceptions import PassworkResponseError def rotate_postgres_password(item_id: str, db_user: str) -> bool: client = Client( url=os.environ["PASSWORK_HOST"], token=os.environ["PASSWORK_TOKEN"], refresh_token=os.environ.get("PASSWORK_REFRESH_TOKEN"), ) # Fetch admin credentials from Passwork admin_item = client.get_password(item_id=os.environ["PG_ADMIN_ITEM_ID"]) new_password = secrets.token_urlsafe(32) try: # Apply to PostgreSQL first conn = psycopg2.connect( host=admin_item.fields["PG_HOST"], dbname="postgres", user=admin_item.login, password=admin_item.password, ) conn.autocommit = True with conn.cursor() as cur: cur.execute("ALTER ROLE %s WITH PASSWORD %s", (db_user, new_password)) conn.close() except psycopg2.Error as e: print(f"Database error — password NOT changed: {e}", file=sys.stderr) return False try: # Only update Passwork after the target system is confirmed updated item = client.get_password(item_id=item_id) item.password = new_password client.update_password(item) except PassworkResponseError as e: # Critical: DB password changed but Passwork not updated # Alert on-call immediately print( f"CRITICAL: DB password rotated but Passwork update failed! " f"API error {e.code}: {e.message}", file=sys.stderr, ) return False print(f"Password rotated for {db_user}") return True if __name__ == "__main__": success = rotate_postgres_password( item_id=os.environ["TARGET_ITEM_ID"], db_user=os.environ["PG_DB_USER"], ) sys.exit(0 if success else 1) ``` --- ## Scheduling rotation ### Cron (Linux) ```bash # /etc/cron.d/passwork-rotation # Rotate production DB passwords every Sunday at 03:00 0 3 * * 0 deploy /opt/scripts/rotate-postgres.sh \ >> /var/log/passwork-rotation.log 2>&1 ``` ### GitLab CI scheduled pipeline ```yaml # .gitlab-ci.yml rotate_secrets: stage: maintenance image: passwork/passwork-cli:latest script: - pip install passwork-python - python /scripts/rotate-all.py rules: - if: '$CI_PIPELINE_SOURCE == "schedule"' ``` Configure the schedule in **CI/CD → Schedules** (e.g. every Sunday at 03:00 UTC). --- ## Recommended rotation frequencies | Secret type | Frequency | |-------------|-----------| | Production database passwords | 30–90 days | | External API keys | 90 days or per vendor policy | | Service account API tokens (Passwork) | 30 days | | SSH keys | 6–12 months | | TLS certificates | Before expiry (automate via `certbot` or ACME) | --- ## Handling downstream consumers When a secret rotates, any service that uses it needs to reload the new value. Common patterns: **Environment variable injection at startup:** if services load secrets at startup via `passwork-cli exec`, they pick up the new value on the next restart. Combine with a rolling restart after rotation. **Sidecar refresh:** in Kubernetes, the sidecar pattern (see [CI/CD patterns](./cicd-patterns.mdx)) periodically refreshes the .env file. The app watches for file changes and reloads without restart. **Notification after rotation:** in the rotation script, after updating Passwork, send a notification to the relevant Slack channel or trigger a deployment pipeline that restarts the affected services. For the full rotation reference, see [Secret rotation practices](https://passwork.pro/tech-guides/secret-management/rotation/).