Secrets Management¶

All secrets are managed by OpenBao (self-hosted Vault fork), currently v2.2.0.

Last verified: 2026-08-22.

Accessing OpenBao¶

URL: https://openbao.heezy.info

Internal only. dnsmasq resolves it to 192.168.1.25 (SWAG), which terminates TLS and proxies to the pod. It never touches Cloudflare.

Prerequisite: DNS¶

Your machine has to be using a lab resolver. On the LAN, FortiGate DHCP hands out 192.168.1.27 (Pi-hole), which forwards everything to dnsmasq — that works with no configuration. If you are on a VPN or have set DNS by hand, point it at 192.168.1.27 or 192.168.1.29.

macOS: System Settings → Network → active interface → Details → DNS → add 192.168.1.27

Verify: dig openbao.heezy.info should return 192.168.1.25.

Logging in to the UI¶

Use Google OIDC. On the login screen pick OIDC, leave the role blank, and sign in with an allow-listed Google account. It is the only human login path that does not involve handling a long-lived credential.

The OIDC method is configured by heezy-k8s/apps/openbao/configure-google-oidc.sh. A daily Raft snapshot already contains the configuration, so a normal restore brings it back; the script only matters when rebuilding OpenBao from empty.

Do not put AppRole credentials in documentation

An earlier version of this page pasted a live role_id and secret_id for the github-heezy AppRole, which grants the ansible-readonly and heezy-backup policies. Those were valid at the time of writing and have been removed from this page. If they have not been rotated yet, rotate them. AppRole is machine auth; humans should use OIDC.

For a break-glass root token, read it out of the k8s secret rather than storing it anywhere:

kubectl get secret openbao-init-keys -n openbao -o jsonpath='{.data.root-token}' | base64 -d

That secret also holds the unseal keys. Treat any use of it as an incident, not as a workflow.

Architecture¶

Browser → https://openbao.heezy.info
  → dnsmasq: openbao.heezy.info → 192.168.1.25 (SWAG MetalLB VIP)
  → SWAG terminates TLS, proxies → openbao.openbao.svc.cluster.local:8200
  → OpenBao pod in openbao namespace

Workloads/CI → http://openbao.heezy.local:8200 → 192.168.1.32 (openbao-lb VIP)

K8s namespace: openbao, single StatefulSet pod openbao-0, Raft storage on a 5Gi Longhorn PVC
Services: openbao ClusterIP + openbao-lb LoadBalancer (192.168.1.32). The LB selector includes openbao-active=true, so the VIP follows the active pod
SWAG proxy conf: openbao.subdomain.conf in the swag-proxy-confs ConfigMap (heezy namespace)
Auto-unseal: CronJob openbao-unseal runs every 5 minutes, checks seal status and unseals

Use openbao.heezy.local:8200 from playbooks, workflows and pods — that is the VIP. nebula1.heezy.local:30820 is one node's NodePort and a single point of failure.

Secret Layout¶

KV engine: secret/ (KV v2). API reads go to secret/data/<path>, CLI reads to secret/<path>
Auth methods enabled: approle (machines), oidc (humans), kubernetes (in-cluster), token
Policies: admin, ansible-readonly, eso-readonly, heezy-backup, oauth-rotator, default, root, plus a stale openclaw policy left from the retired agent

Key Paths¶

Path	Contents
`all/heezy/aws/ecr-credentials`	AWS ECR access keys
`all/heezy/discord/webhooks`	Discord webhooks, including the `claude-code` one
`all/heezy/ubuntu/cloud-init-credentials`	Ubuntu VM default credentials
`production/heezy/github_runner/aws_credentials`	AWS creds the act-runner uses
`production/heezy/terraform/fortigate/secret`	FortiGate API credentials
`production/heezy/terraform/proxmox/secret`	Proxmox API credentials
`production/heezy/terraform/cloudflare/secret`	Cloudflare API token + account id
`production/heezy/cisco-switch/secret`	Cisco switch credentials
`production/heezy/ssh/mcp-admin-private-key`	MCP admin SSH private key
`production/heezy/k8s/kubeconfig`	Cluster kubeconfig
`production/heezy/postgres-credentials`	Postgres credentials
`production/heezy/receipts/aws-credentials`	Textract IAM creds
`production/heezy/gmail/oauth`	Gmail OAuth tokens for the mailbot jobs
`production/heezy/gitea/runner-pat`	Gitea runner PAT
`production/heezy/cloudflare/pages-token`	Cloudflare Pages deploy token
`production/heezy/backup/aws-credentials`	S3 backup credentials
`production/heezy/openbao/google-oidc`	Google OAuth client for UI login

Several paths are left over from retired services and can be deleted: production/heezy/big-boi/{n8n,kiro-gateway,openclaw-github}, production/heezy/n8n/api-key, production/heezy/openclaw/*, and the openclaw policy.

External Secrets Operator¶

ESO runs in external-secrets and syncs OpenBao paths into k8s Secrets in the secrets and heezy namespaces. Its OpenBao policy is eso-readonly (read/list on secret/data/*). Adding a secret to a workload means adding an ExternalSecret, not a hand-created Secret.

Grafana¶

Grafana runs on big-boi (192.168.1.21) as a Docker Compose service, not in k8s.

URL: https://grafana.heezy.info
Username: admin
Password: set via GF_SECURITY_ADMIN_PASSWORD in /mnt/lgtm/docker-compose.yml, rendered by the lgtm Ansible role

If you are locked out:

ssh mcp-admin@192.168.1.21
sudo docker exec grafana /usr/share/grafana/bin/grafana cli admin reset-admin-password <newpassword>

Troubleshooting¶

Can't reach openbao.heezy.info

dig openbao.heezy.info should return 192.168.1.25
If it does not, your resolver is not 192.168.1.27 or 192.168.1.29

OpenBao sealed

curl -sk https://openbao.heezy.info/v1/sys/health | python3 -m json.tool

"sealed": false is healthy. The unseal CronJob recovers it within 5 minutes. If it does not, check that openbao-init-keys still exists in the openbao namespace — the CronJob reads the unseal keys from it.

403 on a write

Check the policy attached to your token, not the path. ansible-readonly and eso-readonly are read and list only.