The nightly Postgres backup CronJob has been failing every run — no backups exist in s3://drops/_pgbackups/.
Cause: the upload container runs as uid 65532 (runAsNonRoot). mc defaults its config to $HOME/.mc = /.mc and dies with mkdir /.mc: permission denied on the non-writable root fs — before any upload.
Fix: set MC_CONFIG_DIR=/work/.mc (the shared emptyDir, writable via fsGroup: 65532). The pg_dump initContainer already succeeds; this lets the upload step actually run.
Validated: kubectl kustomize renders clean; env present on the upload container.
The nightly Postgres backup CronJob has been **failing every run** — no backups exist in `s3://drops/_pgbackups/`.
**Cause:** the upload container runs as uid 65532 (`runAsNonRoot`). `mc` defaults its config to `$HOME/.mc` = `/.mc` and dies with `mkdir /.mc: permission denied` on the non-writable root fs — before any upload.
**Fix:** set `MC_CONFIG_DIR=/work/.mc` (the shared emptyDir, writable via `fsGroup: 65532`). The `pg_dump` initContainer already succeeds; this lets the upload step actually run.
Validated: `kubectl kustomize` renders clean; env present on the upload container.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The backup CronJob runs as uid 65532 (runAsNonRoot). mc defaulted its
config dir to $HOME/.mc = /.mc and failed with "mkdir /.mc: permission
denied" on the non-writable root fs — every nightly run died before
uploading, so there are currently no backups in s3://drops/_pgbackups/.
Point MC_CONFIG_DIR at the shared /work emptyDir (writable via fsGroup).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jorgen.stensrud
marked the pull request as work in progress 2026-06-11 11:18:42 +00:00
jorgen.stensrud
marked the pull request as ready for review 2026-06-12 10:27:25 +00:00
jorgen.stensrud
requested review from danijel.simeunovic 2026-06-12 10:27:30 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
The nightly Postgres backup CronJob has been failing every run — no backups exist in
s3://drops/_pgbackups/.Cause: the upload container runs as uid 65532 (
runAsNonRoot).mcdefaults its config to$HOME/.mc=/.mcand dies withmkdir /.mc: permission deniedon the non-writable root fs — before any upload.Fix: set
MC_CONFIG_DIR=/work/.mc(the shared emptyDir, writable viafsGroup: 65532). Thepg_dumpinitContainer already succeeds; this lets the upload step actually run.Validated:
kubectl kustomizerenders clean; env present on the upload container.🤖 Generated with Claude Code