Scripts that were right on paper
Deployment scripts read correctly and passed their tests while a broken compose file, a leftover file and an ignored folder went unnoticed.
The safety net had a hole
The test, the audit log, the scanner, the CI run — covering less than I assumed it did.
Four deployment problems that all read fine on paper.
A restore script left a temporary file behind. My first edit of a compose file garbled it and the deployment tests still passed, so there's now a test that runs docker compose config on it. A test exported a version variable that overrides .env, so a broken update looked successful. And the reverse proxy's file provider doesn't read subfolders, so routes and certificates placed there were silently ignored. A deleted route file isn't always noticed either, so archiving one now empties it first.
What found these was a smoke test that installs the real stack on free ports, which is why I trust it more than the unit tests.