The transactional outbox

Here's the shape of the bug I was trying to design out.

Someone submits a request. My handler writes a row to the database, then calls an email service to send the confirmation. Two steps. They look like one operation in the code, but they are not atomic — nothing ties them together. If the email API has a bad five seconds right after my database commit, that confirmation email is just gone. No error surfaces to the user. No record exists that it was ever owed. The database cheerfully says "submitted" and the person waits for an email that will never arrive.

I hit this often enough that I stopped solving it per-feature and adopted the outbox pattern everywhere.

The fix

Write the intent to the database in the same transaction as the business change:

[one DB transaction]
  1. INSERT INTO applications (...)
  2. INSERT INTO outbox_events (event_type: 'application.submitted', payload_json: {...})
[commit — both land, or neither does]

[a separate background process, polling WHERE status = 'pending']
  → attempts delivery
  → on success:  status = 'sent'
  → on failure:  attempts += 1, retry with backoff

The event row carries aggregate type, aggregate id, event type, payload and status. Because it's written inside the same transaction as the thing that triggered it, the two genuinely succeed or fail together — I'm leaning on the database's own guarantees instead of trying to coordinate two systems by hand.

A failed delivery is now a row sitting in pending or failed. Visible. Queryable. Retryable. Something I can alert on. Not a silent hole.

What it costs

• Latency. The side effect fires on the next poll cycle, not in the request. Wrong tool for anything that genuinely has to be synchronous.
• Another moving part. The dispatcher itself has to be running and watched. If it quietly dies, events pile up in pending while every database write still looks perfectly healthy — I learned to monitor the queue depth for exactly this.
• At-least-once, not exactly-once. A crash between "delivered" and "marked sent" can double a delivery, so the side effects want to be idempotent rather than assuming they run once.

Worth the complexity when losing the side effect actually matters — a confirmation someone's waiting on, a notification tied to a real workflow step, a webhook another system depends on. Not worth it for best-effort things nobody notices if they occasionally drop.