The Inbox & Outbox Pattern: Reliable Events with Kafka 📬

Kafka gives you a durable, replayable log — but there's a gap it can't close for you: the moment your service updates its database and publishes an event. Get that seam wrong and you either lose events or fire phantom ones. The Outbox and Inbox patterns are how we close it. ☕

The dual-write problem 💥

Picture an order service. It needs to save the order to Postgres and publish order.created to Kafka. Two systems, two network calls, no shared transaction:

• DB commit succeeds, then the app crashes before the Kafka publish → the event is lost. Downstream never hears about the order.
• Kafka publish succeeds, then the DB transaction rolls back → a phantom event for an order that doesn't exist.

You can't wrap a database transaction and a Kafka publish in one atomic unit. Retries and try/catch just move the race around. The fix is to stop dual-writing entirely.

Outbox: the producer side 📤

Don't publish to Kafka from your business logic. Instead, write the event into an outbox table in the same database transaction as your business change:

1. In one transaction: insert the order and insert the event row into outbox. Both commit or neither does.
2. A separate relay (a poller, or Change Data Capture reading the DB log) reads unsent rows and publishes them to Kafka.
3. On success it marks the row as sent.

The table is deliberately boring — it lives in the same database as your business data, so it commits atomically with it:

-- inserted in the SAME transaction as the order
CREATE TABLE outbox (
    id          BIGSERIAL PRIMARY KEY,
    event_type  TEXT        NOT NULL,        -- 'order.created'
    payload     JSONB       NOT NULL,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    sent_at     TIMESTAMPTZ                  -- NULL until the relay ships it
);

The relay is just a loop over the unsent rows:

loop every N ms:
  rows = SELECT * FROM outbox WHERE sent_at IS NULL ORDER BY id LIMIT 100
  for row in rows:
      kafka.publish(row.event_type, row.payload)
      UPDATE outbox SET sent_at = now() WHERE id = row.id

Now the source of truth is your database. If the relay crashes mid-flight, it just re-reads the unsent rows on restart — the event is never lost, only delayed.

Inbox: the consumer side 📥

The relay guarantees at-least-once delivery, which means consumers will occasionally see the same event twice — after a rebalance, a redelivery, or a retry. So the consumer must make reprocessing a no-op.

Keep an inbox table of message IDs you've already handled. Before processing an event, check it in the same transaction as your work:

• New ID → process it and record the ID. Both commit together.
• Seen ID → skip it. The redelivery does nothing.

That's idempotency: process each event exactly once in effect, even when Kafka delivers it more than once.

Putting it together 🔗

flowchart LR
    subgraph Producer[Producer Service]
        BL[Business logic]
        DB[(DB + outbox<br/>one transaction)]
        RELAY[Relay / CDC]
        BL --> DB
        DB --> RELAY
    end
 
    K[(Kafka<br/>append-only log)]
 
    subgraph Consumer[Consumer Service]
        CN[Consumer]
        IN[(inbox: seen IDs<br/>one transaction)]
        CN --> IN
    end
 
    RELAY -->|publish| K
    K -->|at-least-once| CN

The takeaway 💡

Kafka handles durability and replay in the middle. The outbox anchors the event to your database on the way in, and the inbox absorbs redelivery on the way out. Together they turn "at-least-once" plumbing into effectively exactly-once business behavior — without distributed transactions. 🎯