Exactly-Once Without a Message Broker: An Outbox and Idempotency Pattern for Monesize Core

Every piece of software that emits events eventually runs into the same wall. Sending is not delivering, and delivering is not processing. If a listener crashes halfway through writing its side effect, or a message gets redelivered because the sender timed out waiting for an acknowledgment, you're left with exactly two outcomes, and both are bad. You either lose data, or you double-post it.

For a double-entry accounting core, neither of those is a rounding error you can live with. Double-posting a journal debits a customer's account twice. Dropping an event never debits it at all. Either one is the kind of bug that shows up as a support ticket with a screenshot of someone's bank statement attached.

I want to walk through the approach I used to get near-exactly-once semantics for event-driven side effects in a system like that: a relational outbox for reliable delivery, and a per-listener idempotency guard so delivery and processing don't double-count. No message broker involved. Just Postgres, doing what it's already good at.

The Core Tension

The system underneath this is a double-entry accounting core. Finance, inventory, and payment events all produce side effects. An invoice payment creates a journal entry. A sale decrements stock. A recorded project triggers a ledger entry. Every one of these writes lands in the same Postgres database where the triggering business event originates, which matters more than it sounds like it should.

The classic trap that makes an outbox necessary is straightforward to describe and easy to hit in practice. You publish a domain event, and the process crashes before the listener's side effect commits. The event is gone for good. Or the reverse happens: you write the side effect first, and the process crashes before the event ever gets recorded, and now you've committed a write whose partner never runs. Neither failure is acceptable when the write in question moves money.

A message queue solves part of this. It gives you persistence and redelivery, so a crashed consumer doesn't lose the message outright. What it doesn't give you is exactly-once delivery, only at-least-once. Your consumer can see the same message twice, whether from redelivery after a crash mid-commit, a client reconnect, or a dozen other ordinary failure modes. Put those two things together and the shape of the solution falls out on its own: at-least-once delivery plus idempotent processing equals exactly-once side effects. The outbox handles "don't lose it." Idempotency handles "don't apply it twice." Neither piece alone is sufficient, and it took me a while working through failure scenarios to actually believe that.

Part 1: Don't Lose It, the Outbox

The rule here is simple to state and easy to enforce with a database transaction: write the domain row and the outbox row together, in the same transaction. Either both commit, or both roll back. There's no window in which the business change exists without its event being durably queued somewhere, because the database itself won't let that window open.

model EventOutbox {
  id          String   @id @default(uuid())
  module      String
  event       String
  payload     Json
  status      String   @default("PENDING") // PENDING | FAILED
  error       String?
  retryCount  Int      @default(0)
  createdAt   DateTime @default(now())
  processedAt DateTime?

  @@index([status, createdAt])
  @@map("event_outbox")
}

A helper writes the row inside whatever transaction is already handling the business write, so the caller never has to think about it as a separate step:

writeEvent(tx, module, event, payload) {
  return tx.eventOutbox.create({
    data: { module, event, payload },
    select: { id: true },
  });
}

Two delivery paths

Delivery runs on a fast path plus a safety net, and the reasoning behind splitting it that way is worth spelling out.

Figure 1. Transactional outbox flow

The fast path fires right after the business transaction commits. The event gets emitted in-process, and the outbox row gets deleted, and critically, the emit and the delete happen inside the same transaction as each other:

async emitWithDelete(eventBus, outboxId, event, payload) {
  await prisma.$transaction(async (tx) => {
    eventBus.emit(event, { ...payload, _eventInstanceId: outboxId });
    await tx.eventOutbox.delete({ where: { id: outboxId } });
  });
}

The outbox row's id doubles as the event's identifier for everything downstream. When it gets emitted, the payload gets stamped with _eventInstanceId: outboxId, and that stamp is the hook the idempotency layer relies on later. It's a small detail, but it's the detail the whole second half of this design depends on.

The safety net exists for the case where the process crashes between committing the business transaction and running emitWithDelete. In that scenario, the row is still sitting there with status PENDING. A poller sweeps for stragglers every ten minutes and replays them:

await prisma.$transaction(async (tx) => {
  const [row] = await tx.$queryRawUnsafe(
    `SELECT * FROM event_outbox
     WHERE status = 'PENDING'
     ORDER BY created_at ASC
     LIMIT 1
     FOR UPDATE SKIP LOCKED`
  );
  if (!row) return;
  try {
    eventBus.emit(row.event, row.payload);
    await tx.eventOutbox.delete({ where: { id: row.id } });
  } catch (err) {
    // retry up to MAX_RETRIES, then mark FAILED
  }
});

Two details here are doing more work than they look like.

FOR UPDATE SKIP LOCKED is what lets multiple poller workers run concurrently without stepping on each other. Each worker locks the row it grabs and simply skips past any row another worker has already locked, instead of blocking on it. Without this, scaling the poller horizontally would mean workers piling up behind each other's locks, which defeats the point of having more than one.

The ten-minute cadence is a deliberate choice, not an arbitrary default. It means the poller almost never replays an event that was already handled by the fast path. It only ever picks up genuine orphans, the ones left behind by an actual crash. That keeps the poller firmly in the role of recovery mechanism rather than primary delivery path, which matters for latency: the fast path stays fast because it isn't competing with a poller running every few seconds "just to be safe."

One thing worth naming explicitly, since it's easy to gloss over: the poller's retry logic needs a ceiling. A row that fails repeatedly, whether from a genuinely broken payload or a downstream service that's been down for hours, shouldn't retry forever. After MAX_RETRIES, it gets marked FAILED instead of PENDING, which pulls it out of the poller's query entirely and turns it into something a human needs to look at. Without that ceiling, a single bad event can quietly consume poller cycles indefinitely while looking, from the outside, like the system is healthy.

Part 2: Don't Apply It Twice, Idempotency

Here's the harder half. The same event can arrive by both paths, an in-process fast emit followed by a poller replay if the process happened to crash at just the wrong instant, or it can simply get redelivered after a partial crash somewhere downstream. The listeners on the other end open a double-entry journal, debit and credit accounts, and mutate stock levels. None of that is safe to run twice.

The fix is a claim table, keyed uniquely on the combination of listener, event, and event instance:

model EventIdempotency {
  id              String   @id @default(uuid())
  listenerModule  String   @map("listener_module")
  event           String
  eventInstanceId String   @map("event_instance_id")
  processedAt     DateTime @default(now())

  @@unique([listenerModule, event, eventInstanceId])
  @@index([processedAt])
  @@map("event_idempotency")
}

The insight that makes this work is that eventInstanceId is the outbox row id stamped onto the payload earlier. That's what makes the idempotency guard granular to one specific delivery of one specific event, rather than to the event type as a whole. A sale that happens twice, genuinely twice, produces two separate outbox rows, two separate instance ids, and two valid posts, exactly as it should. A sale that gets redelivered because of a network retry carries the same outbox id both times, and the guard blocks the second attempt outright.

The atomic claim

The naive version of this check is SELECT, look at the result, then decide whether to INSERT. That has a race window sitting right in the middle of it: two concurrent deliveries can both run the SELECT, both see nothing, and both proceed to do the work. The fix is claiming the row in a single atomic statement instead of two separate ones:

async checkIdempotency(tx, listenerModule, event, eventInstanceId) {
  const [row] = await tx.$queryRawUnsafe(
    `INSERT INTO event_idempotency (id, listener_module, event, event_instance_id)
     VALUES (gen_random_uuid(), $1, $2, $3)
     ON CONFLICT (listener_module, event, event_instance_id) DO NOTHING
     RETURNING id`,
    listenerModule, event, eventInstanceId
  );
  return !row; // true = already processed
}

If the insert succeeds and returns a row, this is genuinely the first time this instance has been seen, and the function returns false, meaning proceed. If it hits the unique constraint and inserts nothing, no row comes back, and the function returns true, meaning skip, this one's already been handled. Because the guarantee is enforced by a database constraint rather than application logic, two concurrent deliveries racing each other resolve to exactly one winner with no explicit locking on my part and no window for both to slip through.

The part that actually makes it sound: check inside the same transaction as the work

This is the single architectural decision the whole pattern hinges on, and it's easy to get subtly wrong by putting the claim and the side effect in separate transactions.

const entry = await prisma.$transaction(async (tx) => {
  const alreadyProcessed = await checkIdempotency(tx, "ACCOUNTING", data.event, data.eventInstanceId);
  if (alreadyProcessed) return null;              // short-circuit duplicate

  const e = await tx.accountingJournalEntry.create({ ... });  // the actual side effect
  // ... the idempotency insert is committed here too, atomically
});

Because the claim insert and the side effect live inside one transaction together, there are only two possible outcomes. Either the side effect commits and the claim gets recorded, both together, or neither does, and the listener is free to retry the whole thing from scratch. There's no gap in between where the journal entry exists but the claim doesn't, or the claim exists but the journal entry never got written. That gap is precisely the one that causes double-posting whenever a process dies at an inconvenient moment, and closing it is the entire point of putting these two operations in the same transaction rather than treating the idempotency check as a lightweight guard bolted on beforehand.

Every listener, framed this way, becomes a transactional claim-and-perform: claim event instance X, and if the claim wins, do the work, all in one atomic unit that either fully happens or fully doesn't.

Part 3: One Deliberate Exception, Email

Not every domain in the system cares equally about duplication, and I made a conscious carve-out for transactional email triggered by auth and sales events.

case "auth:password_reset_requested":
  return await checkIdempotency(tx, "EMAIL", "auth:password_reset_requested", data._eventInstanceId);

For email, I'd rather duplicate than lose. If a password reset link goes out twice, that's mildly annoying at worst, the user clicks whichever link arrives first and ignores the second. If it never goes out at all because the listener crashed mid-flight and the idempotency guard suppressed the retry, that's a user locked out of their account with no way back in. So email listeners keep the same guard structurally, it still catches the ordinary case of an accidental duplicate send, but the failure posture underneath it is different: when something has to go wrong, resending is the less-bad option, not losing the message entirely.

The general principle worth pulling out of this: idempotency policy is a product decision made per event, not a blanket rule applied uniformly across the system. Money and email do not carry the same cost when duplicated, and treating them identically would have been the easier engineering choice and the wrong one.

Where This Approach Has Limits

It's worth being honest about what this design doesn't give you, since every pattern that solves one problem quietly declines to solve a few others.

There's no ordering guarantee across events. The outbox and the poller both process rows independently, and nothing here enforces that event A, written before event B, gets processed before it. For accounting side effects keyed to independent business objects, that's been fine in practice, an invoice's journal entry doesn't depend on the order it was created relative to some unrelated sale's stock decrement. If your domain has cross-event ordering dependencies, this pattern needs a sequence number and an explicit ordering check added on top, and that's a meaningfully different design.

The idempotency table also grows forever unless something prunes it. Every processed event instance leaves a permanent row behind, and at high event volume that table becomes the largest one in the database given enough time. I run a periodic cleanup job that deletes rows older than a safe retention window, long enough that no realistic redelivery could still be in flight, short enough to keep the table from becoming a liability. The processedAt index exists specifically to make that cleanup query cheap.

And this whole approach assumes a single Postgres database as the source of truth for both the business write and the event. The moment that assumption breaks, multiple services each owning their own database, for instance, you're looking at something closer to Debezium reading the write-ahead log and publishing change events into an actual broker. That's a heavier piece of infrastructure, and it solves a different problem: cross-service consistency rather than same-database reliability. For a single accounting core with one database underneath it, adding that infrastructure would have been solving a problem I didn't have yet.

The Mental Model

Write the domain row and the outbox row in the same transaction. Durability comes from the database, not from a queue.

Deliver through a fast path, with a SKIP LOCKED poller as the recovery mechanism and a retry ceiling before anything gets marked FAILED.

Stamp every event with the outbox row's own id and carry that id through the payload as its instance identifier.

Guard every side effect with a constraint-backed claim, an insert with ON CONFLICT DO NOTHING, inside the same transaction as the side effect itself, so the claim and the work are atomic together.

Treat idempotency policy as a per-domain decision. When both failure modes are bad, pick the less bad one on purpose instead of applying the same rule everywhere out of convenience.

What falls out of all of this is the practical version of exactly-once: at-least-once delivery, paired with listeners that are safe to run once and only once against the database. No message broker, no eventual-consistency black box to reason about, and no double-posted journal entry showing up in someone's account statement.

Comments

Post a Comment

Popular Posts

Exploiting MS17-010 EternalBlue: SMB Flaw to SYSTEM Access

This writeup documents my exploitation of a vulnerable Windows machine using Metasploit. The goal was to identify the exposed attack surface, discover a critical vulnerability, exploit it, obtain privileged access, retrieve the flag file, and extract the NTLM hash of a user account. The exercise covered the full penetration testing workflow: Reconnaissance Vulnerability identification Exploitation Privilege verification Post-exploitation activities Reconnaissance The first step in any penetration test is understanding what is exposed. You cannot attack what you cannot see, and you should not attack blindly what you can. I ran an Nmap version scan against the target: nmap -sV <target-ip> The scan revealed the following open ports and services: 135/tcp open msrpc 139/tcp open netbios-ssn 445/tcp open microsoft-ds 3389/tcp open tcpwrapped 49152/tcp open msrpc 49153/tcp open msrpc 49154/tcp open msrpc 49155/tcp open msrpc The critical discovery was: 445/tcp ope...
Read more

God Never Wrote a Book: A Nigerian Agnostic's Case

Religion is a human system. It claims divine authorship for human documents. It asserts universal jurisdiction while displaying local origins. It promises moral order while producing observable dysfunction. This essay examines those claims directly. Let me be clear about what this is and what it is not. This is not an attack on believers. I grew up around believers. I know good people who believe. I know generous, thoughtful, hardworking people who also read Hebrews every morning and face Mecca every afternoon. This is not about them. It is about the systems they participate in. The claims those systems make. And the evidence those claims are supposed to rest on. Because the claims are extraordinary. And the evidence is not. The Fraud of Divine Authorship Let us start with the most basic question. How does a perfect, infinite, all-knowing being communicate with humanity, and the best delivery mechanism available is a human being with a pen? Every holy book was written by peop...
Read more

How I Patched CVE-2026-42945 on Monesize Nginx

On June 28, 2026, I discovered that our Monesize demo environment was running nginx 1.24.0, a version vulnerable to multiple critical security flaws including CVE-2026-42945, a remote code execution vulnerability with a CVSS score of 9.2. This post documents exactly how I identified the problem, patched it, and verified the fix, including a misconfiguration that took our applications offline for a few minutes in the process. The Discovery I was testing our authentication flow using Burp Suite Community Edition when I noticed something in the response headers coming back from our API gateway: HTTP/1.1 403 Forbidden Server: nginx/1.24.0 (Ubuntu) The Server header was leaking the exact nginx version and operating system. This is not just a cosmetic issue. Version disclosure is a reconnaissance gift. It gives an attacker a precise target to query against known vulnerability databases. Instead of probing blindly, they know exactly what they are dealing with and can search for working e...
Read more