Why Monesize Core Uses No Kafka or Redis

One of the most common patterns in software engineering is solving reliability problems by introducing more infrastructure.

Need background jobs? Add Redis. Need reliable event delivery? Add Kafka. Need asynchronous processing? Add a queue. Need distributed messaging? Add another service.

Sometimes that is the right decision. Sometimes it is not.

Recently, I implemented the Event Outbox pattern across Monesize Core to guarantee reliable cross-module event delivery. The obvious question is: why not just use Redis, Kafka, RabbitMQ, or BullMQ?

The answer comes down to architecture.

Monesize Core is not a typical multi-tenant SaaS platform. Every customer receives their own isolated deployment. Their own application environment. Their own database. Their own infrastructure footprint.

This architectural decision has implications.

Every new infrastructure dependency added to the platform becomes a dependency that must exist in every customer deployment. 

Introducing Redis would not simply mean operating one Redis cluster. It would mean every deployment now requires provisioning, configuration, monitoring, backup strategies, security hardening, version management, incident response, and operational maintenance.

At one customer, this is manageable. At fifty customers, it becomes noticeable. At hundreds of customers, it becomes a permanent operational burden.

This is where engineering decisions become trade-offs rather than technical exercises.

The question is not "Can Redis solve this problem?" Of course it can. The question is "Does this problem require Redis?" Those are different questions.

When I looked at the reliability problem I was trying to solve, I realized something important. The database already contained everything I needed.

Transactions already existed. Persistence already existed. Row-level locking already existed. Retry state could be stored in tables. Dead-letter handling could be stored in tables. Durability already existed.

Why introduce another moving part if the existing infrastructure could already solve the problem?

The failure mode I wanted to eliminate was straightforward. A business transaction succeeds. The database commits. The application attempts to emit an event. The process crashes. The event disappears. One module believes something happened. Another module never receives the message. The system slowly drifts out of sync.

For operational software, that is unacceptable.

The solution I implemented writes events to an Event Outbox table inside the same database transaction as the business operation itself. If the transaction commits, the event exists. If the event exists, a background poller will eventually process it. If processing fails, retries occur automatically. If retries are exhausted, the event moves into a failed state where it becomes visible and recoverable.

The system now has reliable event delivery without requiring additional infrastructure. No Redis. No Kafka. No RabbitMQ. No separate queue service. No additional maintenance burden for future deployments.

Some engineers equate architectural sophistication with the number of technologies involved. I tend to think the opposite.

Good architecture is not about introducing components. It is about solving problems with the smallest number of moving parts necessary.

Every dependency is a future maintenance obligation. Every service added today becomes something that must be monitored, upgraded, secured, and supported tomorrow. Removing infrastructure is often harder than adding it.

Adding Kafka would have been easy. Justifying why Kafka was truly necessary would have been harder. In this case, I could not justify it. The database was already capable of providing the guarantees I needed.

The simplest solution turned out to be the most operationally efficient one.

As Monesize Core grows, I expect many future engineering decisions to follow the same principle. Not every problem deserves another service. Not every reliability challenge requires more infrastructure. Sometimes the best engineering decision is the one that leaves you with fewer things to manage.

Comments

Post a Comment

Popular Posts

Building Event Reliability at Monesize Core

As Monesize Core evolved into a modular business platform, more parts of the system started depending on events. Accounting needed to react to sales. Inventory needed to react to purchases. Activity logs needed to capture important actions. Notifications needed to respond to business events. The architecture naturally moved toward an event-driven approach. A service performs a business operation, emits an event, and other modules react. At first glance, this looks simple. A transaction happens. An event is emitted. Listeners process the event. The problem is that distributed systems are not defined by what happens when everything works. They are defined by what happens when something fails. The question I eventually had to answer was: how do I guarantee that a business event is never silently lost? The Gap Between Database Writes and Events The first version of event handling looked simple: await prisma.sale.create({ data: saleData }); eventBus.emit('sale:created', {...
Read more

RecruitX: From Reconnaissance to Remote Code Execution

I recently completed a TryHackMe room called RecruitX, a web application penetration test that walks through a realistic engagement from start to finish. This post documents the exact chain of vulnerabilities I exploited and what I would fix if I were the developer. The Setup RecruitX is an internal recruitment portal. Hiring managers post jobs, candidates apply, and administrators manage the workflow. I was given a target IP, a web-based attack box, and a simple brief: find the security issues. Reconnaissance and Enumeration I started with Nmap to discover open ports and services: Port 22: SSH Port 80: Apache 2.4.58 running the main application Port 3306: MySQL Port 8080: Apache default page The stack was classic LAMP: Linux, Apache, PHP, MySQL. I confirmed PHP session management from the HTTP headers, which showed a PHPSESSID cookie. Next I ran Gobuster for directory enumeration and found several interesting paths: /admin — admin panel, redirected to login when unauthenticated /api ...
Read more

God Never Wrote a Book: A Nigerian Agnostic's Case

Religion is a human system. It claims divine authorship for human documents. It asserts universal jurisdiction while displaying local origins. It promises moral order while producing observable dysfunction. This essay examines those claims directly. Let me be clear about what this is and what it is not. This is not an attack on believers. I grew up around believers. I know good people who believe. I know generous, thoughtful, hardworking people who also read Hebrews every morning and face Mecca every afternoon. This is not about them. It is about the systems they participate in. The claims those systems make. And the evidence those claims are supposed to rest on. Because the claims are extraordinary. And the evidence is not. The Fraud of Divine Authorship Let us start with the most basic question. How does a perfect, infinite, all-knowing being communicate with humanity, and the best delivery mechanism available is a human being with a pen? Every holy book was written by peop...
Read more