Building a Self-Hosted SIEM for Monesize Infrastructure
As Monesize's infrastructure keeps growing, security monitoring had to evolve beyond checking individual servers, reviewing application logs and responding to vulnerabilities when they surface. A production environment generates too much security information for that approach to remain practical. System logs, application events, package inventories, authentication activity and network traffic all contain useful signals, but those signals become much more valuable when they can be collected centrally and investigated in context.
That led us to build a self-hosted Security Information and Event Management system, or SIEM, for Monesize. The objective was not simply to install a security dashboard. We wanted a central security monitoring layer that could collect telemetry from our workloads, identify vulnerabilities in installed software, inspect network activity, generate security alerts and give us a single place from which to investigate what was happening across the environment.
We engineered the initial system around Wazuh and Suricata. Wazuh provides the central SIEM and endpoint monitoring capability, while Suricata provides network intrusion detection on the workloads where network-level visibility is important. We then connected Suricata's structured event stream to Wazuh so that network detections become part of the same security monitoring workflow as endpoint and vulnerability events.
The implementation also required us to think about the security of the SIEM itself. The Wazuh dashboard is an administrative interface, so we did not want to leave it publicly accessible simply because the server had a public address. We put the dashboard behind our VPN, placed NGINX in front of it, configured a proper domain and TLS certificate, and restricted access at the AWS network layer.
The result is a security monitoring architecture that gives Monesize visibility at both the endpoint and network levels while remaining under our operational control.
Why We Needed a SIEM
The need for a SIEM came from a fairly straightforward infrastructure problem. Security information existed in several places, but it was distributed across the environment.
An individual Linux server knows a lot about itself. It knows which packages are installed, which processes are running, which users have authenticated, which services have generated errors and which system events have occurred. An application server can also produce application-specific logs that contain information unavailable at the operating system level.
Network activity adds another layer. A server can receive connection attempts against ports that its applications never process. An external host can probe an SSH service, scan a database port or send traffic associated with known malicious infrastructure without producing an application-level event that tells us much about what happened.
Those sources answer different questions. Endpoint telemetry helps explain what happened inside a workload. Application logs explain what happened within an application. Network telemetry helps explain what reached the workload in the first place. Vulnerability intelligence tells us where the software running on the workload has known weaknesses.
Without centralization, investigating those signals requires moving between systems and manually establishing the relationship between events. That is workable for occasional troubleshooting, but it becomes increasingly inefficient as infrastructure and telemetry grow.
A SIEM gives us a central layer for collecting and analyzing those events. More importantly, it gives us a common investigation interface. Instead of treating every server and every security data source as an isolated system, we can search and correlate security information through the same platform.
That was the capability we wanted to establish at Monesize.
Why We Chose Wazuh
Wazuh became the foundation because it already combines several capabilities that we needed rather than forcing us to assemble completely separate systems for endpoint monitoring, vulnerability detection, log analysis and security alerting.
At a high level, Wazuh has agents that run on monitored endpoints and central components that receive, analyze, index and present the resulting security data. The Wazuh agent can collect logs, monitor files, gather system and package information, perform security configuration assessments and provide other endpoint telemetry. The central Wazuh server processes that information, the Wazuh indexer stores the data, and the Wazuh dashboard provides the interface through which we investigate it.
That architecture suited our requirements because we could put the collection component close to the workloads while keeping the analysis and investigation layer centralized.
We deployed Wazuh using its All-In-One deployment available through AWS Marketplace. An all-in-one deployment combines the core Wazuh components on the same system, which gave us a practical starting point without introducing distributed infrastructure before we needed it. Wazuh supports more distributed and clustered architectures as environments grow, so the initial deployment does not prevent us from expanding the architecture later.
The choice was also consistent with our preference for understanding and controlling the infrastructure ourselves. A managed security platform can remove some operational responsibilities, but a self-hosted platform gives us direct visibility into how the security monitoring stack works, where the data resides, how it is accessed and how it integrates with our infrastructure.
That tradeoff is important. Self-hosting does not eliminate operational work. It moves some of that responsibility to us. We therefore approached the SIEM as infrastructure that needs to be secured, maintained and monitored rather than as a piece of software that can simply be installed and forgotten.
Establishing Endpoint Monitoring
Once the central Wazuh deployment was running, we installed Wazuh agents on the workloads we wanted to monitor.
The agent gives each monitored system a direct connection to the central security platform. Instead of logging into a server whenever we want to investigate its security state, we can inspect its telemetry through Wazuh.
One of the first capabilities we used was vulnerability detection. Wazuh maintains package inventory for monitored endpoints and uses vulnerability intelligence to identify packages associated with known vulnerabilities. This provided a useful connection between what was actually installed on our servers and the vulnerabilities that affected those packages.
During the initial review, Wazuh identified vulnerabilities in software running on the production workload. One of the findings concerned the Linux AWS kernel. Rather than treating the vulnerability count as a reason to immediately change production, we inspected the installed package version and the candidate version available through the operating system's package repositories.
The installed kernel was older than the available candidate, so we upgraded it and rebooted the server. After the reboot, we verified the running kernel directly. This gave us a complete remediation path from detection to verification rather than simply marking a vulnerability as resolved because a package command had been executed.
We encountered a similar situation with PM2. Wazuh identified a vulnerability affecting the installed PM2 version. We checked the version actually being used by the system, upgraded PM2, verified the resulting version and restored the application processes through PM2's process management mechanism.
That experience demonstrated one of the practical benefits of having vulnerability information integrated into the SIEM. The security platform tells us what requires attention, but the actual remediation still requires engineering judgment. We need to establish whether the vulnerable package is actually installed, whether the affected version is in use, what update is available and what operational consequences the update may have.
The SIEM provides the information required to make those decisions, although it does not replace the engineering process around them.
Why Endpoint Monitoring Was Not Enough
Endpoint monitoring gave us a useful security view, but it did not answer every question we had.
Suppose an external host starts scanning a database port. The operating system may not produce a particularly useful security event simply because a connection attempt reached the port. If the database never accepted the connection, the application may have no record of the interaction at all.
From the perspective of the application, nothing meaningful may have happened. From the perspective of network security, the event can still matter.
The same applies to reconnaissance against SSH, web services and other exposed ports. An attacker does not need to authenticate successfully or exploit a vulnerability for the reconnaissance itself to provide useful information about activity against the infrastructure.
We therefore needed another source of telemetry that operated at the network level. This is where Suricata makes it into the "first eleven." Lol.
Adding Suricata for Network Detection
Suricata is an open-source network analysis and threat detection engine. It can inspect network traffic and identify patterns that match its detection rules, producing structured events that describe the traffic and the associated detection.
We use Suricata as a network intrusion detection layer. Its role is to observe traffic and identify suspicious or malicious patterns, while Wazuh provides the central platform through which those detections are collected and investigated.
This division of responsibility was important to us. We did not want to turn the initial deployment into an automated network prevention system before understanding the detections it produced. Detection gives us visibility into what is happening without immediately introducing automated blocking decisions into production traffic.
Suricata also produces EVE JSON, a structured event format that contains considerably more useful information than an ordinary text log. An event can include the timestamp, source and destination addresses, ports, protocol, flow information and details about the detection rule that matched the traffic.
That made Suricata a natural fit for the SIEM architecture.
Understanding Network Visibility in AWS
Deploying Suricata required us to account for the way network visibility works in AWS.
A common assumption when designing an intrusion detection system is that one sensor can observe traffic belonging to every system on a network. That is not automatically true in a cloud environment.
An EC2 instance does not simply receive a copy of all traffic moving between other instances in the environment. Each workload has its own network interface and AWS networking determines which traffic reaches that interface.
That meant we could not install Suricata on a single server and assume it would passively see all traffic associated with every other workload.
For the current architecture, we instead installed Suricata directly on the application servers where network-level monitoring matters. Each sensor therefore observes traffic reaching and leaving its own network interface.
This approach has a clear limitation, but it also has a useful property: we know exactly what each sensor is responsible for observing. We are not claiming visibility that the underlying network architecture does not provide.
Configuring Suricata on Production
We started with the production workload. The first thing we checked was the actual network interface. The server was using an AWS interface named ens3, while the default Suricata configuration referenced eth0. That configuration had to change.
Suricata supports AF_PACKET for packet acquisition on Linux. AF_PACKET allows Suricata to receive packets directly from the Linux network interface, making it suitable for inspecting the traffic reaching the workload.
We changed the configured AF_PACKET interface from eth0 to ens3. This was a small change, but it illustrates why security tooling should be configured against the actual infrastructure rather than against assumptions from a generic installation guide.
We then updated Suricata's detection rules. The initial configuration expected a Suricata rules file that was not yet present. Running the configuration test exposed that problem before we started the service. We used Suricata Update to retrieve and prepare the ruleset, then ran the configuration test again.
The second validation completed successfully. Only after the configuration and rules loaded successfully did we enable Suricata as a system service.
That sequence gave us a much stronger indication that the service was ready than simply checking whether systemd reported it as running.
Seeing the First Network Events
The next question was whether Suricata was actually seeing traffic.
We checked the Suricata log directory and initially found an empty EVE JSON file. An empty event file is not necessarily evidence of a failed deployment. It can simply mean that no event has been generated yet.
We watched the EVE stream while the server continued operating. DNS events appeared first. Those events were useful because they demonstrated that Suricata was observing traffic through the correct interface and producing structured output.
Then the first security detections appeared. Suricata identified inbound traffic associated with known hostile or compromised infrastructure. It also produced detections associated with scanning activity and threat intelligence blocklists.
The events contained the information we needed to investigate them, including source address, destination address, destination port, protocol and the detection signature.
One of the early detections identified traffic associated with the DShield blocklist. The event showed an external source attempting to reach the server over TCP port 80, and Suricata classified the source as belonging to a blocklisted group.
This was the first clear demonstration that the network sensor was doing more than collecting generic packet information. It was applying security intelligence to the traffic and producing an event that could be investigated.
The events also showed an action of allowed. That was expected because the initial configuration was detection-oriented. Suricata identified the traffic but did not automatically block it.
For the first deployment, that distinction was intentional. We wanted to understand the traffic and establish confidence in the detection layer before introducing automated prevention.
Connecting Suricata to Wazuh
Suricata was now generating useful network security events, but those events were still sitting on the production server. The next step was to make them part of the central SIEM.
Suricata writes its structured events to /var/log/suricata/eve.json. Wazuh agents can collect local JSON log files, so we added that file to the Wazuh agent's local file configuration and specified JSON as the log format.
After restarting the Wazuh agent, we checked the agent's own log collector output. The agent reported that it was analyzing the Suricata EVE file.
That confirmed that the Wazuh agent had accepted the configuration and was actively watching the file. The next validation happened in the Wazuh dashboard.
Wazuh's Threat Hunting interface allows us to filter events by their associated fields and rule groups. We filtered for the production agent and the Suricata rule group.
The resulting events confirmed the complete integration.
Suricata generated the detection. The event was written to EVE JSON. The Wazuh agent collected it. Wazuh processed it. The event became searchable in the central SIEM.
That end-to-end path was the important milestone because it established that network detections were no longer isolated on individual servers.
Why Structured JSON Matters
The use of EVE JSON is more significant than it might initially appear.
A conventional text log might contain a sentence describing a security event. An engineer can read that sentence, but machines have to parse the text to reliably extract the individual fields.
A structured JSON event already separates those fields.
Suricata can identify the source IP, destination IP, source port, destination port, protocol, event type and alert signature as distinct fields. Wazuh can then process those fields and make them available for searching and analysis.
This matters when investigating an event.
If I want to find Suricata detections associated with a particular endpoint, I can filter by the Wazuh agent. If I want to investigate network intrusion events, I can filter for the Suricata rule group. If a detection involves a particular destination port or source address, those fields are available as data rather than buried inside a paragraph of log text.
The structured event model therefore makes the SIEM considerably more useful than simply forwarding Suricata's raw log file somewhere else.
Repeating the Deployment on Test
After proving the architecture on production, we repeated it on the test workload.
The first step was again to inspect the actual network interface rather than assuming that it matched production. The test server uses ens5, so we changed Suricata's AF_PACKET configuration accordingly.
We installed the same Suricata release, updated the detection rules and validated the configuration before starting the service.
Once Suricata was running, we watched the EVE output.
The test workload produced normal DNS telemetry, confirming that packet capture was functioning. It then produced an actual security detection for inbound traffic from an external source. Suricata identified the source through its DShield rules and recorded the resulting event in EVE JSON.
That gave us confidence that the second sensor was functioning independently rather than simply inheriting assumptions from the production deployment.
We then added the EVE file to the existing Wazuh agent configuration on the test server and restarted the agent.
The Wazuh log collector confirmed that it was analyzing the Suricata file. Finally, we returned to Threat Hunting and filtered for the test agent and Suricata rule group. The SIEM showed a Suricata detection for suspicious inbound traffic targeting PostgreSQL on port 5432.
That detection was particularly relevant because it demonstrated the type of visibility we were looking for. A remote system was probing a database service, and the network intrusion detection layer identified the activity even though there was no requirement for the application itself to generate a security event.
Securing the SIEM
Once the monitoring platform was operational, we had to apply the same security thinking to the monitoring platform itself.
The Wazuh dashboard is an administrative interface. It provides access to security telemetry and administrative capabilities, so exposing it directly to the public internet would create an unnecessary attack surface.
During the initial setup, the dashboard was reachable through the server's public address. That was useful for installation and testing, but it was not the final architecture we wanted. We already had a VPN infrastructure in place, so we used that as the access boundary.
At the AWS security group level, HTTPS access to the Wazuh server was restricted to the public address of the VPN gateway. This meant that a request coming directly from the public internet could not reach the dashboard, while an administrator connected through the VPN could.
We tested both conditions rather than relying solely on the security group configuration. With the VPN disconnected, the dashboard request timed out. After reconnecting the VPN, the dashboard became accessible again.
That gave us a straightforward administrative boundary without requiring the Wazuh dashboard itself to understand our VPN architecture.
Putting NGINX in Front of Wazuh
We also wanted the SIEM to have a proper hostname and TLS configuration rather than requiring administrators to use a raw IP address and a non-standard dashboard port.
We configured NGINX as a reverse proxy in front of the Wazuh dashboard.
The Wazuh dashboard was moved to an internal service port, while NGINX became the externally addressed HTTPS endpoint. Requests to siem.monesize.com terminate at NGINX and are proxied internally to the Wazuh dashboard.
This gives the architecture a clean separation between the public-facing web endpoint and the dashboard service itself.
NGINX also provided a convenient place to manage the TLS configuration.
We installed Certbot and the NGINX integration, requested a Let's Encrypt certificate for siem.monesize.com and allowed Certbot to deploy the resulting certificate into the NGINX configuration. The certificate was issued successfully, and automatic renewal was configured.
The final request path is therefore straightforward. An administrator connects to siem.monesize.com over HTTPS through the VPN. AWS permits the connection because it originates from the approved VPN address. NGINX terminates the HTTPS connection and proxies the request to the Wazuh dashboard on its internal port.
The Wazuh dashboard itself is never directly exposed to the public internet.
Administrative Access
We also created a dedicated Wazuh administrative account rather than relying exclusively on the default admin identity.
The new account was mapped to the appropriate administrator role and tested independently to ensure that it had the expected access.
We retained the default admin account as a fallback. Removing the original administrative path immediately would introduce unnecessary recovery risk, particularly while the deployment was still new.
This is a small operational decision, but it reflects the broader approach we took throughout the implementation. Security controls should improve the system's security without creating avoidable operational failure modes.
Detection Before Automated Blocking
The initial Suricata deployment deliberately focuses on detection.
The internet generates a considerable amount of background scanning and malicious traffic. Some of it will be clearly hostile, some of it will be irrelevant to the workload and some detections will require additional investigation before we decide that they represent a meaningful security event.
Automated blocking therefore needs to come after we understand the detection environment.
For example, Suricata identified suspicious inbound traffic targeting PostgreSQL on the test workload. That is useful information even without an automated block. It tells us that an external system is probing a service that should not necessarily be exposed to arbitrary internet traffic.
The correct engineering response depends on the context. We may need to inspect the network boundary, confirm which services are intentionally exposed, examine the source and destination information, and determine whether the detection represents routine internet scanning or something requiring a stronger response.
Starting with detection gives us that visibility without immediately allowing the security system to make potentially disruptive changes to production traffic.
As we accumulate telemetry and understand the detection patterns, we can make more informed decisions about where automated response is appropriate.
What the Completed Architecture Gives Us
The completed architecture combines several layers of security visibility.
Wazuh provides centralized SIEM functionality, endpoint monitoring, vulnerability detection, event processing and investigation through the dashboard. Its agents provide telemetry from the workloads.
Suricata adds network-level intrusion detection to the application servers where we need that visibility. It inspects traffic on the host network interfaces and generates structured EVE JSON events.
The Wazuh agents collect those events and forward them to the central platform. Wazuh then processes the events so that they can be searched alongside the rest of the security telemetry.
NGINX provides the HTTPS reverse proxy in front of the dashboard, while Let's Encrypt provides the TLS certificate. The VPN provides the administrative network path, and AWS security groups enforce the network-level restriction around the dashboard.
Each component has a defined responsibility, but the important part is the integration between them.
A vulnerability finding in Wazuh tells us about the software running on an endpoint. An endpoint event tells us what happened on that system. A Suricata event tells us what the network sensor observed. The SIEM gives us one place where those different perspectives can be investigated together.
What the First Detections Showed Us
The first network detections reinforced why network telemetry needed to be part of the architecture.
On production, Suricata identified traffic associated with SSH scanning, known compromised or hostile hosts and threat intelligence blocklists. On test, it detected suspicious inbound traffic targeting PostgreSQL on port 5432.
None of these events required a successful compromise to be useful.
A scan against SSH tells us that someone is probing the service. Traffic from an infrastructure source identified by threat intelligence gives us additional context about the origin of a connection. A PostgreSQL probe tells us that an external system is interested in a database service exposed on the network interface.
This is exactly the type of information that endpoint monitoring alone may not provide.
The value of the SIEM is therefore not limited to detecting successful attacks. It gives us visibility into activity that may indicate reconnaissance, attempted exploitation or other behavior that deserves investigation.
That distinction changes the way we think about security monitoring. Waiting for an application to report a compromise is far too late to be the only security strategy. We want visibility into the activity surrounding our workloads as well.
What We Learned About Building the System
The implementation reinforced a few engineering principles that are easy to overlook when deploying security tooling.
The first is that a security tool being installed does not mean that the security control is operational. Suricata had to be configured against the correct network interface, its rules had to be available, its configuration had to validate and its output had to be inspected before we could consider the sensor operational.
The same principle applied to Wazuh. Adding a log collector configuration was not enough. We checked the agent logs to confirm that the collector was actively analyzing the Suricata file, and then we verified the resulting events in the central dashboard.
The second is that security visibility depends heavily on architecture. Suricata can only inspect traffic that reaches the interface on which it is operating. Wazuh can only analyze telemetry that reaches the platform. A SIEM does not magically create visibility into systems or network paths that have not been instrumented.
The third is that centralization makes security information substantially more useful. Suricata by itself can generate excellent network detections, but the events become more useful when they sit alongside endpoint and vulnerability information. Likewise, Wazuh becomes more useful when it can incorporate network detections rather than relying only on host-level telemetry.
The fourth is that the SIEM needs to be treated as production infrastructure. Its dashboard needs access controls. Its network exposure needs to be deliberate. Its certificates need to remain valid. Its administrative identities need to be managed. Its data needs to be protected. Building a security platform and then leaving the platform itself exposed would defeat much of the purpose of the exercise.
Way Forward
The current deployment establishes the foundation rather than completing the security program.
The next stage is to improve the quality of the monitoring rather than simply adding more tools. That means reviewing the detections we receive, understanding which events represent meaningful threats, tuning unnecessary noise and establishing clear response procedures for the events that matter.
There is also room to bring additional security telemetry into the SIEM as the Monesize infrastructure evolves. The architecture already provides a central location for those integrations, so new data sources can be evaluated based on the security questions they help answer rather than added simply because they are available.
We will also continue to evaluate the boundary between detection and prevention. Automated response can be valuable, but it should be introduced where the detection quality and operational consequences are well understood. The fact that Suricata can identify a suspicious connection does not automatically mean that blocking every matching connection is the correct response.
The architecture can also evolve technically. Wazuh supports more distributed and clustered deployments as event volume and availability requirements increase. We do not need that complexity simply because the technology supports it. The current architecture is appropriate for the environment it monitors, and we can introduce additional infrastructure when there is a concrete operational reason to do so.
To cut the story short...
Building a self-hosted SIEM at Monesize was fundamentally about creating a central security observation layer around the company's infrastructure.
Wazuh provides the central platform for endpoint telemetry, vulnerability visibility, security event analysis and investigation. Suricata extends that visibility into network traffic, allowing us to detect reconnaissance and suspicious connections that may never become application-level events. The integration between the two means those network detections become part of the same central security workflow rather than remaining isolated on individual servers.
The implementation also demonstrated why security monitoring has to be treated as an engineering system rather than a collection of installed products. The network interface has to be correct. Detection rules have to be present. Configuration has to be validated. Logs have to be collected. Events have to reach the central platform. Administrative access has to be controlled. Every part of the pipeline needs to be tested rather than assumed to work.
The most useful outcome is the visibility the system now provides. We can identify vulnerable software on monitored workloads, inspect endpoint telemetry and see network reconnaissance against the infrastructure through the same central security platform. The first Suricata detections made that value tangible by showing actual hostile and reconnaissance traffic reaching the workloads.
For Monesize, the SIEM is therefore not simply another infrastructure component. It is becoming part of the operational layer through which we understand the security state of the systems we build and run.
Comments
Post a Comment