Home SOC Suite — Night's Watch

A personal Security Operations Center (SOC) built from scratch during an Anwendungsentwicklung retraining program. This suite monitors a Windows endpoint for network anomalies, resource spikes, process execution, registry changes, Security Event Log activity, scheduled task changes, DNS queries, and power events — logging everything to structured files for weekly analysis and correlation through a Python engine.

Single-page Flask dashboard. All 12 collectors + correlation engine active, live resource monitor, outbound connection feed, collector event tiles, and streaming engine alerts.

Background

Built as a practical learning project alongside formal IT retraining, this suite applies real SOC concepts — baseline collection, anomaly detection, severity classification, log aggregation, and cross-source correlation — to a personal Windows machine. The design process was adversarial from the start: each phase was stress-tested against a red team analysis before the next was built, with 48 correlation rules defined from those findings driving the architecture of the Python engine.

Roadmap

Development is tracked in roadmap.md — current state, Phase 9 Forensic Engine plans, and longer-term hardening goals.

Dashboard Setup

The dashboard runs as a Flask server and requires Administrator elevation for collector process detection. Create a shortcut once after cloning:

1. Right-click the Desktop → New → Shortcut
2. Set the location to:
   root@kitploit:~

   pwsh.exe -ExecutionPolicy Bypass -WindowStyle Minimized -File "C:\path\to\SOC\Dashboard\Launch_Dashboard.ps1"
   

   Replace C:\path\to\SOC with your actual install path.
3. Name it: SOC Dashboard
4. Right-click the shortcut → Properties → Advanced → check Run as administrator
5. Click OK

The shortcut starts Flask, waits for it to be ready, then opens the browser automatically. If Flask is already running it skips startup and opens the browser directly. Status dots will show grey if the shortcut is run without Administrator elevation.

Dashboard Lifecycle

The dashboard enforces a clean start/stop sequence so the Auditor bookends (morning integrity check, evening hash chain validation) always run in order.

1. Pre-start — Suite Not Started

All collector heartbeats are red (health files missing or stopped). The Shutdown Dashboard button is hidden — the dashboard will not allow shutdown until End Day has run.

2. Start Day — Suite Running

Start Day runs the morning Auditor bookend (SHA256 log verification, archive hash chain validation) and launches all 12 collectors plus the correlation engine. Heartbeats turn green as each collector reports in.

3. End Day — Running Evening Auditor

End Day stops all collectors in sequence and runs the evening Auditor bookend. Collector dots stay green while the sequence runs; the status line shows Running Auditor Evening…

4. Post-shutdown — Safe to Close

Once the evening audit completes, all collector dots go red, the status line reads Evening audit complete. Safe to shut down., and the Shutdown Dashboard button appears — clicking it terminates the Flask process cleanly.

The browser tab must be closed manually (browsers do not expose a close API to external processes).

Live Alerts

Collector event tiles show the most recent severity-tagged events per source. The Engine Alerts feed streams correlated findings from the Python engine in real time, with rule number, confidence score, full evidence chain, and SHA256 evidence hash.

Example: Rule 35 (Raw Disk Read) firing on powershell.exe opening a raw handle to \Device\HarddiskVolume3, and Rule 31 (DLL Load Anomaly) flagging Defender DLLs loading from a user-writable path. Both with full process provenance and file hashes.

Architecture

The suite follows a three-layer architecture:

Collection Layer — PowerShell collectors that run continuously in the background, each monitoring a specific data source and writing structured, severity-tagged log entries. All require Administrator elevation.

Analysis Layer — PowerShell analyst scripts that parse collected logs and generate weekly summary reports. Run as standard user.

Correlation Engine (Phase 7 — complete) — Python engine that ingests all collector logs, normalises events to a common schema, and runs rule-based and risk-scored correlation across all data streams. Three programs:

Scripts

Collectors

Analysts

Correlation Rules

Defined through multi-round red team analysis (see Documentation). Implemented in Phase 7.

Severity System

All collectors use a consistent four-level severity system:

Log Structure

All logs follow a consistent format for Python parser compatibility:

[yyyy-MM-dd HH:mm:ss] [SEVERITY] Event details

Repository structure:

home_SOC_suite/
├── Analysts/           — PowerShell analyst scripts
├── Collectors/         — PowerShell collector scripts
├── Dashboard/          — Flask single-page SOC dashboard (Phase 8)
│   ├── app.py              — Flask server, all routes, process detection, DB reads
│   ├── Launch_Dashboard.ps1 — Launcher: checks if Flask running, starts it, opens browser
│   ├── templates/
│   │   └── index.html      — Single-page dashboard layout
│   └── static/
│       ├── style.css       — Dark terminal aesthetic, severity colours, bar graph tiers
│       └── dashboard.js    — All polling, Start Day / End Day sequence, session alert counter
├── Engine/             — Python correlation engine
│   ├── engine.py           — Main loop and orchestration
│   ├── log_parser.py       — Collector log ingestion
│   ├── normalizer.py       — Event normalisation to canonical schema
│   ├── correlator.py       — Correlation rules engine
│   ├── alert_manager.py    — Alert deduplication, flood detection, log writing
│   ├── db.py               — SQLite operations (batch ingest, query, retention)
│   ├── health_db.py        — Heartbeat SQLite store (collector_status, heartbeats tables)
│   ├── config.py           — All thresholds and time windows in one place
│   ├── test_parser.py      — Log parser test suite (7 tests)
│   ├── test_normalizer.py  — Normaliser test suite (32 tests)
│   └── test_correlator.py  — Correlator test suite (135 tests)
└── Documentation/      — Research documents, red team analysis, and dashboard screenshots

Generated at runtime (not tracked in repository):

Logs/               — Active collector log files
│   └── Archives/   — 7-day rotated log archives
Reports/            — Weekly analyst report output
Config/             — Baseline JSON files and Sysmon config
Engine/hocsoc.db    — SQLite operational database
Engine/hocsoc_health.db — Heartbeat and collector status store (engine correlation use only)

Supported OS Languages

Steward automatically detects the correct CPU performance counter path for the following OS languages:

To add support for another language, add the localized counter path to the $PathsToTry array in the Get-WorkingCounterPath function in Steward.ps1.

Documentation & Research

The Documentation/ folder contains the full research and planning record that drove the design of this suite. The suite was built adversarially — each layer was planned against a threat model before it was built, and stress-tested against a red team analysis before the next layer was started.

Key Features

• Adversarial design — 48 correlation rules defined through multi-round red team analysis before the Python engine was built. Detection logic is grounded in real attack chains, not hypothetical scenarios
• Behavioral baseline collection — all collectors build baselines over time, enabling anomaly detection relative to observed normal rather than static rules
• Geolocation monitoring — outbound connections, inbound connections, authentication source IPs, and DoH connections tracked by country with TELEMETRY_GAP alerting on geo failures
• Process fingerprinting — every network, port, process, and authentication event logged with full process path, detecting masquerading and LOLBin attacks
• Suite self-protection — Warden monitors all scripts and configurations via SHA256 FIM with a three-layer protection chain: manifest integrity, external scheduled task, and CityGuard task monitoring
• Zero-Trust whitelisting — all whitelists start empty. 30-day baseline collection precedes any exception grants
• Severity-tagged logging — structured log entries designed for Python parser compatibility and future SIEM integration
• 7-day log rotation — automatic archiving with SHA256 verification and archive hash chain integrity checks at morning/evening bookends via Auditor.ps1
• Multi-language Windows compatibility — performance counter paths auto-detected across 8 OS languages at startup
• Single-page SOC dashboard — Flask-based live visibility replacing all individual PowerShell windows. Includes suite launcher (Start Day / End Day), live Steward resource monitor, Sentinel connection feed, three-state heartbeat dots (green active / amber not started / red down — read directly from collector health JSON files, independent of engine state), SUSPICIOUS/CRITICAL event cards, scrollable engine alert feed, session alert counter, and one-click weekly report generation. End Day stops all collectors and runs the evening Auditor bookend — a Shutdown Dashboard button then appears to terminate the Flask process cleanly. The browser tab must be closed manually (browsers do not expose a close API to external processes)

Current State — Phase 8 Complete

The Python correlation engine is built and running live against all 12 collectors, including SysmonWatcher (Sysmon kernel-level events).

Engine pipeline:

log_parser.py → normalizer.py → correlator.py → alert_manager.py

All ingest, normalisation, and correlation steps use batch SQLite transactions. 174 tests across three test suites — all passing.

Rules implemented: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 16, 17, 18, 21, 22, 41, 42, 43, 44, 45, 46, 47, 48 (Tier 1 single-event and Tier 2 windowed correlation) and Rules 28–39 (Sysmon kernel events via SysmonWatcher). Rules 12, 14, 15, 20 deferred to Tier 4 (require 30-day baseline data). Rules 23, 24, 25 blocked pending specialised collectors.

Detection model: Hybrid — deterministic rules for clear attack chains, risk scoring for cumulative weak signals, visibility alerts for collector silence and ingest quality degradation.

Next phases:

See roadmap.md for the full development timeline.

Known Noisy Alerts — Before Hardening

On a fresh install, several alert types will fire regularly until machine-specific hardening is applied. These are expected and do not represent real detections.

Machine-specific hardening (trusted process lists, path whitelists, known-safe entries) is applied locally and is not included in this repository.

Notes

• Set $RootPath in each script to the folder where you installed the SOC Suite. Default is Desktop\SOC
• Scripts require PowerShell execution policy set to Bypass
• Collector scripts require Administrator elevation for full system visibility
• Analyst scripts run as standard user
• Geolocation provided by ip-api.com free tier
• DNS Client event log must be enabled before running Bloodhound.ps1: wevtutil sl "Microsoft-Windows-DNS-Client/Operational" /e:true

Learning Context

Built during Fachinformatiker Anwendungsentwicklung retraining, Germany 2026. Targeting a career in cyber defense and threat intelligence. Built with AI assistance as a learning tool. Every design decision, debugging session and architectural choice was driven by the developer with full understanding of the underlying concepts.