limeyard

Deliberately vulnerable Docker lab with a routable DNS estate and machine-readable answer keys per target, scoring scanner precision, recall and scope locally.

The old fleet scored 9 of 9, zero misses, zero false positives. A benchmark that cannot fail cannot detect a regression. Three things were wrong structurally:

• Three of five engines had no ground truth. Every app was 127.0.0.1:70xx, so subdomain enumeration had nothing to enumerate, port scanning was handed its answer, and service fingerprinting never saw a non-HTTP daemon.
• 11 of 101 detection templates had ever fired. The other 90 shipped with no live target of any kind.
• Nothing measured precision. Every target was genuinely vulnerable, so "zero false positives" was unfalsifiable.

Quick start

You need Docker (with the compose plugin) and git. Nothing else.

That clones the lab into ./limeyard, writes its .env with a fresh API token, builds and starts the control plane, then asks what to run. Before anything starts it shows what the selection costs, measured idle on the reference box, against what your machine has free:

Non-interactive: curl ... | bash -s -- --light --yes (or --all, --none, --pick dvwa,juice-shop,estate). Put the checkout elsewhere with LIMEYARD_DIR=/path.

The panel is then at http://127.0.0.1:7000, and the same things by hand:

By hand, without the installer:

Every manifest carries a measured resources block (containers, idle RAM, image disk, idle CPU). The wizard, the panel's selection strip and each target's page sum from it, so the estimate is the same everywhere.

Branches

Two, and only two.

• main is what the installer clones and what you get if you do nothing. It moves by pull request, never by a direct push.
• dev is the default branch and where work lands. Open pull requests against it.

Concepts

Kinds: web api bench cve service estate edge control mobile.

Layout

Control panel

docker compose up -d starts two containers and nothing else: limeyard_control (the limed daemon, holding the Docker socket) and limeyard_ui (the SvelteKit panel). Both bind loopback only. The panel is at http://127.0.0.1:7000 and the raw API at http://127.0.0.1:7099. Both need .env, and LIME_TOKEN in it is mandatory: the panel holds the token server-side and the browser never sees it.

The panel updates itself: limed streams state transitions over SSE, so a target started from the CLI shows up without a refresh. The header carries the host's CPU, RAM and free disk from the same stream, coloured only when they are worth noticing. Every table sorts by clicking a column header; a second click flips the direction.

After a change to the panel or the daemon, rebuild the pair:

To work on the panel against a running daemon without rebuilding the image:

API

Every call except /api/health needs X-Lime-Token.

Networks

Three tiers, because one was the original problem.

• lime-web bridge. Web and API targets, published on 127.0.0.1:70xx.
• lime-lab bridge, 10.66.0.0/16, static IPs, no host binding. A scanner joins this network as a container and sees a real subnet with real hosts and real ports instead of a loopback port list.
• lime-edge the WAF tier, with the origin reachable but absent from DNS.

DNS is authoritative BIND on .test zones (RFC 6761 reserves it). Docker network aliases are deliberately not the source of truth: they never appear in a zone transfer, and would make the topology disagree with the answer key.

Ports

Scoring

Every target ships a truth.yml. The scorer turns findings into precision, recall and F1, per target and per class:

Three things are counted, not one. Recall: did we find what is there. Precision: did we avoid reporting what is not, measured against the negative entries every answer key carries. Scope: what we correctly did not attempt, recorded so nobody re-litigates it every quarter.

The mirage target exists only for the second one. Nothing in it is vulnerable and everything in it looks like it is, so any finding against it is a false positive by construction.

Contributing

CONTRIBUTING.md has the whole of it: what a contribution usually is, the invariants ./lime audit enforces, and why a correction to an answer key is worth more here than a new feature. The short version of the rules is below. Everyone taking part is held to the code of conduct.

Adding a target

Rules that keep the lab clean:

• Bind only the target port, to 127.0.0.1.
• Single-container targets join [lime-web] only. Do not add a per-project network: too many networks and Docker runs out of address pool.
• Targets with a database join [default, lime-web] and put the database on [default] only. Each target gets its own database service and volume.
• Targets that exist to be discovered rather than browsed join [lime-lab] with a static IP and publish no host port.
• upstream is mandatory. lime doctor fails a target without it and the manager refuses to register one. See below.
• limeyard vendors no third-party source. Put repo: and compose: in the manifest and the source is fetched at runtime into <target>/src instead.
• Record what it costs. Start it, let it settle, and ./lime measure <slug> prints the resources block to paste in. The installer and the panel add these up to warn someone before they start it, so a guess here is a lie there.

Trust and isolation

Targets are other people's deliberately vulnerable software, so provenance is recorded rather than assumed, and the runtime is constrained rather than trusted. SECURITY.md has the full picture; the short version:

• Every image is pinned by digest, not a floating tag, to the digest that was pulled and tested here. ./lime pin reports drift.
• Publishers are documented per image: Docker Official Images, project organisation accounts (OWASP, ISC, Traefik, Prometheus), or the author's own namespace. The two weakest links, raesene/bwapp (archived, last rebuilt 2022, no licence) and delfer/alpine-ftp-server (one individual), are named as such.
• Every service runs with no-new-privileges, cap_drop: ALL plus a minimum per-image cap_add, a pid ceiling and a memory ceiling. Database tiers sit on internal: true networks with no route out.
• limed holds the Docker socket, which is root on the host, so it requires a shared secret on every call. A popped target can reach it and learn nothing.
• ./lime audit fails on privileged, host networking, a socket mount in a target, an off-loopback bind, an unpinned image or a missing token.

Nothing here defends against a kernel-level container escape. For that, use a disposable VM.

Attribution

Nearly everything here was written by someone else, and several targets declare no licence at all. So crediting the author is a hard gate, not a convention:

• target.yml carries a required upstream block: author, repo, licence, and the date we last verified it builds. packager records the person who containerised something when that differs from who wrote it.
• Every dashboard card shows the author under the target name, linked to the source, with the licence beside it. A licence of none declared renders as a warning, which is also the do-not-redistribute signal.
• Each target's detail view opens with a credit block, above the vulnerability list, with our answer key clearly separated from upstream's own docs.
• /credits in the UI and ./lime credits list every target, author and licence. ./lime credits --markdown regenerates the section below.

Credits

limeyard runs other people's work. Every target and scenario below was built by someone else unless it says Clickswave.

api

bench

control

cve

edge

mobile

scenario

service

web