DROS-VEP-lite

Open-source, 100% reproducible AI Agent Runtime Security Benchmark & Sandbox Environment (RFC-010 Draft Protocol).

English | 繁體中文

[!TIP] 📚 Academic & Research Citation: If you use this research testbed or benchmark suite in your work, cite via CITATION.cff or see RFC-010 Specification.
🔬 Open Research Infrastructure: Built on the OpenShip containerized substrate, VEP allows researchers to independently swap reasoning models (LLMs), agent frameworks, and defense kernels without vendor lock-in.
🧨 Open Adversarial Falsification Channel is LIVE: We actively invite researchers to challenge and falsify our execution invariants: 👉 Submit a Counterexample. All submissions are triaged against formal criteria.

🧭 Product Positioning: Deterministic Runtime Execution Governance

1. What DROS Is

DROS is a deterministic execution-governance substrate for AI agents and tool-enabled systems.

It establishes an explicit, in-band enforcement boundary between an agent’s decision to act and the system action that follows.

2. What Problem It Solves (Post-Compromise Containment)

Traditional AI security focuses on prompt inspection, guardrails, or post-hoc log observation. When an agent's cognitive layer is compromised (via direct/indirect prompt injection, context hijacking, or tool hallucination), these outer defenses fail silently.

DROS solves the post-compromise confinement problem: even if an agent’s cognitive loop is fully hijacked, its authority to invoke underlying operating system calls, file APIs, network sockets, and enterprise tools remains deterministically bounded.

3. Why DROS Is Intentionally Minimal

Doctrine: "Narrow in responsibility. Deep in enforcement."
DROS deliberately does less.

DROS is an execution-governance substrate, not a general-purpose AI security suite or all-in-one platform. Its responsibility is deliberately narrow: deterministic authorization and interception at the execution boundary.

By keeping the enforcement surface bounded, DROS avoids expanding into adjacent domains:

• Identity, authentication, and credentials remain with enterprise IAM.
• Business orchestration and workflows remain with agent orchestration frameworks.
• Log aggregation and security monitoring remain with SIEM and telemetry stacks.

"Infrastructure doesn’t need to be intelligent. It needs to be dependable."

🏛️ The Three-Domain Architecture Model

To eliminate conceptual ambiguity and separate decision inputs, runtime actions, and integration boundaries, DROS is structured across three distinct dimensions:

[!IMPORTANT] The Architecture Doctrine:
6P defines what DROS must know. (Decision context)
The enforcement layers define what DROS must do. (Enforcement path)
The surrounding infrastructure defines what DROS does not need to replace. (Integration boundary)

DROS deliberately narrows its product responsibility without narrowing its enforcement model.

4. 6P Governance Context (What DROS Must Know)

The 6-Pillars trust model defines the multi-dimensional context that DROS evaluates before permitting any execution. These are decision inputs, not six separate software products:

5. L1–L4 Enforcement Layers (What DROS Must Do)

DROS enforces governance along a unified, in-band execution path across four defense-in-depth layers. These represent stages on the single execution boundary, not four independent commercial products:

1. L1 Boundary Filter: Ingests incoming tool invocations and filters syntactically malformed or out-of-boundary requests.
2. L2 Capability Bound: Enforces $O(1)$ capability bitmasks ensuring the agent possesses explicit, non-escalatable rights for the specific task.
3. L3 Topology Isolation: Confines execution within bounded directory descriptors, process namespaces, and network egress policies.
4. L4 Deterministic GuardVM Enforcement: Sub-microsecond C-ABI binary guard delivering hard stop containment without heap allocations.

6. Existing Enterprise Stack (What DROS Does Not Need to Replace)

DROS is designed to drop into enterprise infrastructures as an execution gate without rip-and-replace disruption:

📊 Post-Compromise Property × Enforcement Layer × Semantic Coverage Matrix

Core Research Finding: Capability isolation, resource sandboxing, formal assurance, and Agent-level execution governance represent distinct security properties. They cannot be collapsed into a single security score, nor can one substitute for another.

* Modeled conditional on capability authority in the modeled execution domain; seL4 enforces capability authority, not abstract Agent task authorization.
** Modeled conditional on tools being explicitly represented as distinct capability endpoints in userspace architecture.
*** Modeled conditional on target resource/device being represented as a bounded memory/MMIO capability object.
**** Enforced strictly within the configured preopen directory descriptor boundary.
***** Models revocation of derived capability copies via seL4_CNode_Revoke(), not abstract Agent token revocation.
****** Under pure CHERI ISA (CHERI_PURE_ISA_CAPABILITY_MODEL), reported as UNSUPPORTED. Under CHERI_CHERIBSD_RUNTIME, CheriBSD OS provides temporal heap sweep.

For complete formal definitions, see Property Enforcement Coverage Matrix (Full Document).

🤝 Want to Evaluate Your Substrate? (Substrate Contribution Protocol)

Golden Rule: "Add substrates, not benchmark exceptions."

VEP is designed as an open, implementation-independent testbed. If you develop an execution substrate (capability operating system, sandbox runtime, hardware architecture, microkernel, or formal model), you can integrate and evaluate it in 7 standardized steps:

1. Implement Adapter: Create a new adapter under substrates/<your_substrate>/adapter.py inheriting from BaseSubstrateAdapter.
2. Declare Execution Profile: State your substrate's architectural boundary (E1_APPLICATION_GATEWAY, E2_SANDBOX_RUNTIME, E3_OS_KERNEL, E4_HARDWARE_ISA, or E5_FORMAL_ASSURANCE).
3. Map Semantic Scope: Explicitly declare whether each property enforcement is NATIVE, PROFILE, APPLICATION, or UNSUPPORTED. Never inflate substrate semantics.
4. Run Canonical Scenarios: Execute standard test scenarios without modifying scenarios:
   root@kitploit:~

   python vep.py benchmark post-compromise --substrate <your_substrate>
   

5. Produce Canonical Evidence: Output execution records to reports/benchmarks/post_compromise/.
6. Verify Deterministic Replay: Ensure 100% decision and parameter match across identical runs:
   root@kitploit:~

   python vep.py replay
   

7. Submit Results: Open a PR with your adapter, unit tests, and generated evidence logs.

📑 Canonical Scenario & Evaluation Registry

VEP unifies evaluation across four foundational dimensions: Scenario $\to$ Security Property $\to$ Substrate Capability $\to$ Composition Gain.

📖 Full Formal Registry: See docs/research/SCENARIO_REGISTRY.md for canonical scenario definitions, threat models, expected outcomes per substrate, evidence requirements, and deterministic replay contracts.

Traditional AI security benchmarks measure prompt toxicity or rely on out-of-band proxy monitors that cannot prevent post-compromise execution escapes. VEP combines OpenShip containerized composability with a system-level in-band execution governance loop:

⚡ 5-Minute Research Experiment (Reproduce in 60 Seconds)

Evaluate post-compromise containment on your local machine with zero proprietary dependencies:

Inspect interactive audit logs and evidence artifacts in real time at http://localhost:8080.

🎯 Cross-Domain Research Testbed Matrix

VEP provides multi-domain evaluation fixtures reproducing 2026 real-world security incidents across enterprise cloud, on-device mobile, and physical robotics:

🏛️ Scientific Evidence & Benchmark Index

📖 Research Note: How to Break Your AI Agent in 5 Minutes (And Rebuild It Stronger)
🛂 Open Agent Passport SDK: libdros-id (RFC-010 W3C DID & Ed25519 SDK)
🧭 Reading Guide to Trajectory: DROS Trilogy Reading Guide

🔬 Research Discovery & Academic Scope

VEP evaluates whether Agent execution authority remains constrained after Agent compromise, with particular emphasis on runtime enforcement, execution-boundary containment, revocation, provenance, and reproducible security evaluation.

This repository and protocol may be relevant to researchers, evaluators, and system architects studying:

• Agent Execution Authority & Governance: Formalizing the transition from non-deterministic agent cognition to bounded physical execution.
• Agent-to-Execution Attribution: Cryptographically linking agent intents, authorization tokens, and physical system calls.
• Runtime Enforcement for Autonomous AI Agents: Deterministic in-process C-ABI / kernel interception versus probabilistic semantic guardrails.
• Post-Compromise Agent Security: Containing unauthorized system effects when the agent reasoning layer is assumed fully compromised.
• Execution-Boundary Security: Preserving containment invariants under multi-hop confused deputy and prompt-injected delegation chains.
• Agent Capability & Dynamic Authorization: Fine-grained capability bitmask evaluations ($O(1)$ constant time) and zero-window RCU policy revocation.
• Deterministic Runtime Enforcement: Enforcing fail-closed containment under adversarial resource starvation and syscall flood conditions.
• Agent Security Benchmarks & Testbeds: Providing reproducible, multi-track testbeds across Cloud B2B, Physical Robotics/Drones, and Mobile on-device SDKs.
• Execution Provenance & Cryptographic Audit: Maintaining append-only, tamper-evident Merkle hash chains supporting technical traceability relevant to EU AI Act / NIST SP 800-207 requirements.

💡 Conformance & Substrate Decoupling:
DROS is not required for VEP conformance. VEP defines an open, vendor-neutral evaluation protocol; DROS is provided as one concrete executable reference substrate for demonstrating, benchmarking, and validating VEP experiments.

⚡ Quick Start (60 Seconds)

🏢 B2B Multi-Enterprise Supply Chain Mode (Federated Defense)

Want to evaluate cross-enterprise Agent interactions and supply chain attacks?

• Corp-Alpha (Core Enterprise / LLM Orchestrator): Operates GuardVM at localhost:8082
• Corp-Beta (Third-Party External Repository Supplier): Operates GuardVM at localhost:9082
• EP4 Scenario (ATS-004: Federated Cross-Enterprise Supply Chain Poisoning Simulation): Simulates an autonomous Agent retrieving an unverified dataset/model from an external repository supplier. The embedded Indirect Prompt Injection (IPI) attempts to hijack the agent to exfiltrate Corp-Alpha's financial secrets. Even with valid OAuth tokens, Corp-Alpha's GuardVM intercepts the cross-enterprise attack at the C-ABI boundary in <500ns! (Note: Synthetic evaluation fixture inspired by industry threat patterns; does not reference or implicate any specific real-world corporate incident).

3. Open Interactive Web Dashboard

Navigate to http://localhost:8080 in your browser

🧨 Submit a Counterexample (Open Falsification Protocol)

DROS-VEP adheres strictly to the principle of Open Adversarial Falsification. We invite the academic community, security researchers, and engineers to submit reproducible counterexamples that violate our empirical core invariants:

Within the explicitly instrumented operation classes $X_{\text{covered}}$, whenever Auth_E(x) = DENY:
Unauthorized execution count is zero ($Exec_{\text{unauthorized}} = 0$) and observable state drift is zero ($\Delta S_{\mathcal{S}_{\text{obs}}} = 0$).

Criteria for a Valid Counterexample

• Deterministic Reproducibility: 100% reliably reproducible under the official DROS / PGM containerized environment.
• Scope Alignment: Falls within the instrumented operation classes $X_{\text{covered}}$ ($X_{\text{fs}} \cup X_{\text{proc}} \cup X_{\text{net}} \cup X_{\text{ipc}}$) or demonstrates an uninstrumented execution escape path.
• Actionable Evidence: Includes concrete reproduction steps, environment specs, expected vs. actual behavior, raw syscall traces, WAL diffs, or replay scripts.

How to Submit

1. Use our Counterexample Issue Template (or open a GitHub Issue labeled counterexample).
2. Provide all environment metadata and reproduction steps.
3. Submissions will be triaged publicly, evaluated against the formal invariants, and recorded in the permanent evaluation matrix.

Current Status (as of 2026-08-28 Benchmark Record): Valid Counterexamples = 0

Note: Even if a submission is ultimately triaged as "Out of $X_{\text{covered}}$ Design Scope" or an environmental artifact, we deeply value boundary clarification reports and will acknowledge contributions publicly.

💡 Why Existing AI Benchmarks Are Not Enough

Most AI benchmarks measure LLM intelligence, coding skills, or prompt toxicity. DROS-VEP measures a completely different dimension: Runtime Tool-Call Authorization & Privileged Execution Governance.

🏗️ Testbed Architecture & Evaluation Ecosystem

DROS-VEP Lite's OpenShip-based testbed composes OpenAI's official Terraform Provider (for organization/project provisioning) alongside DROS runtime defense, simulating a realistic enterprise deployment topology for execution-boundary testing:

In this evaluation topology, while OpenAI's Terraform Provider establishes the Control Plane Provisioning baseline (Projects, IAM, Rate Limits), DROS GuardVM is evaluated as the Runtime Execution Defense layer — validating that when an agent holding legitimately provisioned credentials is hijacked via Indirect Prompt Injection (IPI), unauthorized tool calls are deterministically intercepted at the C-ABI boundary.

💡 Why Traditional Security (WAF/Keycloak) Is Blind to ATS Scenarios

In an indirect prompt injection attack (ATS-001), the hijacked AI Agent possesses a valid Keycloak JWT token. When the agent queries /api/erp/finance, WAF inspects the request: "Valid HTTPS, clean JSON, valid OAuth token. Access Granted!"

Traditional WAFs see a 100% legitimate user making a clean REST API call. The attack is hidden inside the LLM Semantic Context. This is why DROS PEP/PDP is required at the tool execution boundary.

🎯 Threat Scenarios & Research Fixtures (RFC-010 Standard Matrix)

[!NOTE] Synthetic Benchmark Disclaimer
All threat scenarios in this repository (ATS-001 through ATS-005, AS-001 through AS-005, and PC-001 through PC-010) are synthetic, architectural evaluation fixtures. They are designed exclusively to model and evaluate runtime system-call boundaries, tool authorization contracts, and post-compromise containment invariants mapped to MITRE ATLAS categories. They do not simulate, represent, or attribute actions to any specific commercial platform, model provider, or real-world organization.

VEP provides standardized, synthetic evaluation fixtures reproducing critical post-compromise threat models, mapped directly to MITRE ATLAS:

🧪 Engineer Proof of Integrity: Dissect & Replay

Engineers don't trust static dashboards. They ask: "If I unplug your guard, does the result actually change?"

1. Counterfactual Control Group (Disable DROS Guard Toggle)

Open http://localhost:8080 and check ☑ Disable DROS Guard (Debug Mode):

• Guard Active (Normal): 100% Defense Integrity (AS-001 ~ AS-005 | Decision: DENY | Pass Rate: 100%).
• Guard Disabled (Control Group): PEP bypasses interception. The agent penetrates target endpoints. Pass rate plummets from 100% ===> 0% (LEAKED).

2. Deterministic Replay Engine (benchmark/replay.py)

Replay any historical audit log or evidence artifact package deterministically:

📊 Benchmark Methodology & Operational Distinction

To ensure scientific transparency, VEP explicitly distinguishes between two fundamentally different execution paths:

1. Full Cryptographic Policy Evaluation Path (P50: 26.1 μs):
   • Evaluates 3-tier certificate validation (Root CA -> AIA -> Leaf DIT Token), capability bitmask matching ($O(1)$), and structured audit attestation.
   • Median decision speed: 26.1 μs (P99: 41.2 μs, Stddev: ±3.4 μs, $N=10,000$).
2. Emergency Fail-Closed Panic Path (<500 ns):
   • Short-circuit hardware/C-ABI boundary abort triggered when an unmapped tool call, memory fault, or revoked token attempts immediate execution.
   • Execution abort latency: <500 ns.

🔬 Reproducibility & Research Artifact Harness

To support independent scientific reproduction without corporate telemetry or external dependency:

• Hardware & OS Baseline: x86_64 or ARM64, Linux Kernel $\ge 5.15$, Docker Engine $\ge 24.0$, Python 3.10+.
• Deterministic Benchmark Command:
  root@kitploit:~

  python scripts/run_cybermes_crucible.py --reproduce --iterations 1000
  

• Raw Empirical Artifacts: Raw latency measurements, audit logs, and replay traces are systematically persisted in:
  • reports/evidence/
  • reports/CYBERMES_POST_COMPROMISE_REPORT.md
• Cryptographic Trace Replay:
  root@kitploit:~

  python benchmark/replay.py --trace-dir reports/evidence/
  

🏅 RFC-010 Draft Protocol Conformance Harness

Third-party AI Agent Frameworks (OpenAI Agent SDK, LangGraph, CrewAI, AutoGen, OpenClaw) can evaluate their runtime security across 3 certification tiers:

• Level 1 (Core): Identity Token (DIT) + PEP Tool Interception + Structured Audit Logging.
• Level 2 (Enterprise): Policy Explainability (Policy ID) + Evidence Package (SHA-256 Digest) + Multi-Agent Role Isolation.
• Level 3 (High Assurance): Cryptographic Attestation + Tamper Detection + Deterministic Replay.

ℹ️ Disclaimer: The included conformance harness validates implementations against the RFC-010 Draft specification. Passing the test indicates conformance to this draft, not certification by an independent standards body.

🏴‍☠️ Autonomous Post-Compromise Crucible (Cybermes Integration)

Core Premise: Control-Execution Separation: Agent Compromise $\neq$ Execution Authority.

When an AI Agent is subverted via spear-phishing or compromised dependencies, traditional perimeter defenses (WAF/IAM) fail because the attacker inherits legitimate API credentials. DROS enforces deterministic execution containment at the C-ABI binary boundary.

📊 3-Phase Scientific Benchmark Summary

• Read the full technical benchmark report: CYBERMES_POST_COMPROMISE_REPORT.md
• Inspect scenario details & capability matrix: scenarios/ATS-005

👥 Open Source & Community Resources

DROS-VEP Lite is released under Apache 2.0 to provide an open, transparent, and fully reproducible benchmark evaluation environment for the global AI safety community:

• 🧪 Evaluation Sandbox (DROS-VEP Lite): Freely available to clone, test, and design custom security benchmark scenarios. Refer to Quick Start (60 Seconds) to run the RFC-001 suites immediately.
• 🛡️ Local Execution Guard (Reference Substrate): For independent developers and researchers seeking local execution-boundary protection against untrusted tool calls and prompt injection, access the Open Source Reference Tools.
• 🌐 Scientific Governance & Research: For detailed formal theorems, architectural whitepapers, and extended benchmarking artifacts, explore the Technical Foundations & Benchmark Publications below or visit dr-os.io.

📜 Technical Foundations & Benchmark Publications

📚 Core Publications, Trilogy & DOI Citations

If you reference our zero-trust runtime governance evaluation or use DROS-VEP Lite in your security research, please cite our published peer-reviewed papers on Zenodo:

• 📖 DROS Trilogy Reading Guide (導讀 Technical Note): An Agent Runtime Operation Substrate
  • DOI: 10.5281/zenodo.22114036 | Zenodo Record: zenodo.org/records/22114036
• 🏛️ DROS-6P: A Unified Deterministic Runtime Governance Architecture Closing the Six Fundamental Trust Boundaries of Enterprise AI Agents: Specification Overview (README)
  • DOI: 10.5281/zenodo.21833970 | Zenodo Record: zenodo.org/records/21833970
• 🏛️ DROS 4-Layer (v4.0) Deterministic Runtime Substrate & Adversarial Validation: Paper (EN) | Paper (ZH) | Download PDF via Zenodo
  • DOI: 10.5281/zenodo.21755653 | Zenodo Record: zenodo.org/records/21755653
• 🏛️ DROS 4-Layer (v3) Defense-in-Depth Architecture for Autonomous AI Workloads
  • DOI: 10.5281/zenodo.22092008 | Zenodo Record: zenodo.org/records/22092008
• 🏛️ DROS-PGM: A Deterministic Post-Compromise Execution Containment Substrate (v2.0): Paper (EN) | Paper (ZH) | Download PDF via Zenodo
  • DOI: 10.5281/zenodo.21903687 | Zenodo Record: zenodo.org/records/21903687
• 🌐 DROS-WebMCP: A Cryptographically Attributable Execution Governance Layer for the Agentic Web: Open Governance Draft (DWGR-8)
  • DOI: 10.5281/zenodo.22290238 | Zenodo Record: zenodo.org/records/22290238
• 📱 Post-Compromise Security for Autonomous Mobile Agents: Paper (EN) | Paper (ZH)
  • DOI: 10.5281/zenodo.22253147 | Zenodo Record: zenodo.org/records/22253147
• 🛸 Post-Compromise Security for Physical AI: Autonomous UAVs: Paper (EN) | Paper (ZH)
  • DOI: 10.5281/zenodo.22254372 | Zenodo Record: zenodo.org/records/22254372
• 🧭 Reading Guide to the DROS Research Trajectory (v2.0): Guide (EN) | Guide (ZH)
  • Permanent Record: zenodo.org/records/22255275

📖 Whitepapers & Protocol Specifications

• 📖 Full Whitepaper (English v2.0): Zero-Trust Execution Governance for Autonomous AI Workloads (DROS 4-Layer Paradigm)
• 📖 完整白皮書 (繁體中文 v2.0): 自主型 AI 工作負載的零信任執行治理 (DROS 四層防禦縱深架構)
• ⚡ 4-Page A4 Executive Summary (HTML): Fast visual summary for CISOs & Security Researchers
• 📋 RFC-010: DROS-VEP Specification Protocol: Open Agent Security & Threat Scenario Protocol

❓ Frequently Asked Questions (FAQ)

Why does VEP use open-spec policy representations rather than compiled policy.bin binaries?

VEP Lite is engineered as a human-readable, open-spec evaluation sandbox (RFC-010) to allow security researchers, CISOs, and developers to easily audit policy rules, inspect threat scenarios, and conduct red-teaming without proprietary compiled binaries.
In DROS Enterprise Production, policies are compiled by VajraCompiler into cryptographically signed, immutable, lock-free C-ABI binary microkernels (policy.bin) with zero-heap memory allocation and anti-reverse-engineering seals.

Will PGM's strict $\mathcal{O}(1)$ Bitmap mechanism cause high false positives and block legitimate business workflows (Over-Blocking)?

No. PGM is fundamentally engineered to guarantee high business availability while enforcing zero-trust execution.
Unlike heuristic WAFs or probabilistic LLM guards that rely on fuzzy regex pattern matching (which often mistake benign input for attacks), PGM operates on Multidimensional Positive Capability Bitmasks (正向能力白名單矩陣):

1. Positive Capability Inclusion (Not Heuristic Guessing): PGM assigns fine-grained capability vectors (Role $\times$ Tool $\times$ Method $\times$ Resource Scope). Legitimate operations matching the agent's designated task evaluate to bitwise 1 (Pass) in a single CPU cycle ($26.1\mu s$), resulting in 0% false positive blockage on valid business paths.
2. Graduated Enforcement (Progressive Gates): For sensitive or cross-boundary operations (e.g., large payouts, confidential record exports), PGM does not crudely terminate the entire connection. Instead, it triggers In-Band Dynamic Redaction (18-PHI Masking) or Human-in-the-Loop (HITL) Soft Suspension, allowing standard workflows to proceed securely without business disruption.
3. Sub-Millisecond Zero-Downtime RCU Policy Tuning: If business requirements evolve or new endpoints are onboarded, security operators can update policies via background shadow compilation in <1ms. The master pointer is updated via lock-free RCU atomic swap with zero downtime and zero traffic stalls.

🔒 Patent & Intellectual Property Notice

The deterministic runtime governance architecture, in-band C-ABI interception mechanism, and zero-heap execution boundaries are protected under U.S. Provisional Patent Application No. 64/111,973 (Patent Pending). All commercial deployment rights are reserved by Top Celestial Company Ltd.

📄 Benchmark Harness License

The evaluation benchmark harness scripts and RFC-010 scenario definitions are released under Apache 2.0 for academic reproducibility and independent verification.