Project Structure

Setup

Usage

Run Full Experiment Suite

This runs all experiments and auto-generates charts:

• TruthfulQA main experiment (100 samples, mixed poisoning)
• Per-strategy experiments (100 samples each: injection, contradiction, entity swap, subtle)
• FEVER dataset experiment (100 samples)
• Ablation study (5 weight configurations, 60 samples each)

Other Experiment Options

The interactive prompt lets you select:

• LLM: Llama 3.3 70B (primary), Qwen 3.5 35B or Mistral 7B Instruct (comparison)
• Embedding: all-MiniLM-L6-v2 (local) or snowflake-arctic-embed2 (API)
• K: Retrieval depth — K=3 (faster) or K=5 (standard, default)

You can also pin the generator non-interactively via the LLM_MODEL environment variable (matching configs/config.yaml and run_experiment.py's MODEL_DESCRIPTIONS):

Secure-Coding SDLC Use Case

Reproduce the secure-coding assistant experiment (40 OWASP/CWE rules) from the project root. It reuses the existing ExperimentRunner + PoisonedDatasetGenerator and writes results to data/experiments/seccode_*.json:

Regenerate Charts Only

Run Tests

Key Results

Primary Results (Llama 3.3 70B + all-MiniLM-L6-v2, TruthfulQA, K=5, 100 samples)

Naive always-trust baseline: 85% (TruthfulQA), 85% (FEVER). FEVER underperforms the baseline — the Trust Index requires domain-specific calibration for short factual claims.

With 95% confidence intervals (Wilson for proportions, percentile bootstrap B=20,000 for F1), the primary TruthfulQA mixed result is: accuracy 91% (CI 83.8–95.2), recall 40% (CI 19.8–64.3), F1 57.1% (CI 25.0–80.0), Δ=0.225. Intervals are wide because each run has only 15 poisoned samples, so we report them to avoid overstating precision.

Per-Strategy Detection (TruthfulQA, 100 samples each)

2×2 Factorial Grid (K=3, TruthfulQA, 100 samples)

Key findings:

• Embedding invariance for Llama: both embedding models yield identical detection results — Trust Index is LLM-driven, not retrieval-driven
• Qwen generation-style problem: verbose/hedged outputs reduce NLI entailment for clean samples → 71% accuracy, 14 percentage points below the 85% naive baseline
• K sensitivity null result: K=5 produces identical detection performance to K=3 for Llama 3.3 70B — trust score separation changes by only 0.015; detection is bottlenecked by attack strategy difficulty, not retrieval depth

Secure-Coding SDLC Use Case (OWASP/CWE)

A software-engineering application of the agent: a secure-coding assistant that retrieves from a curated knowledge base of 40 rules drawn from the OWASP Top 10 and CWE (e.g. parameterized queries for SQL injection, adaptive password hashing, TLS configuration). A developer query retrieves guidance that the Evaluation Agent screens before the LLM emits a recommendation. We poison 30% of the rules with the five strategies as security payloads (e.g. a spurious CORRECTION: directive, a swapped CWE identifier).

Detection by strategy (Llama 3.3 70B + all-MiniLM-L6-v2, K=5):

• Injection is blocked near-perfectly (F1 92.3%, Δ=0.54): the agent reliably catches overt attempts to insert unsafe advice (e.g. disabling input validation) before they reach the developer.
• Entity swap rises to 58% recall (vs. 0% on open-domain TruthfulQA): altering a structured CWE identifier introduces cross-document inconsistencies the NLI signals can catch.
• Contradiction and subtle manipulation still evade (0% recall): a quietly weakened recommendation carries no surface artifact and passes review — the dangerous, low-visibility case in a security workflow.

Three-LLM Comparison & Threshold Independence

Beyond the τ=0.5 operating point, the Trust Index has strong threshold-independent signal across three LLMs (pooled mixed-strategy runs):

• Generation style > model size: Mistral 7B outperforms Qwen 3.5 35B despite being ~5× smaller — Qwen's hedged, verbose answers depress NLI entailment.
• τ is an LLM-specific hyperparameter: the three models need three different optimal thresholds (0.71 / 0.58 / 0.43). Calibrating τ on clean-only scores restores Qwen from 65.5% → 74.5% accuracy by cutting false positives.
• All three sit well above chance (ROC-AUC > 0.70), so Qwen's weak accuracy at τ=0.5 is a thresholding artifact, not an absence of signal.

Stability & Overhead

• Run-to-run variance is low: over 5 independent repeats the mixed configuration scores 90.6 ± 0.5% accuracy and 56.1 ± 1.3% F1; injection is invariant at 99.0 ± 0.0%. Verdicts are driven by retrieved evidence, not by single-generation wording.
• Overhead: evaluation adds ≈14.7 s/sample (≈17× over baseline RAG), dominated by up to 20 CPU NLI passes at K=5. This suits batch/asynchronous use (e.g. a code-review/CI gate); GPU inference is projected to cut this below 2 s.

Tech Stack

• LLMs: Llama 3.3 70B (primary), Qwen 3.5 35B and Mistral 7B Instruct (comparison) — FARMI cluster, Tampere University
• NLI Model: facebook/bart-large-mnli
• Embeddings: all-MiniLM-L6-v2 (384-dim), snowflake-arctic-embed2 (1024-dim)
• Vector Store: FAISS (CPU)
• Datasets: TruthfulQA, FEVER (HuggingFace), and a secure-coding KB of 40 curated OWASP Top 10 / CWE rules
• Framework: Python 3.10+, PyTorch, Transformers, LangChain

Trust Index Formula

A non-linear dampener applies when poison probability exceeds 0.7: delta = 1 - 0.4 * (P - 0.70) / 0.30 — at P=1.0, trust is reduced by 40%.

Contributions

• An autonomous Evaluation Agent that operates as defensive middleware inside the RAG inference loop
• A five-signal poison detector with relevance-weighted aggregation (linguistic, structural, intra-/cross-document NLI, semantic-outlier)
• A composite Trust Index with a non-linear dampener for high-contamination contexts
• Characterization of a per-strategy detection hierarchy (injection solved → entity swap undetected)
• Evidence that generation style matters more than model size, plus a per-LLM threshold calibration method
• A software-engineering secure-coding (OWASP/CWE) SDLC use case
• A reproducible framework, attack generator, and experiment release

Paper

• Conference paper — Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems, ICSEA 2026. LaTeX sources in [Conference_ICSEA2026/](https://github.com/gpt-laboratory/trustworthyrag/blob/HEAD/Conference_ICSEA2026/).

The compiled PDF is intentionally not included in this release; build it from the LaTeX sources.

Authors & Contact

ICSEA 2026 conference paper authors — Tampere University (TUNI).

Balkrishna Giri, MSc Researcher, Faculty of Information Technology and Communication Sciences, Tampere University
Email: [email protected], [email protected]

Md Toufique Hasan, Doctoral Researcher, GPT Lab, Faculty of Information Technology and Communication Sciences, Tampere University
Email: [email protected]

Co-authors — Faculty of Information Technology and Communication Sciences, Tampere University:

• Dr. Jussi Rasku — [email protected]
• Dr. Muhammad Waseem — [email protected]
• Professor Dr. Pekka Abrahamsson — [email protected]

Developed at GPT Lab, Tampere University.