EuConform

🇪🇺 Open-Source Evidence Toolkit For AI Compliance

Open evidence format • Local bias evaluation • Schema validation • CycloneDX interoperability
Offline-first • Privacy-preserving • Reusable artifacts • WCAG 2.2 AA accessible

EuConform defines an open evidence format for AI compliance and provides the tools to produce, validate, and empirically evaluate it — offline and vendor-independent.

[!IMPORTANT] Legal Disclaimer: This tool provides technical guidance only. It does not constitute legal advice and does not replace legally binding conformity assessments by notified bodies or professional legal consultation. Always consult qualified legal professionals for compliance decisions.

🚀 Quick Start · 📖 Docs · 🌐 Deploy · 🐛 Report Bug

✨ Features

🧰 CLI At A Glance

🚀 Quick Start

Want to try it without installation? Click the 🌐 Deploy link above to start your own instance on Vercel.

Prerequisites

• Node.js ≥ 18
• pnpm ≥ 10 (recommended) or npm/yarn

Installation

# Clone the repository
git clone https://github.com/Hiepler/EuConform.git
cd EuConform

# Install dependencies
pnpm install

# Start development server
pnpm dev

# Open http://localhost:3001

Build The CLI

The repo-local examples below use the built CLI directly:

# Build the CLI
pnpm --filter @euconform/cli build

Workflow 1: Scan A Repository

Generate native EuConform artifacts from a real codebase:

node packages/cli/dist/index.js scan . --scope production

This writes:

• .euconform/euconform.report.json
• .euconform/euconform.aibom.json
• .euconform/euconform.summary.md
• .euconform/euconform.bundle.json

Typical use:

• evidence collection for local OSS or internal AI projects
• CI gating and reviewer handoff
• portable artifact generation without a vendor platform

For CI usage, add GitHub-native annotations and fail thresholds:

node packages/cli/dist/index.js scan . --scope production --ci github --fail-on high

For portable artifact exchange, create a bundle archive:

node packages/cli/dist/index.js scan . --scope production --zip true

Workflow 2: Validate And Verify Existing Artifacts

Validate individual EuConform JSON documents against the published schemas:

node packages/cli/dist/index.js validate .euconform

Typical output:

• one line per file such as euconform.aibom.json — valid (euconform.aibom.v1)
• exit code 0 for fully valid input, 1 for schema errors, 2 when no EuConform JSON files are found

Verify a bundle manifest, extracted bundle directory, or ZIP archive:

node packages/cli/dist/index.js verify .euconform/euconform.bundle.json

Typical use:

• reviewer-side schema checking before manual analysis
• CI sanity checks for artifact sets already produced elsewhere
• portability checks before sharing bundles with downstream tools

Workflow 3: Evaluate Model Bias Locally

Run a reproducible CrowS-Pairs bias evaluation against a local Ollama model:

node packages/cli/dist/index.js bias llama3.2 --lang de --output all

This is EuConform's distinctive empirical evidence layer. It produces model-behavior data that no other open-source compliance tool currently offers — completely offline, reproducible, and independent of any vendor API.

Typical use:

• empirical model-behavior evidence for Art. 10 bias/fairness documentation
• reproducible local evaluation before and after model updates
• adding a behavioral evidence layer on top of structural evidence from scan

Workflow 4: Import An External CycloneDX SBOM

Map an external CycloneDX JSON file into the EuConform AI BOM layer:

node packages/cli/dist/index.js import ./third-party.cdx.json \
  --scope production \
  --output /tmp/euconform-import

node packages/cli/dist/index.js validate /tmp/euconform-import/euconform.aibom.json

This writes:

• /tmp/euconform-import/euconform.aibom.json

Important notes:

• import accepts CycloneDX JSON and maps only the AI-relevant subset into euconform.aibom.v1.1
• --scope production excludes optional and excluded components
• the importer is intentionally conservative and does not infer compliance capabilities from an SBOM
• project naming may come from BOM metadata or the source filename, depending on the input

Try The Format In 10 Minutes

If you want to evaluate the current adoption path as an OSS builder, use one of the reference projects in examples/:

# 1. Build the CLI
pnpm --filter @euconform/cli build

# 2. Scan a reference project
node packages/cli/dist/index.js scan examples/ollama-chatbot \
  --scope production \
  --output /tmp/euconform-ollama

# 3. Verify the generated bundle
node packages/cli/dist/index.js verify /tmp/euconform-ollama/euconform.bundle.json

# 4. Open the web app and import the generated artifacts
pnpm dev

For a retrieval-first example, replace examples/ollama-chatbot with examples/rag-assistant.

Using with Local AI Models (Optional)

For enhanced bias detection with your own models:

1. Install Ollama: Download from ollama.ai
2. Pull a model: ollama pull llama3.2
3. Start Ollama: ollama serve
4. Select "Ollama" in the web interface

Supports Llama, Mistral, and Qwen variants with automatic log-probability detection.

[!WARNING] Vercel / Cloud Deployment: This feature requires running EuConform locally (pnpm dev).

📖 Documentation

Legal Foundation & Compliance Coverage

[!NOTE] Primary Legal Source: Regulation (EU) 2024/1689 (EU AI Act)

Tool Coverage:

Implementation Timeline: Obligations become effective in stages. High-risk obligations apply from 2027. Always verify current guidelines and delegated acts.

CLI Scanner & CI

EuConform's CLI is designed as reusable evidence infrastructure:

• euconform scan produces native EuConform artifacts from a repository.
• euconform bias provides EuConform's distinctive empirical model-behavior evidence layer.
• euconform validate checks individual EuConform JSON files against the published schemas.
• euconform verify checks artifact-set integrity for manifests, directories, and ZIP bundles.
• euconform import bridges external CycloneDX JSON into the EuConform AI BOM layer.

The web app remains the place for role and risk classification with human context.

GitHub Actions Example

- name: Build CLI
  run: pnpm --filter @euconform/cli build

- name: Run EuConform scan
  run: node packages/cli/dist/index.js scan . --scope production --ci github --fail-on high

In GitHub Actions, EuConform emits:

• workflow annotations for top compliance gaps
• a markdown step summary
• machine-readable CI artifacts: euconform.ci.json and euconform.ci-summary.md

EuConform Evidence Format

EuConform implements the EuConform Evidence Format, an open specification for portable, machine-readable AI compliance evidence.

• euconform.report.v1 captures compliance evidence, gaps, and open questions
• euconform.aibom.v1 is the AI Bill of Materials (AI BOM) inventory layer
• euconform.ci.v1 captures CI thresholds, status, and top findings
• euconform.bundle.v1 binds artifact sets into a portable, integrity-aware manifest

Current workflow boundaries:

• scan produces native EuConform artifacts from source repositories
• import can ingest external CycloneDX JSON into the current AIBOM layer for interoperability workflows
• validate makes the schema and compatibility story inspectable at the JSON-document level
• verify checks portable bundle integrity before exchange or automation

Stage 1 documentation, schemas, and example artifacts live in docs/spec/README.md. Reference source projects for OSS builders live in examples/README.md.

Bias Testing Methodology

We use the CrowS-Pairs methodology (Nangia et al., 2020) to measure social biases in language models.

Calculation Methods

[!TIP] For best accuracy, use Ollama v0.1.26+ with models supporting the logprobs parameter (Llama 3.2+, Mistral 7B+).

Ethics Statement

The stereotype pairs are used solely for scientific evaluation and do not reflect the opinions of the developers. Individual pairs are not displayed in the UI to avoid reinforcing harmful stereotypes – only aggregated metrics are shown.

Custom Test Suites

Create domain-specific bias tests by uploading your own test cases:

[!TIP] Custom test suites are processed entirely in your browser – your proprietary test cases never leave your device.

@inproceedings{nangia-etal-2020-crows,
    title = "{C}row{S}-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models",
    author = "Nangia, Nikita and Vania, Clara and Bhalerao, Rasika and Bowman, Samuel R.",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    year = "2020",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.emnlp-main.154",
    doi = "10.18653/v1/2020.emnlp-main.154",
    pages = "1953--1967"
}

🏗️ Project Structure

euconform/
├── apps/
│   ├── web/                  # Next.js 16 production app
│   └── docs/                 # Documentation site (WIP)
├── packages/
│   ├── cli/                  # Local repo scanner and CI integration
│   ├── core/                 # Risk engine, scanner engine, fairness metrics, types
│   ├── ui/                   # Shared UI components (shadcn-style)
│   ├── typescript-config/    # Shared TypeScript configuration
│   └── tailwind-config/      # Shared Tailwind configuration
├── .github/
│   ├── workflows/            # CI/CD pipelines
│   └── ISSUE_TEMPLATE/       # Issue templates
├── biome.json                # Biome linter config
└── turbo.json                # Turborepo pipeline config

🧪 Testing

# Run unit tests
pnpm test

# Run with coverage
pnpm test -- --coverage

# Run E2E tests (requires Playwright)
pnpm test:e2e

# Type checking
pnpm check-types

# Linting
pnpm lint

🛠️ Tech Stack

❓ FAQ

🤝 Contributing

We welcome contributions! Please read our Contributing Guide and Code of Conduct first.

# Fork and clone
git clone https://github.com/yourusername/EuConform.git
cd EuConform

# Install and develop
pnpm install
pnpm dev

# Before submitting
pnpm lint && pnpm check-types && pnpm test

See CONTRIBUTING.md for detailed guidelines.

🔒 Security

For security concerns, please see our Security Policy. Do not create public issues for security vulnerabilities.

📄 License

Dual-licensed under:

• MIT License – for maximum compatibility
• EUPL-1.2 – for EU institution compatibility

Made with ❤️ for responsible AI in Europe

Issues · Discussions