Open Source Link Analysis & OSINT Framework
<div align="center"> <img src="https://assets.kitploit.com/production/public/readmes/12241/4d60ae367bb5f1d9b7d1d95df2b16186f01454c9610e40703f636923a0a70ced.png" alt="OGI Mascot" width="200" /> # OpenGraph Intel (OGI) **Open-source visual link analysis and OSINT framework.** **Free, self-hostable, and community-driven.** [](https://www.gnu.org/licenses/agpl-3.0) [](https://github.com/khashashin/ogi/actions/workflows/ci.yml) [](https://github.com/khashashin/ogi/actions/workflows/docker-publish.yml) [](https://www.python.org) [](https://www.typescriptlang.org) [](https://react.dev) [](https://fastapi.tiangolo.com)  [Features](#features) • [Screenshots](#screenshots) • [Quick Start](#quick-start) • [Docker](#docker) • [Transform Hub](#transform-hub) • [Architecture](#architecture) • [Contributing](#contributing) </div> --- > **Heads up:** This project is actively evolving. It has solid core capabilities and test coverage, and we continue to improve documentation, hardening, and feature depth with each release. Contributions, bug reports, and feedback are very welcome. ## Features - **Visual graph investigation** — drag-and-drop entities, explore connections interactively - **20+ built-in transforms** — DNS, WHOIS, SSL certs, geolocation, web/email/hash/social enrichment, and more - **Transform Hub** — browse and install community transforms from the [registry](https://github.com/opengraphintel/ogi-transforms) - **AI Investigator** — prompt-driven agentic investigations that plan transform runs, stream progress live, and summarize findings with auditability - **Import / Export** — JSON, CSV, GraphML, and MTGX format import - **Graph analysis** — centrality, community detection, shortest paths - **Real-time collaboration** — projects, sharing, and live sync via Supabase Realtime - **Async transform jobs** — Redis/RQ queue with WebSocket progress updates - **Plugin system** — directory-based plugin discovery with YAML manifests - **CLI tool** — search, install, and manage transforms from the terminal - **Runs anywhere** — local SQLite mode (zero config) or PostgreSQL + Supabase for team/cloud setups - **Docker ready** — one-command deployment with `docker compose up` ## Screenshots <div align="center"> | Graph Investigation | Entity Enrichment | | :-------------------------------------------: | :-----------------------------------------: | |  |  | | Transform Hub | Export / Import | | :-------------------------------------: | :-------------------------------------: | |  |  | </div> ## Quick Start ### Prerequisites | Tool | Version | | -------------------------------- | ------- | | [Python](https://www.python.org) | 3.14+ | | [uv](https://docs.astral.sh/uv/) | latest | | [Node.js](https://nodejs.org) | 20+ | | [pnpm](https://pnpm.io) | latest | ### Backend ```bash cd backend uv sync uv run uvicorn ogi.main:app --reload ``` The API will be available at `http://localhost:8000`. For local transform execution, Redis and the transform worker must also be running. Redis alone is not enough. Start Redis: ```bash docker run -d --name ogi-redis -p 6379:6379 redis:7-alpine ``` Start the transform worker in a second terminal: ```bash cd backend uv run python -m ogi.worker.run_worker ``` If you see `Job queue not available` or `Redis not available`, verify that: - `OGI_REDIS_URL` points to `redis://localhost:6379/0` for local host-based runs - the backend was restarted after Redis came up - the separate worker process is running - you are not using the Docker-only hostname `redis` outside Docker Compose If you run the backend against PostgreSQL (`OGI_USE_SQLITE=false`), startup will automatically apply Alembic migrations before serving requests. Docker deployments do the same in the backend container entrypoint. AI Investigator runs are processed by another separate worker: ```bash cd backend uv run python -m ogi.agent.run_worker ``` ### Frontend ```bash cd frontend pnpm install pnpm dev ``` Open http://localhost:5173. That's it. ### AI Investigator AI Investigator is an optional provider-backed workflow that can plan transform sequences, request approvals, and summarize investigation progress inside the workspace. - Open the `AI Investigator` tab inside a project - Configure a provider and model in the investigator settings dialog - Store provider API keys in `API Keys` - Run the separate `agent-worker` process alongside the backend The current implementation supports per-user provider configuration and a dedicated worker that executes investigation steps independently from the main API server. ### CLI Two supported ways to run the CLI: **Recommended** (no activation required): ```bash cd backend uv sync uv run ogi --help ``` **Activated virtualenv** (plain `ogi` command): ```bash cd backend uv venv # PowerShell: .\.venv\Scripts\Activate.ps1 uv pip install -e . ogi --help ``` ## Docker ### Development ```bash cp .env.example .env docker compose up ``` ### Production Use the prebuilt GHCR images. Important: [docker-compose.prod.yml](https://github.com/khashashin/ogi/blob/main/docker-compose.prod.yml) requires an external PostgreSQL database. It does not include a `db` service. If you want the all-in-one local Docker stack with built-in Postgres and Redis, use: ```bash docker compose up -d ``` Use the production compose file only when you already have a reachable PostgreSQL instance and have set `OGI_DATABASE_URL` accordingly: ```bash docker compose -f docker-compose.prod.yml pull docker compose -f docker-compose.prod.yml up -d ``` Set `OGI_IMAGE_TAG` in `.env` to pin a specific release image tag (e.g. `v0.2.6`). Defaults to `latest`. ### Hetzner Cloud [](https://hetzner.cloud/?ref=LspEGc99USBa) [](https://console.hetzner.com/deploy/docker-ce) New Hetzner users can use the first link to sign up with Hetzner Cloud credits. It is a referral link, so the OGI maintainer may also receive Hetzner Cloud credits if the referral qualifies. After creating a server, use the Docker deployment commands above to run OGI. ### Configurable transform caps OGI ships with sensible per-transform max values for things like `max_results`, `max_links`, `max_urls`, and `max_content_chars`, but those caps are now centrally overridable so cloud deployments can enforce them without hardcoding local limits. Use `OGI_TRANSFORM_SETTING_MAX_OVERRIDES` in `.env`: ```env OGI_TRANSFORM_SETTING_MAX_OVERRIDES=max_results=50,max_urls=25,max_links=40,max_content_chars=20000 ``` To remove specific caps in a local-first deployment: ```env OGI_TRANSFORM_SETTING_MAX_OVERRIDES=max_results=none,max_urls=none,max_links=none,max_content_chars=none ``` The override is keyed by transform setting name and applies to built-in transforms and community plugins that use OGI's shared transform base/runtime. ### Hosted cloud instance and Supporter plan The hosted cloud version at [ogi.khas.app](https://ogi.khas.app) was initially built so users could try OGI quickly before deciding whether to self-host it. It has since become the most popular way to use the project, with over 350 active cloud users. That usage also increased the infrastructure costs for this side project. The repository therefore includes a cloud-only Supporter subscription and transformer-run cooldown system. On the public cloud instance, free users may be limited by a timeout between transformer runs, while Supporter users can subscribe for a symbolic USD 3/month to help cover Supabase and VPS costs. Supporter subscriptions can be cancelled from the profile billing controls or the Stripe billing portal. Cancellation stops future billing, but paid subscription periods are not refunded or prorated. This billing code is intentionally guarded by configuration: it only has an effect when `OGI_DEPLOYMENT_MODE=cloud` and `OGI_CLOUD_BILLING_ENABLED=true`. Self-hosted deployments should leave billing disabled and should not see any billing or paywall behavior. ### Telemetry OGI includes installation-level product telemetry intended to help us understand real-world usage, especially across self-hosted deployments. - `OGI_TELEMETRY_ENABLED=true` enables telemetry collection - `OGI_TELEMETRY_LEVEL=full` is the default collection level - set `OGI_TELEMETRY_ENABLED=false` to disable telemetry entirely - set `OGI_TELEMETRY_LEVEL=basic` to reduce what is sent `basic` sends: - OGI version - a daily active installation ping `full` additionally sends: - instance created date - aggregate counts for projects, entities, edges, transform runs, investigator runs, and active users for the period - installed transforms with their versions Telemetry is designed to avoid sending graph contents, entity values, API keys, prompts, or other investigation data. The current policy details are also published in the [Privacy Policy](https://ogi.khas.app/privacy#telemetry). <details> <summary><strong>Development Compose Services</strong></summary> | Service | Description | Port | | -------------- | -------------------------- | ---- | | `backend` | FastAPI application server | 8000 | | `worker` | RQ async job worker | - | | `agent-worker` | AI Investigator worker | - | | `frontend` | React app served via nginx | 80 | | `db` | PostgreSQL 16 | 5432 | | `redis` | Redis 7 (job queue) | 6379 | </details> <details> <summary><strong>Production Compose Services</strong></summary> | Service | Description | Port | | -------------- | -------------------------- | ---- | | `backend` | FastAPI application server | 8000 | | `worker` | RQ async job worker | - | | `agent-worker` | AI Investigator worker | - | | `frontend` | React app served via nginx | 80 | | `redis` | Redis 7 (job queue) | 6379 | `docker-compose.prod.yml` expects an external PostgreSQL database via `OGI_DATABASE_URL`. </details> --- <details> <summary><strong>Boot-time plugin dependencies</strong></summary> If you use prebuilt images and a plugin needs extra Python libraries, OGI installs plugin dependencies at container startup from: 1. `plugins/requirements.txt` (preferred), or 2. auto-generated requirements derived from `plugins/ogi-lock.json` when `requirements.txt` is missing. **Environment variables:** | Variable | Default | Description | | --------------------------------- | ------------------------------- | ----------------------------------------- | | `OGI_BOOT_REQUIREMENTS_ENABLE` | `true` | Enable/disable boot-time install | | `OGI_BOOT_REQUIREMENTS_FILE` | `/app/plugins/requirements.txt` | Path to requirements file | | `OGI_BOOT_LOCK_FILE` | `/app/plugins/ogi-lock.json` | Path to lock file | | `OGI_BOOT_REQUIREMENTS_STRICT` | `false` | Fail startup if requirements missing | | `OGI_BOOT_REQUIREMENTS_CACHE_DIR` | `/tmp/ogi-boot` | Temp/cache dir for generated requirements | </details> --- ## Transform Hub OpenGraph Intel has a built-in transform marketplace. Browse, install, and manage transforms from the [community registry](https://github.com/opengraphintel/ogi-transforms). > **Security note:** Plugins that require API keys should be treated as privileged code. If a plugin can access your secrets and make outbound network requests, it can misuse or exfiltrate those secrets. Review trust tier, permissions, and required services before installing or running third-party plugins. ```bash # Search for transforms uv run ogi transform search dns # Install a transform uv run ogi transform install shodan-host-lookup ``` ### Built-in Transform Categories | Category | Examples | | -------- | ------------------------------------- | | DNS | A, AAAA, MX, NS, CNAME records | | IP & ASN | GeoIP, reverse IP, ASN info | | SSL/TLS | Certificate transparency, SSL labs | | Email | Mailserver validation, breach checks | | Web | WHOIS, domain info, web extraction | | Social | Username enumeration, social profiles | | Hash | MD5, SHA1, SHA256 lookups | | Location | Geocoding, weather, nearby ASN data | ### Supported Entity Types `Person` • `Username` • `Domain` • `IPAddress` • `EmailAddress` • `PhoneNumber` • `Organization` • `URL` • `SocialMedia` • `Hash` • `Document` • `Location` • `ASNumber` • `Network` • `MXRecord` • `NSRecord` • `Nameserver` • `SSLCertificate` • `Subdomain` • `HTTPHeader` ### Building Your Own Transforms Want to build your own? See the [contributing guide](https://github.com/opengraphintel/ogi-transforms/blob/main/CONTRIBUTING.md). If your transform needs external service credentials, declare them in `api_keys_required`. Do not store secrets in transform settings. OGI manages those under `API Keys`, and secret-using plugins are considered privileged code. If your transform exposes capped settings such as `max_results` or `max_content_chars`, prefer stable setting names over bespoke one-off names where possible. OGI can centrally override max values with `OGI_TRANSFORM_SETTING_MAX_OVERRIDES`, which helps cloud deployments enforce limits while local users can remove them. Each plugin is a directory with a `plugin.yaml` manifest: ```yaml name: my-transform version: "1.0.0" display_name: My Transform description: What it does author: Your Name license: MIT category: dns input_types: [Domain] output_types: [IPAddress] permissions: network: true filesystem: false subprocess: false ``` ## Architecture ### Tech Stack | Layer | Technology | | ---------------- | --------------------------------------------------------------------------- | | Backend | Python 3.14+, FastAPI, SQLModel, asyncpg / aiosqlite | | Frontend | React 19, TypeScript 5.9, Sigma.js (graphology), Zustand, Tailwind CSS 4 | | Database | PostgreSQL 16 (primary) / SQLite (local fallback) | | Auth & Realtime | Supabase Auth + JWT + Realtime (optional in local mode) | | Job Queue | Redis 7 + RQ (async transforms) | | AI Runtime | Provider-backed AI Investigator worker with audited transform orchestration | | Package Managers | uv (backend), pnpm (frontend) | | Deployment | Docker, nginx, GHCR | ### Project Structure ``` ogi/ ├── backend/ │ └── ogi/ │ ├── api/ # REST API routes │ ├── cli/ # CLI tool (Typer) │ ├── db/ # Database layer (asyncpg + aiosqlite) │ ├── engine/ # Graph engine & transform engine │ ├── models/ # SQLModel definitions │ ├── store/ # Data stores │ ├── transforms/ # Built-in transforms │ │ ├── dns/ # DNS resolution transforms │ │ ├── cert/ # SSL certificate transforms │ │ ├── email/ # Email enrichment │ │ ├── hash/ # Hash lookups │ │ ├── ip/ # IP/ASN/geolocation │ │ ├── org/ # Organization info │ │ ├── social/ # Social media │ │ └── web/ # Web scraping/extraction │ ├── worker/ # Async job queue (RQ) │ ├── config.py # Pydantic settings │ └── main.py # FastAPI entry point ├── frontend/ │ └── src/ │ ├── api/ # API client │ ├── components/ # React components │ ├── hooks/ # Custom hooks (realtime sync, etc.) │ ├── stores/ # Zustand state management │ ├── types/ # TypeScript types │ └── App.tsx # Main application ├── plugins/ # Community plugins directory ├── docs/ # Documentation & images ├── docker-compose.yml # Development deployment ├── docker-compose.prod.yml # Production deployment └── .env.example # Environment template ``` ### Configuration OGI uses [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/) with `.env` file support. Runtime/backend variables are prefixed with `OGI_`. List-style settings accept either: - JSON arrays, for example `["plugins","../plugins"]` - comma-separated strings, for example `plugins,../plugins` <details> <summary><strong>Key environment variables</strong></summary> | Variable | Description | Default | | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- | | `OGI_APP_NAME` | Application name | `OGI` | | `OGI_HOST` | Backend bind host | `0.0.0.0` | | `OGI_PORT` | Backend port | `8000` | | `OGI_CORS_ORIGINS` | Allowed frontend origins | `http://localhost:5173,http://localhost:3000` | | `OGI_USE_SQLITE` | Use SQLite instead of PostgreSQL | `true` | | `OGI_DB_PATH` | SQLite database file path | `ogi.db` | | `OGI_DATABASE_URL` | PostgreSQL connection string | `postgresql://postgres:postgres@localhost:5432/ogi` | | `OGI_REDIS_URL` | Redis connection string for RQ/jobs | `redis://localhost:6379/0` | | `OGI_RQ_QUEUE_NAME` | Queue name for transform jobs | `transforms` | | `OGI_TRANSFORM_TIMEOUT` | Per-transform job timeout in seconds | `300` | | `OGI_AGENT_WORKER_POLL_INTERVAL_SEC` | Poll interval for the AI Investigator worker | `2.0` | | `OGI_AGENT_CLAIM_TIMEOUT_SEC` | Stale-claim timeout for AI Investigator step recovery | `120` | | `OGI_AUTO_RUN_MIGRATIONS` | Auto-run Alembic on local non-SQLite app startup | `true` | | `OGI_RUN_DB_MIGRATIONS` | Run DB migrations in container entrypoint | `false` | | `OGI_DB_MIGRATION_RETRIES` | Entry-point migration retry count | `30` | | `OGI_DB_MIGRATION_DELAY_SECONDS` | Delay between entry-point migration retries | `2` | | `OGI_PLUGIN_DIRS` | Plugin search directories | `plugins,../plugins` | | `OGI_DEPLOYMENT_MODE` | Deployment mode (`self-hosted` or `cloud`) | `self-hosted` | | `OGI_CLOUD_BILLING_ENABLED` | Enable cloud-only Supporter billing and transform cooldown enforcement | `false` | | `OGI_FREE_TRANSFORM_COOLDOWN_SECONDS` | Cooldown between free cloud-user transformer runs | `1800` | | `OGI_PAID_TRANSFORM_COOLDOWN_SECONDS` | Cooldown between paid cloud-user transformer runs | `0` | | `OGI_STRIPE_SECRET_KEY` | Stripe server-side secret key for Checkout and portal sessions | unset | | `OGI_STRIPE_WEBHOOK_SECRET` | Stripe webhook signing secret | unset | | `OGI_STRIPE_SUPPORTER_PRICE_ID` | Stripe recurring Price ID for the Supporter plan | unset | | `OGI_STRIPE_SUPPORTER_AMOUNT_CENTS` | Display amount for the Supporter plan | `300` | | `OGI_STRIPE_SUPPORTER_CURRENCY` | Display currency for the Supporter plan | `usd` | | `OGI_BILLING_SUCCESS_URL` | Optional Stripe Checkout success redirect | derived from request | | `OGI_BILLING_CANCEL_URL` | Optional Stripe Checkout cancel redirect | derived from request | | `OGI_BILLING_PORTAL_RETURN_URL` | Optional Stripe billing portal return URL | derived from request | | `OGI_REGISTRY_REPO` | Transform registry GitHub repo | `opengraphintel/ogi-transforms` | | `OGI_REGISTRY_CACHE_TTL` | Registry cache TTL in seconds | `3600` | | `OGI_TRANSFORM_SETTING_MAX_OVERRIDES` | Optional global max override map for transform settings | empty | | `OGI_GITHUB_TOKEN` | Optional GitHub token for registry/API rate limits | unset | | `OGI_SUPABASE_URL` | Supabase project URL | unset | | `OGI_SUPABASE_ANON_KEY` | Supabase anon/public key | unset | | `OGI_SUPABASE_SERVICE_ROLE_KEY` | Supabase service-role key | unset | | `OGI_SUPABASE_JWT_SECRET` | Supabase JWT secret | unset | | `OGI_SUPABASE_REDIRECT_URL` | Redirect URL used by frontend auth flows | unset | | `OGI_ADMIN_EMAILS` | Admin users for registry/plugin management | unset | | `OGI_TELEMETRY_ENABLED` | Enable or disable installation-level telemetry | `true` | | `OGI_TELEMETRY_LEVEL` | Telemetry level (`basic` or `full`) | `full` | | `OGI_API_KEY_ENCRYPTION_KEY` | Fernet key for encrypted API key storage | unset but strongly recommended | | `OGI_API_KEY_INJECTION_ALLOW_COMMUNITY_PLUGINS` | Allow community plugins to receive stored API keys | `true` | | `OGI_API_KEY_INJECTION_TRUSTED_TIERS_ONLY` | Restrict stored key injection to trusted tiers only | `false` | | `OGI_API_KEY_INJECTION_ALLOWED_TIERS` | Tiers allowed when trusted-only mode is enabled | `official,verified` | | `OGI_API_KEY_SERVICE_ALLOWLIST` | Optional allowed services for stored key injection | empty | | `OGI_API_KEY_SERVICE_BLOCKLIST` | Optional blocked services for stored key injection | empty | | `OGI_LLM_PROVIDER` | Default AI Investigator provider fallback | `openai` | | `OGI_LLM_MODEL` | Default AI Investigator model fallback | `gpt-4.1-mini` | | `OGI_EXPOSE_ERROR_DETAILS` | Include internal details in 500 responses | `false` | | `OGI_SANDBOX_ENABLED` | Enable sandbox execution mode | `false` | | `OGI_SANDBOX_TIMEOUT` | Sandbox timeout in seconds | `30` | | `OGI_SANDBOX_MEMORY_MB` | Sandbox memory limit in MB | `256` | | `OGI_SANDBOX_ALLOWED_TIERS` | Allowed tiers in cloud sandbox mode | `official,verified` | | `OGI_BOOT_REQUIREMENTS_ENABLE` | Enable boot-time plugin dependency install | `true` | | `OGI_BOOT_REQUIREMENTS_FILE` | Boot requirements file path | `/app/plugins/requirements.txt` | | `OGI_BOOT_LOCK_FILE` | Boot lock file path | `/app/plugins/ogi-lock.json` | | `OGI_BOOT_REQUIREMENTS_STRICT` | Fail startup if plugin requirements are missing | `false` | | `OGI_BOOT_REQUIREMENTS_CACHE_DIR` | Temp/cache directory for boot requirement generation | `/tmp/ogi-boot` | | `OGI_FRONTEND_PORT` | Local Docker frontend port mapping | `3000` | --- [Read more](https://github.com/khashashin/ogi)
