---

Why GraphIP-Bench

Graph neural networks deployed as cloud services can be stolen through model-extraction attacks: an adversary submits carefully chosen queries, records the labels or confidence scores that the endpoint returns, and trains a local surrogate which reproduces the target's behaviour. A successful theft leaks the owner's intellectual property, undermines pay-per-query revenue, and lets competitors recreate proprietary functionality at low cost.

A growing line of ownership defenses — watermarking, fingerprinting, output perturbation, query-pattern detection — tries to prevent or trace such theft. But experimental practice in this area is fragmented: studies use private splits, incompatible budgets, and inconsistent metrics; the few existing testbeds focus on robustness or privacy and exclude model extraction together with ownership verification.

GraphIP-Bench addresses this gap. It standardises the evaluation of two complementary tracks — the extraction track (an adversary trains a surrogate that imitates a deployed GNN) and the ownership track (the model owner verifies a watermark or fingerprint after extraction) — under a single black-box protocol with shared splits, queries, and budgets. We then add a joint attack-and-defense track that runs every extraction attack on every defended target and measures watermark survival on the extracted surrogate — the setting that actually determines whether a defense is useful.

Component	What it gives you
12 attacks	Six MEA-style baselines (MEA0–MEA5), adversarial (AdvMEA), centrality-driven (CEGA), structure-aware (Realistic), and three data-free variants (DFEA_I/II/III)
12 defenses	5 ownership-tracing (BackdoorWM, RandomWM, SurviveWM, ImperceptibleWM, Integrity) + 7 information-limiting (OP_low, OP_high, PR_2bit, PR_top1, PRADA, AdaptMisinfo, GradRedir)
10 datasets	Homophilic citation (Cora, CiteSeer, PubMed), coauthor/product (CoauthorCS, CoauthorPhysics, Computers, Photo), large-scale (OGBN-Arxiv), heterophilic (RomanEmpire, AmazonRatings)
3 backbones × 3 tasks	GCN, GAT, GraphSAGE on node classification, link prediction, graph classification
Joint track	Surrogate fidelity and watermark survival on the extracted surrogate, all under one protocol
Reproducibility	Public splits, shared query sets, fixed seeds, JSON-Lines outputs, ready-made aggregation scripts

---

Highlights

One protocol, two sides Every attack is evaluated against every defense under the same splits, budgets, and endpoint assumptions. Fidelity, ownership verification, utility, and compute cost are reported on a shared metric suite.	Watermark survival, not just embedding The joint track measures whether a watermark still verifies after the model is actually extracted — exposing the gap that single-model evaluations miss.
Heterophilic & large-scale graphs Beyond the seven homophilic graphs of the core protocol, we add OGBN-Arxiv (169K nodes, 40 classes) and the heterophilic RomanEmpire / AmazonRatings to stress test attack and defense behaviour.	Reproducible by construction Fixed seeds, shared query sets, and JSON-Lines output records per (dataset, attack, defense, regime, budget, seed) cell. A single-seed run is directly comparable to the corresponding cell in the paper.

---

At A Glance

Signal	Scale in the benchmark
Extraction attacks	12
Ownership / information-limiting defenses	12
Graph datasets	10
GNN backbones	3 (GCN, GAT, GraphSAGE)
Graph-learning tasks	3 (node / link / graph)
Data-availability regimes	4 (both, features-only, structure-only, data-free)
Standardized query budgets	5 (0.05× — 1.00× test size)
Seeds per cell	3
Evaluation tracks	3 (extraction, ownership, joint)

---

Quick Start

git clone https://github.com/LabRAI/GraphIP-Bench.git
cd GraphIP-Bench
conda create -n graphip python=3.11 -y
conda activate graphip
pip install -r requirements.txt

Run one cell of the extraction track:

python scripts/run_rq1_single.py \
  --dataset Cora \
  --attack MEA0 \
  --regime both \
  --budget 0.25 \
  --seed 0 \
  --gpu \
  --output-dir outputs/RQ1

Run the joint attack-and-defense track on a single dataset:

python examples/run_joint_evaluation.py \
  --dataset Cora \
  --seed 0 \
  --gpu

Each run writes one JSON-Lines record per (attack, defense, regime, budget, seed) tuple to outputs/. Records can be merged and analysed with the helpers in scripts/ (e.g. merge_rq1_rq5_faithful_results.py).

---

Installation

GraphIP-Bench supports Linux (recommended) with Python 3.10+. We pin PyTorch / DGL / PyG versions because DGL 2.1.0 graphbolt kernels only ship pre-built shared libraries for PyTorch 2.0–2.2 on CUDA 12.1.

Option A: Conda (recommended)

conda create -n graphip python=3.11 -y
conda activate graphip
pip install -r requirements.txt

Option B: pip + venv

python -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

📦 Pinned stack

Library	Version	Why pinned
PyTorch	2.2.1 + cu121	DGL 2.1.0 graphbolt wheels only support PT 2.0–2.2 / CUDA 12.1
DGL	2.1.0 + cu121	Graphbolt kernels used by the loaders
PyTorch Geometric	2.7.0	Link-prediction and graph-classification adapters
OGB	1.3.6	OGBN-Arxiv loader
NumPy	< 2.0	DGL kernel ABI

---

Research Questions

GraphIP-Bench is organised around five research questions; each maps to a top-level reproduction script.