将源代码解析为可查询的函数、类、调用和语义注解图,用于安全分析。
Trailmark 使用 tree-sitter 进行语言无关的 AST 解析,并使用 rustworkx 进行高性能图遍历。其长远愿景是将此图与变异测试和覆盖率引导的模糊测试相结合,以识别从用户输入可达的假设与测试覆盖率之间的差距。
Trailmark 分三个阶段运行:解析、索引和查询。```mermaid flowchart TD A["Source Files"] --> B["tree-sitter Parser"] B --> C["CodeGraph (nodes + edges)"] C --> D["rustworkx GraphStore"] D --> E["QueryEngine"] E --> F["JSON / Summary / Hotspots"]
classDef src fill:#007bff26,stroke:#007bff,color:#007bff
classDef parse fill:#28a74526,stroke:#28a745,color:#28a745
classDef data fill:#6f42c126,stroke:#6f42c1,color:#6f42c1
classDef query fill:#ffc10726,stroke:#e6a817,color:#e6a817
class A src
class B parse
class C,D data
class E,F query
### 1. 解析
一种特定语言的解析器遍历目录,将每个文件解析为 tree-sitter AST,并提取:
- **节点** — 函数、方法、类、结构体、接口、特性、枚举、模块、命名空间
- **边** — 调用、继承、实现、包含、导入
- **元数据** — 类型注解、圈复杂度、分支、文档字符串、异常类型
### 支持的语言
| 语言 | 扩展名 | 关键构造 |
| --- | --- | --- |
| Python | `.py` | 函数、类、方法 |
| JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | 函数、类、箭头函数 |
| TypeScript | `.ts`, `.tsx` | 函数、类、接口、枚举 |
| PHP | `.php` | 函数、类、接口、特性(traits) |
| Ruby | `.rb` | 方法、类、模块 |
| C | `.c`, `.h` | 函数、结构体、枚举 |
| C++ | `.cpp`, `.hpp`, `.cc`, `.hh`, `.cxx`, `.hxx` | 函数、类、结构体、命名空间 |
| C# | `.cs` | 方法、类、接口、结构体、枚举、命名空间 |
| Java | `.java` | 方法、类、接口、枚举 |
| Go | `.go` | 函数、方法、结构体、接口 |
| Rust | `.rs` | 函数、结构体、特性(traits)、枚举、impl 块 |
| Solidity | `.sol` | 合约、接口、库、函数、修饰符、结构体、枚举 |
| Cairo | `.cairo` | 函数、特性、结构体、枚举、impl 块、StarkNet 合约 |
| Circom | `.circom` | 模板、函数、信号、组件 |
| Haskell | `.hs` | 函数、数据类型、类型类、实例 |
| Erlang | `.erl` | 函数、记录、行为、模块 |
| Miden Assembly | `.masm` | 过程、入口点、常量、调用 |
| Swift | `.swift` | 函数、类、结构体、枚举、协议、扩展 |
| Objective-C | `.m`, `.mm`, `.h` | C 函数、类、方法(基于选择器的命名) |
| Kotlin | `.kt`, `.kts` | 函数、类、接口、数据类、对象、方法 |
| Dart | `.dart` | 函数、类、抽象类、方法、构造函数 |
| Move | `.move` | 模块、函数、导入、直接调用 |
| Tact | `.tact` | 合约、结构体、接收器、函数 |
| Func | `.fc`, `.func` | 函数、包含、直接调用 |
| Sway | `.sw` | ABI 接口、结构体、impl 方法、函数 |
| Rego | `.rego` | 包、导入、策略规则、规则调用 |
| Proto | `.proto` | 服务、RPC、消息、字段、枚举 |
| Thrift | `.thrift` | 服务、函数、结构体、字段、枚举 |
| GraphQL | `.graphql`, `.gql` | 对象类型、根操作、字段、枚举 |
| SQL | `.sql` | 模式、表、视图、函数、过程 |```mermaid
flowchart TD
subgraph "Per-File Parsing"
F["Source file"] --> TS["tree-sitter AST"]
TS --> EX["Extract nodes"]
TS --> EC["Extract call edges"]
TS --> EB["Count branches"]
TS --> ET["Resolve types"]
end
EX --> CG["CodeGraph"]
EC --> CG
EB --> CG
ET --> CG
classDef src fill:#007bff26,stroke:#007bff,color:#007bff
classDef parse fill:#28a74526,stroke:#28a745,color:#28a745
classDef extract fill:#ffc10726,stroke:#e6a817,color:#e6a817
classDef data fill:#6f42c126,stroke:#6f42c1,color:#6f42c1
class F src
class TS parse
class EX,EC,EB,ET extract
class CG data
节点ID遵循module:function、module:Class或module:Class.method的格式,实现无歧义查找。目录解析能够在唯一定义存在时解析裸跨文件调用;存在歧义的跨文件调用则保留其原始最佳猜测目标,并标记为uncertain。边置信度标记为certain(直接调用、self.method())、inferred(对非self对象的属性访问)或uncertain(动态调度或歧义解析)。
GraphStore将CodeGraph加载到一个rustworkx PyDiGraph中,并构建双向ID/索引映射以实现快速遍历。
QueryEngine为上层的索引图提供了一个高级API:
classDiagram class CodeGraph { language: str root_path: str nodes: dict[str, CodeUnit] edges: list[CodeEdge] annotations: dict[str, list[Annotation]] entrypoints: dict[str, EntrypointTag] dependencies: list[str] add_annotation(node_id, annotation) clear_annotations(node_id, kind=None) merge(other) }
class CodeUnit {
id: str
name: str
kind: NodeKind
location: SourceLocation
parameters: tuple[Parameter]
return_type: TypeRef
exception_types: tuple[TypeRef]
cyclomatic_complexity: int
branches: tuple[BranchInfo]
docstring: str
}
class CodeEdge {
source_id: str
target_id: str
kind: EdgeKind
confidence: EdgeConfidence
}
class Annotation {
kind: AnnotationKind
description: str
source: str
}
class EntrypointTag {
kind: EntrypointKind
trust_level: TrustLevel
description: str
asset_value: AssetValue
}
CodeGraph "1" *-- "*" CodeUnit
CodeGraph "1" *-- "*" CodeEdge
CodeGraph "1" *-- "*" Annotation
CodeGraph "1" *-- "*" EntrypointTag
**节点种类:** `function`, `method`, `class`, `module`, `struct`, `interface`, `trait`, `enum`, `namespace`, `contract`, `library`, `template`, `proxy`
**节点起源:** `source`, `proxy`, `binary`, `synthetic`
**边种类:** `calls`, `inherits`, `implements`, `contains`, `imports`, `resolves_to`, `type_uses`, `specializes`, `corresponds_to`
**边置信度:** `certain`, `inferred`, `uncertain`
未解决的调用会被具体化为代理节点,例如 `proxy.unresolved:<raw-symbol>`,这样遍历结果可以显示源分析在何处丢失了解析,而不是默默地丢弃该边。二进制分析支持导入外部 JSON 调用图;Trailmark 本身不会反汇编可执行文件。
### 示例图
给定以下 Python 代码:
```python```python
class Auth:
def verify(self, token: str) -> bool:
return self._check_sig(token)
def _check_sig(self, token: str) -> bool:
...
def handle_request(req: Request) -> Response:
auth = Auth()
if auth.verify(req.token):
return process(req)
return deny()
Trailmark 生成一个类似如下的图形:```mermaid graph TD HR["handle_request"] -->|calls| AV["Auth.verify"] HR -->|calls| P["process"] HR -->|calls| D["deny"] AV -->|calls| CS["Auth._check_sig"] A["Auth"] -->|contains| AV A -->|contains| CS
classDef fn fill:#007bff26,stroke:#007bff,color:#007bff
classDef cls fill:#6f42c126,stroke:#6f42c1,color:#6f42c1
class HR,P,D fn
class A,AV,CS cls
## 安装
下面的示例追踪当前开发分支。对于最新发布的包,请从PyPI安装。要获得此处描述的精确功能集,请从源代码检出安装,并通过 `uv run` 运行命令。```bash
# Latest published release
uv pip install trailmark
# Current checkout / development branch
uv sync --all-groups
需要 Python ≥ 3.12。
Trailmark 使用 tree-sitter-language-pack 来管理大多数语法解析器。当前版本使用平台证书存储来下载语法文件。在 TLS 检测或离线环境中,请先在匹配的平台上通过 python -c "import tree_sitter_language_pack as p; p.download_all()" 预填充包缓存,然后将生成的 tree-sitter-language-pack 缓存目录复制到目标机器。HTTPS_PROXY 环境变量同样被支持。SQL 语法以 tree-sitter-sql wheel 依赖形式提供,不使用该缓存。
trailmark --version # or: trailmark -V trailmark version # subcommand form
trailmark analyze path/to/project
trailmark analyze --language rust path/to/project trailmark analyze --language javascript path/to/project
trailmark analyze --language auto path/to/project trailmark analyze --language python,rust,solidity path/to/project
trailmark analyze --summary path/to/project
trailmark analyze --complexity 10 path/to/project
trailmark augment --sarif results.sarif path/to/project trailmark augment --weaudit findings.json path/to/project trailmark augment --sarif a.sarif --sarif b.sarif --json path/to/project
trailmark entrypoints path/to/project trailmark entrypoints --json path/to/project
trailmark diff before/ after/ trailmark diff --repo . main HEAD # compare git refs trailmark diff --json before/ after/ # machine-readable output
trailmark diagram --target path/to/project --type call-graph trailmark diagram -t path/to/project -T call-graph -f parse_file --depth 3 trailmark diagram -t path/to/project -T complexity --threshold 5 --direction LR
### 入口点检测
Trailmark 自动填充 `graph.entrypoints`,使 `attack_surface()`、污染传播和权限边界跨越得以使用数据。检测分四层进行,每一层覆盖上一层:
1. **通用 `main` 启发式。** 任何语言中名为 `main` 的函数。标记为 `user_input` / `trusted_internal` / `low`。
2. **框架感知扫描。** 每种语言的装饰器、属性和可见性模式 — 见下表。
3. **`pyproject.toml [project.scripts]`。** 显式 CLI 目标获得升级的信任/资产分类。
4. **仓库本地覆盖文件。** `.trailmark/entrypoints.toml` 中手工整理的入口点始终优先。
框架覆盖范围:
| 语言 | 检测到的框架 |
| --- | --- |
| Python | Flask, FastAPI, aiohttp, Click, Typer, Celery |
| JavaScript / TypeScript | NestJS, Next.js (App Router + Pages API), AWS Lambda |
| Java | Spring MVC / WebFlux, JAX-RS, Kafka listeners, servlets |
| C# | ASP.NET Core, Azure Functions |
| PHP | Symfony `#[Route]` attributes + legacy annotations |
| Rust | actix-web, rocket, FFI exports (`#[no_mangle]`, `pub extern "C"`), async-main attributes |
| Solidity | `external` / `public` visibility |
| Cairo / StarkNet | `#[external]`, `#[view]`, `#[l1_handler]`, `#[constructor]` |
| Circom | `component main` declarations |
| Miden Assembly | `export.<name>` directives |
| Haskell | top-level `main ::` / `main =` |
| Erlang | functions listed in `-export([...])` |
| Swift | `@main` app attribute |
| Objective-C | `UIApplicationDelegate` lifecycle selectors (e.g. `application:openURL:options:`) |
| Kotlin | Spring MVC / WebFlux annotations (shared with Java), Android component lifecycle methods (`onCreate`, `onReceive`, `onBind`, ...) |
| Dart | `@pragma('vm:entry-point')` native-callable markers |
| Go | `http.HandleFunc` / `http.Handle` stdlib registrations, gin/chi/echo-style `<router>.GET/POST/...` handler registrations |
| Ruby | Rails controller actions (classes inheriting `ApplicationController` / `ActionController::*`), Sidekiq worker `perform` methods |
| C / C++ | `extern "C"` linkage, `__attribute__((visibility("default")))`, `__declspec(dllexport)` |
对于启发式未能检测到的任何内容,请在项目根目录下的 `.trailmark/entrypoints.toml` 中显式声明入口点。该文件支持单节点和基于规则的条目:
```toml
# Example: single explicit entrypoint
[entrypoints.cli_main]
function = "my_package.cli:main"
trust = "high"
asset = "sensitive"
# Example: rule-based catch-all for Spring @RequestMapping
[[entrypoint_patterns]]
pattern = "^(org\\.example\\.).* (get|post|put|delete|patch)$"
trust = "medium"
asset = "api"
``````toml
# Single-node entry
[[entrypoint]]
node = "my_module:handle_request" # node id, or "module.path:function"
kind = "api" # user_input | api | database | file_system | third_party
trust = "untrusted_external" # untrusted_external | semi_trusted_external | trusted_internal
asset_value = "high" # high | medium | low
description = "HTTP POST /auth"
# Rule: every PHP script under public_html/ is a web-exposed entrypoint.
[[entrypoint]]
file_glob = "public_html/**/*.php"
kind = "user_input"
trust = "untrusted_external"
asset_value = "high"
description = "Web-exposed PHP script"
# Rule: any function that takes a PSR-7 request object.
[[entrypoint]]
param_type = "ServerRequestInterface"
kind = "api"
trust = "untrusted_external"
asset_value = "high"
description = "PSR-7 HTTP handler"
# Rule: functions named `handle_*`.
[[entrypoint]]
name_regex = "^handle_"
kind = "api"
trust = "untrusted_external"
# Rule: conditions compose with AND — web.py files AND name starts with handle_.
[[entrypoint]]
file_glob = "public/*.py"
name_regex = "^handle_"
kind = "api"
trust = "untrusted_external"
Later entries override earlier ones when two rules tag the same node, so place broad rules first and specific corrections after.
当两条规则标记同一节点时,后出现的条目会覆盖先出现的条目,因此请将宽泛的规则放在前面,具体的修正放在后面。
See docs/entrypoint-patterns.md for the full reference, including frameworks not yet implemented (Express / Koa / Fastify, Laravel, Cobra, axum, warp, clap, and others) with grep-ready patterns contributors can use to add new detectors.
完整参考请参见 docs/entrypoint-patterns.md,其中包含尚未实现的框架(Express / Koa / Fastify、Laravel、Cobra、axum、warp、clap 等),并附有 contributors 可直接使用的 grep 模式,用于添加新的检测器。
Solidity detection uses parser metadata rather than signature regexes. Interface
declarations are excluded and a derived override suppresses the matching base
implementation. Concrete public and external functions remain entrypoints,
including view and pure functions; their solidity_visibility and
solidity_mutability attributes are returned by attack_surface() so callers
can distinguish read-only exposure. attack_surface() includes parser-specific
entrypoint attributes when they are attached to the underlying graph node.
Solidity 检测使用解析器元数据而非签名正则表达式。接口声明被排除,派生覆盖会抑制匹配的基实现。具体的 public 和 external 函数仍然作为入口点,包括 view 和 pure 函数;它们的 solidity_visibility 和 solidity_mutability 属性由 attack_surface() 返回,以便调用者区分只读暴露。当底层图节点附带了解析器特定的入口点属性时,attack_surface() 也会包含这些属性。
Polyglot parsing merges language graphs, but many RPC, FFI, subprocess, and
host/contract relationships are not visible in source syntax. Declare these
deterministically in .trailmark/links.toml:
多语言解析会合并语言图,但许多 RPC、FFI、子进程以及主机/合约关系在源代码语法中不可见。请在 .trailmark/links.toml 中确定性地声明这些关系:```toml
[[link]]
source = "backend:submit"
target = "contract:Verifier.verify"
kind = "calls" # defaults to calls
confidence = "certain" # defaults to inferred
description = "JSON-RPC eth_call"
[[link]] source = "backend:notify" target = "payments-webhook" external = true # required when either endpoint is unresolved
引用可以是精确的节点ID或唯一的名称/后缀。歧义引用、未知内部端点、无效枚举值和格式错误的TOML会引发`ValueError`。将`external = true`显式允许未解析的端点并创建代理节点。此文件是稳定的公共配置接口。
### 分析限制
- `entrypoint_paths_to()`报告调用图可达性,而非攻击者控制的数据流。使用预分析污点结果作为粗略的独立信号;Trailmark尚不执行过程间污点分析。
- TypeScript解析直接调用和通过`new ConcreteClass()`分配的简单接收者。通过清单、计算属性名、依赖注入容器和其他动态机制的接口分派仍是尽力而为。
- SQL支持面向PostgreSQL,提取模式、表、视图、函数、过程以及例程/视图依赖关系。它不是完整的SQL方言验证器或查询语义分析器。
### 编程API```python
from trailmark.parse import parse_directory, parse_file
from trailmark.query.api import QueryEngine
# Parse-only API: get the raw CodeGraph without building GraphStore/QueryEngine.
graph = parse_file("path/to/file.py")
graph = parse_directory("path/to/project", language="auto")
# Single-language (default) or auto-detect + merge across all languages
engine = QueryEngine.from_directory("path/to/project")
engine = QueryEngine.from_directory("path/to/project", language="auto")
engine = QueryEngine.from_directory("path/to/project", language="python,rust")
# Direct neighbors
engine.callers_of("handle_request")
engine.callees_of("handle_request")
# Transitive slicing — who could reach this sink, or what could it reach?
engine.ancestors_of("Auth._check_sig")
engine.reachable_from("handle_request")
# Attack-surface paths from any detected entrypoint
engine.entrypoint_paths_to("Auth._check_sig")
# All call paths between two nodes
engine.paths_between("handle_request", "Auth._check_sig")
# Functions with cyclomatic complexity >= 10
engine.complexity_hotspots(10)
# What functions can raise a given exception? (uses parser-detected
# exception_types; no runtime tracing required)
engine.functions_that_raise("PermissionError")
# Add and query semantic annotations
from trailmark.models.annotations import AnnotationKind
engine.annotate(
"handle_request",
AnnotationKind.ASSUMPTION,
"Caller has already authenticated the session token",
source="llm",
)
engine.annotations_of("handle_request")
engine.nodes_with_annotation(AnnotationKind.FINDING)
# Diff against an earlier snapshot of the same codebase
before = QueryEngine.from_directory("before/")
diff = engine.diff_against(before)
# diff contains: summary_delta, nodes {added/removed/modified},
# edges {added/removed}, entrypoints {added/removed/modified}
# Run the built-in audit-oriented preanalysis passes
engine.preanalysis()
engine.findings()
engine.subgraph_names()
# Programmatic augmentation hooks for external tooling
engine.augment_sarif("results.sarif")
engine.augment_weaudit("findings.json")
NodeKind.SCHEMA、TABLE、VIEW 和 PROCEDURE 在 v0.5.0 中是新增的;需要穷举匹配枚举值的消费者应为其添加分支。
uv sync --all-groups
uv run ruff check --fix uv run ruff format
uv tool install ty && ty check
uv run pytest -q tests/
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES uv run mutmut run
## 许可证
Apache-2.0
| 方法 | 描述 |
|---|
callers_of(name) | 指定目标的所有直接调用方 |
callees_of(name) | 指定源的所有直接被调用方 |
ancestors_of(name) | 所有可传递到达目标(向上切片)的函数 |
reachable_from(name) | 所有从源可传递到达的函数 |
paths_between(src, dst) | 两个节点间的所有简单调用路径 |
connect_subgraphs(source, target) | 连接两个命名子图的路径 |
entrypoint_paths_to(name) | 从任何检测到的入口点到目标的路径 |
attack_surface() | 标记了信任级别、资产价值以及解析器属性(如果存在)的入口点 |
complexity_hotspots(n) | 圈复杂度 ≥ n 的函数 |
functions_that_raise(exc) | 解析器检测到的异常列表中包含exc的函数 |
generic_parameters(name) | 节点声明的泛型类型参数 |
type_references(name) | 参数、返回值、异常以及泛型边界的类型引用 |
annotate(name, kind, description, source) | 向节点添加语义注解 |
annotations_of(name, kind=None) | 获取一个节点的注解,可选的通过kind过滤 |
nodes_with_annotation(kind) | 所有标记了给定注解类型的节点 |
clear_annotations(name, kind=None) | 移除一个节点的注解 |
diff_against(other) | 当前引擎的图与另一个图的结构化差异 |
preanalysis() | 运行内置预处理分析并向图形添加注解/子图 |
augment_sarif(path) | 将SARIF发现合并到图中 |
augment_weaudit(path) | 将weAudit发现合并到图中 |
augment_binary(path) | 合并外部二进制分析图JSON文件 |
findings(kind=None) | 返回带有查找类型注解的节点 |
subgraph(name) | 返回指定命名子图中的节点 |
subgraph_edges(name) | 返回指定命名子图内部的导出边 |
subgraph_names() | 列出图中当前所有命名的子图 |
summary() | 节点计数、边计数、依赖关系 |
to_json() | 导出完整图形 |