pytm：一个Python式的威胁建模框架

简介

传统的威胁建模往往姗姗来迟，有时甚至根本不会进行。此外，手动创建数据流和报告极其耗时。pytm 的目标是将威胁建模前置，使其更加自动化并以开发人员为中心。

功能特性

基于您对架构设计的输入和定义，pytm 可以自动生成以下内容：

数据流图（DFD）
序列图
与您的系统相关的威胁

依赖要求

Linux/MacOS
Python 3.11+
Graphviz 软件包
Java（OpenJDK 10 或 11）
plantuml.jar

快速开始

tm.py 是一个示例模型。您可以运行它以生成其引用的报告和图表图像文件：``` mkdir -p tm ./tm.py --report docs/basic_template.md | pandoc -f markdown -t html > tm/report.html ./tm.py --dfd | dot -Tpng -o tm/dfd.png ./tm.py --seq | java -Djava.awt.headless=true -jar $PLANTUML_PATH -tpng -pipe > tm/seq.png

还有一个示例 `Makefile`，它将所有这些内容封装成目标，可以轻松地共享给多个模型。如果你已安装 [GNU make](https://www.gnu.org/software/make/)（Linux 发行版默认安装，但 OSX 未预装），只需运行：```
make MODEL=the_name_of_your_model_minus_.py

你应该将 plantuml.jar 放在与你的模型相同的目录中，或者设置 PLANTUML_PATH。

为了避免安装所有依赖项（如 pandoc 或 Java），该脚本可以在容器内运行：```

do this only once

export USE_DOCKER=true make image

call this after every change in your model

make

### 入门指南 - Devbox 变体

为了简化 `pytm` 的使用，可以将主机依赖项完全隔离在
[`Devbox`](https://github.com/jetify-com/devbox) 环境中。这通常是
比 OCI 容器方法更低开销且更方便的替代方案。

- 在 Linux/MacOS 上安装 Devbox：`curl -fsSL https://get.jetify.com/devbox | bash`
- 在 [Windows/WSL](https://www.jetify.com/docs/devbox/installing-devbox/index#installing-wsl2) 上安装 Devbox
- 更新到最新版本的 devbox：`devbox version update`
- 在 `~/.config/nix/nix.conf` 文件中设置你的 GitHub 访问令牌：`access-tokens = github.com=YOUR_TOKEN_HERE`
- 创建一个新的、隔离的 shell 环境，其中包含项目 `devbox.json` 文件中指定的所有工具和包：`devbox shell`
- 使用 `which python` 命令显示在终端中直接输入 `python` 时将使用的 Python 可执行文件的完整路径。输出应为以下路径：`.devbox/nix/profile/default/bin/python`
- 通过运行以下命令进行测试，该命令将生成一个名为 `sample.png` 的 DFD 图片文件：`./tm.py --dfd | dot -Tpng -o sample.png`
- 退出 Devbox shell 环境：`exit`

## 使用方法

所有可用参数：```text
usage: tm.py [-h] [--debug] [--dfd] [--report REPORT]
             [--exclude EXCLUDE] [--seq] [--list] [--describe DESCRIBE]
             [--list-elements] [--json JSON] [--levels LEVELS [LEVELS ...]]
             [--stale_days STALE_DAYS]

optional arguments:
  -h, --help            show this help message and exit
  --debug               print debug messages
  --dfd                 output DFD
  --report REPORT       output report using the named template file (sample
                        template file is under docs/template.md)
  --exclude EXCLUDE     specify threat IDs to be ignored
  --seq                 output sequential diagram
  --list                list all available threats
  --colormap            color the risk in the diagram
  --describe DESCRIBE   describe the properties available for a given element
  --list-elements       list all elements which can be part of a threat model
  --json JSON           output a JSON file
  --levels LEVELS [LEVELS ...]
                        Select levels to be drawn in the threat model (int
                        separated by comma).
  --stale_days STALE_DAYS
                        checks if the delta between the TM script and the code
                        described by it is bigger than the specified value in
                        days

stale_days 参数试图确定你正在编写的模型脚本与实现被建模系统的代码之间相隔的天数。理想情况下，在大多数活跃开发的系统中，两者应相当接近。你可以定期运行此参数来衡量项目的脉搏以及威胁模型的“新鲜度”。目前可用的元素包括：TM、Element、Server、ExternalEntity、Datastore、Actor、Process、SetOfProcesses、Dataflow、Boundary、Lambda、LLM 和 Agent。可以使用 --describe 后面跟上元素名称来列出元素的可用属性：```text

(pytm) ➜ pytm git:(master) ✗ ./tm.py --describe Element Element class attributes: OS definesConnectionTimeout default: False description handlesResources default: False implementsAuthenticationScheme default: False implementsNonce default: False inBoundary inScope Is the element in scope of the threat model, default: True isAdmin default: False isHardened default: False name required onAWS default: False

*colormap* 参数与 *dfd* 结合使用时，会输出一个颜色编码的数据流图（DFD），其中元素根据其风险等级（通过运行规则识别）被涂成红色、黄色或绿色。


## 使用方法 - Devbox 变体

- `devbox shell`
- `pytm` 使用方式与平常相同
- `exit`

## 创建威胁模型

以下是一个示例 `tm.py` 文件，描述了一个简单的应用程序：用户登录应用程序并在应用上发布评论。应用服务器将这些评论存储到数据库中。还有一个 AWS Lambda 函数定期清理数据库。```python

#!/usr/bin/env python3

from pytm import TM, Server, Datastore, Dataflow, Boundary, Actor, Lambda, LLM, Data, Classification

tm = TM("my test tm")
tm.description = "another test tm"
tm.isOrdered = True

User_Web = Boundary("User/Web")
Web_DB = Boundary("Web/DB")

user = Actor("User")
user.inBoundary = User_Web

web = Server("Web Server")
web.OS = "CloudOS"
web.isHardened = True
web.sourceCode = "server/web.cc"

db = Datastore("SQL Database (*)")
db.OS = "CentOS"
db.isHardened = False
db.inBoundary = Web_DB
db.isSql = True
db.inScope = False
db.sourceCode = "model/schema.sql"

comments = Data(
    name="Comments", 
    description="Comments in HTML or Markdown",  
    classification=Classification.PUBLIC,  
    isPII=False,
    isCredentials=False,  
    # credentialsLife=Lifetime.LONG,  
    isStored=True, 
    isSourceEncryptedAtRest=False, 
    isDestEncryptedAtRest=True 
)

results = Data(
    name="results", 
    description="Results of insert op",  
    classification=Classification.SENSITIVE,  
    isPII=False, 
    isCredentials=False,  
    # credentialsLife=Lifetime.LONG,  
    isStored=True, 
    isSourceEncryptedAtRest=False, 
    isDestEncryptedAtRest=True 
)

my_lambda = Lambda("cleanDBevery6hours")
my_lambda.hasAccessControl = True
my_lambda.inBoundary = Web_DB

llm_api = LLM("AI Writing Assistant")
llm_api.isThirdParty = True
llm_api.processesPersonalData = True
llm_api.hasContentFiltering = False
llm_api.hasSystemPrompt = True
llm_api.processesUntrustedInput = True

my_lambda_to_db = Dataflow(my_lambda, db, "(&lambda;)Periodically cleans DB")
my_lambda_to_db.protocol = "SQL"
my_lambda_to_db.dstPort = 3306

user_to_web = Dataflow(user, web, "User enters comments (*)")
user_to_web.protocol = "HTTP"
user_to_web.dstPort = 80
user_to_web.data = comments

web_to_user = Dataflow(web, user, "Comments saved (*)")
web_to_user.protocol = "HTTP"

web_to_db = Dataflow(web, db, "Insert query with comments")
web_to_db.protocol = "MySQL"
web_to_db.dstPort = 3306

db_to_web = Dataflow(db, web, "Comments contents")
db_to_web.protocol = "MySQL"
db_to_web.data = results

web_to_llm = Dataflow(web, llm_api, "Chat completion request")
web_to_llm.protocol = "HTTPS"
web_to_llm.dstPort = 443

tm.process()

您也可以选择使用 pytmGPT 从文本创建模型！

生成图表

图表输出为 Dot 和 PlantUML 格式。

当将 --dfd 参数传递给上述 tm.py 文件时，它会生成输出到 stdout，然后由 Graphviz 的 dot 处理以生成数据流图：```bash

tm.py --dfd | dot -Tpng -o sample.png

生成此图：

dfd.png

为元素添加“.levels = [1,2]”属性将使其（以及其关联的数据流，如果两个流端点位于同一DFD级别）根据命令行参数“--levels 1 2”来渲染（或不渲染）。

以下命令生成一个序列图。```bash

tm.py --seq | java -Djava.awt.headless=true -jar plantuml.jar -tpng -pipe > seq.png

生成此图：

seq.png

创建报告

可将图表和发现结果包含在模板中，以生成最终报告：```bash

tm.py --report docs/basic_template.md | pandoc -f markdown -t html > report.html

报告中使用的模板格式非常简单：```text

# Threat Model Sample
***

## System Description

{tm.description}

## Dataflow Diagram

![Level 0 DFD](https://raw.githubusercontent.com/izar/pytm/master/dfd.png)

## Dataflows

Name|From|To |Data|Protocol|Port
----|----|---|----|--------|----
{dataflows:repeat:{{item.name}}|{{item.source.name}}|{{item.sink.name}}|{{item.data}}|{{item.protocol}}|{{item.dstPort}}
}

## Findings

{findings:repeat:* {{item.description}} on element "{{item.target}}"
}

要按元素对发现进行分组，请使用更高级的嵌套循环：```text

Findings

{elements🔁{{item.findings:if:

{{item.findings🔁 Threat: {{{{item.id}}}} - {{{{item.description}}}}

Severity: {{{{item.severity}}}}

Mitigations: {{{{item.mitigations}}}}

References: {{{{item.references}}}}

}}}}}

所有循环内的条目必须转义，将大括号加倍，因此 `{item.name}` 变成 `{{item.name}}`。  
上面的例子使用了两个嵌套循环，因此内层循环中的条目必须转义两次，这就是为什么它们使用了四个大括号。

### 覆写

你可以覆写发现（威胁匹配模型资产和/或数据流）的属性，例如设置自定义的CVSS评分和/或响应文本：```python
user_to_web = Dataflow(user, web, "User enters comments (*)", protocol="HTTP", dstPort="80")
user_to_web.overrides = [
    Finding(
        # Overflow Buffers
        threat_id="INP02",
        cvss="9.3",
        response="""**To Mitigate**: run a memory sanitizer to validate the binary""",
        severity="Very High",
    )
]

如果你正在添加一个发现项，请确保添加严重性："Very High", "High", "Medium", "Low", "Very Low"。

威胁数据库

对于安全从业者，你可以通过设置 TM.threatsFile 来提供自己的威胁文件。它应包含如下条目：```json { "SID":"INP01", "target": ["Lambda","Process"], "description": "Buffer Overflow via Environment Variables", "details": "This attack pattern involves causing a buffer overflow through manipulation of environment variables. Once the attacker finds that they can modify an environment variable, they may try to overflow associated buffers. This attack leverages implicit trust often placed in environment variables.", "Likelihood Of Attack": "High", "severity": "High", "condition": "target.usesEnvironmentVariables is True and target.controls.sanitizesInput is False and target.controls.checksInputBounds is False", "prerequisites": "The application uses environment variables.An environment variable exposed to the user is vulnerable to a buffer overflow.The vulnerable environment variable uses untrusted data.Tainted data used in the environment variables is not properly validated. For instance boundary checking is not done before copying the input data to a buffer.", "mitigations": "Do not expose environment variable to the user.Do not use untrusted data in your environment variables. Use a language or compiler that performs automatic bounds checking. There are tools such as Sharefuzz [R.10.3] which is an environment variable fuzzer for Unix that support loading a shared library. You can use Sharefuzz to determine if you are exposing an environment variable vulnerable to buffer overflow.", "example": "Attack Example: Buffer Overflow in $HOME A buffer overflow in sccw allows local users to gain root access via the $HOME environmental variable. Attack Example: Buffer Overflow in TERM A buffer overflow in the rlogin program involves its consumption of the TERM environmental variable.", "references": "https://capec.mitre.org/data/definitions/10.html, CVE-1999-0906, CVE-1999-0046, http://cwe.mitre.org/data/definitions/120.html, http://cwe.mitre.org/data/definitions/119.html, http://cwe.mitre.org/data/definitions/680.html" }

`target` 字段列出了要与此威胁匹配的模型元素类别。这些可以是资产，例如：Actor、Datastore、Server、Process、SetOfProcesses、ExternalEntity、Lambda、LLM、Agent 或 Element（基类，可匹配任何元素）。它也可以是连接两个资产的数据流（Dataflow）。

所有其他字段（除 `condition` 外）均可用于显示，并可在模板中使用以在最终[报告](#report)中列出发现项。

> **警告**
>
> `threats.json` 文件包含会通过 `eval()` 执行的字符串。请确保该文件具有正确的权限，否则攻击者可能修改字符串并导致您替他们运行代码。

逻辑存在于 `condition` 中，在此可以对 `target` 的成员进行逻辑评估。返回 true 表示规则生成一项发现，否则不发。条件可以比较 `target` 的属性以及/或者 `target.control` 的控制属性，还可以调用以下任一方法：

* `target.oneOf(class, ...)` 其中 `class` 是一个或多个：Actor、Datastore、Server、Process、SetOfProcesses、ExternalEntity、Lambda、LLM、Agent 或 Dataflow，
* `target.crosses(Boundary)`,
* `target.enters(Boundary)`,
* `target.exits(Boundary)`,
* `target.inside(Boundary)`.

如果 `target` 是一个数据流（Dataflow），请记住你可以访问 `target.source` 和/或 `target.sink` 以及其他属性。

资产的条件下可以检查 `target.input` 和 `target.output` 属性来分析所有传入和传出的数据流。例如，仅匹配具有传入流量的服务器威胁，使用 `any(target.inputs)`。更高级的例子，匹配连接到 SQL 数据存储的元素，可以写为 `any(f.sink.oneOf(Datastore) and f.sink.isSQL for f in target.outputs)`。

## Importing from JSON

通过一点 Python 代码，可以从 JSON 导入威胁模型（注意在 `tests/input.json` 中示例的特殊格式）。以下示例导入了在测试中的 `input.json` 示例。将以下代码保存为 `tm2.py`。```python

#!/usr/bin/env python3
# Example tm2.py contents
# Run: python tm2.py --dfd | dot -Tpng -o sample_json.png

from pytm import (
    TM,
    Actor,
    Boundary,
    Classification,
    Data,
    Dataflow,
    Datastore,
    Lambda,
    Server,
    DatastoreType,
    Assumption,
    load,
)

json_file_string = './tests/input.json'
with open(json_file_string) as input_json:
    TM.reset()
    tm = load(input_json)
    tm.process()

我们可以像之前一样调用tm2.py，这里使用--dfd参数，然后将输出重定向到Graphviz（dot）：```bash

python tm2.py --dfd | dot -Tpng -o sample_json.png

## 制作幻灯片！

一旦威胁模型完成并准备就绪，就进入令人畏惧的演示阶段——现在 pytm 也可以在此帮助您，通过使用 (RevealMD)[https://github.com/webpro/reveal-md] 的功能，提供了一个模板，将您的威胁模型以幻灯片形式呈现！只需使用模板 docs/revealjs.md，您就能获得一些漂亮的幻灯片，完全可配置，可以从浏览器中演示和分享。

https://github.com/izar/pytm/assets/368769/30218241-c7cc-4085-91e9-bbec2843f838

## 当前支持的威胁```text
INP01 - Buffer Overflow via Environment Variables
INP02 - Overflow Buffers
INP03 - Server Side Include (SSI) Injection
CR01 - Session Sidejacking
INP04 - HTTP Request Splitting
CR02 - Cross Site Tracing
INP05 - Command Line Execution through SQL Injection
INP06 - SQL Injection through SOAP Parameter Tampering
SC01 - JSON Hijacking (aka JavaScript Hijacking)
LB01 - API Manipulation
AA01 - Authentication Abuse/ByPass
DS01 - Excavation
DE01 - Interception
DE02 - Double Encoding
API01 - Exploit Test APIs
AC01 - Privilege Abuse
INP07 - Buffer Manipulation
AC02 - Shared Data Manipulation
DO01 - Flooding
HA01 - Path Traversal
AC03 - Subverting Environment Variable Values
DO02 - Excessive Allocation
DS02 - Try All Common Switches
INP08 - Format String Injection
INP09 - LDAP Injection
INP10 - Parameter Injection
INP11 - Relative Path Traversal
INP12 - Client-side Injection-induced Buffer Overflow
AC04 - XML Schema Poisoning
DO03 - XML Ping of the Death
AC05 - Content Spoofing
INP13 - Command Delimiters
INP14 - Input Data Manipulation
DE03 - Sniffing Attacks
CR03 - Dictionary-based Password Attack
API02 - Exploit Script-Based APIs
HA02 - White Box Reverse Engineering
DS03 - Footprinting
AC06 - Using Malicious Files
HA03 - Web Application Fingerprinting
SC02 - XSS Targeting Non-Script Elements
AC07 - Exploiting Incorrectly Configured Access Control Security Levels
INP15 - IMAP/SMTP Command Injection
HA04 - Reverse Engineering
SC03 - Embedding Scripts within Scripts
INP16 - PHP Remote File Inclusion
AA02 - Principal Spoof
CR04 - Session Credential Falsification through Forging
DO04 - XML Entity Expansion
DS04 - XSS Targeting Error Pages
SC04 - XSS Using Alternate Syntax
CR05 - Encryption Brute Forcing
AC08 - Manipulate Registry Information
DS05 - Lifting Sensitive Data Embedded in Cache
SC05 - Removing Important Client Functionality
INP17 - XSS Using MIME Type Mismatch
AA03 - Exploitation of Trusted Credentials
AC09 - Functionality Misuse
INP18 - Fuzzing and observing application log data/errors for application mapping
CR06 - Communication Channel Manipulation
AC10 - Exploiting Incorrectly Configured SSL
CR07 - XML Routing Detour Attacks
AA04 - Exploiting Trust in Client
CR08 - Client-Server Protocol Manipulation
INP19 - XML External Entities Blowup
INP20 - iFrame Overlay
AC11 - Session Credential Falsification through Manipulation
INP21 - DTD Injection
INP22 - XML Attribute Blowup
INP23 - File Content Injection
DO05 - XML Nested Payloads
AC12 - Privilege Escalation
AC13 - Hijacking a privileged process
AC14 - Catching exception throw/signal from privileged block
INP24 - Filter Failure through Buffer Overflow
INP25 - Resource Injection
INP26 - Code Injection
INP27 - XSS Targeting HTML Attributes
INP28 - XSS Targeting URI Placeholders
INP29 - XSS Using Doubled Characters
INP30 - XSS Using Invalid Characters
INP31 - Command Injection
INP32 - XML Injection
INP33 - Remote Code Inclusion
INP34 - SOAP Array Overflow
INP35 - Leverage Alternate Encoding
DE04 - Audit Log Manipulation
AC15 - Schema Poisoning
INP36 - HTTP Response Smuggling
INP37 - HTTP Request Smuggling
INP38 - DOM-Based XSS
AC16 - Session Credential Falsification through Prediction
INP39 - Reflected XSS
INP40 - Stored XSS
AC17 - Session Hijacking - ServerSide
AC18 - Session Hijacking - ClientSide
INP41 - Argument Injection
AC19 - Reusing Session IDs (aka Session Replay) - ServerSide
AC20 - Reusing Session IDs (aka Session Replay) - ClientSide
AC21 - Cross Site Request Forgery
DS06 - Data Leak
DR01 - Unprotected Sensitive Data
AC22 - Credentials Aging (deprecated)
AC23 - Credentials Disclosure
AC24 - Use of hardcoded credentials
LLM01 - Direct Prompt Injection
LLM02 - Indirect Prompt Injection via Retrieved Content
LLM03 - Sensitive Data Leakage to Third-Party Provider
LLM04 - Training Data Poisoning
LLM05 - Excessive Agency via Unauthorized Tool Use
LLM06 - Arbitrary Code Execution via LLM Agent
LLM07 - Jailbreaking and Safety Bypass
LLM08 - Sensitive Information Disclosure Through Output
LLM09 - Untrusted Tool Launch Configuration

pytm

最受欢迎