pytm：一个基于 Python 的威胁建模框架

简介

传统的威胁建模往往在开发后期才介入，有时甚至根本不做。此外，手动创建数据流和报告可能极其耗时。pytm 的目标是将威胁建模左移，使其更加自动化并以开发者为中心。

功能

根据您对架构设计的输入和定义，pytm 可以自动生成以下内容：

数据流图 (DFD)
序列图
与您的系统相关的威胁

要求

Linux/MacOS
Python 3.11+
Graphviz 包
Java (OpenJDK 10 或 11)
plantuml.jar

快速开始

tm.py 是一个示例模型。您可以通过运行它来生成它引用的报告和图表图像文件：``` mkdir -p tm ./tm.py --report docs/basic_template.md | pandoc -f markdown -t html > tm/report.html ./tm.py --dfd | dot -Tpng -o tm/dfd.png ./tm.py --seq | java -Djava.awt.headless=true -jar $PLANTUML_PATH -tpng -pipe > tm/seq.png

还有一个示例 `Makefile`，它将所有这些包装成目标，可以轻松共享给多个模型。如果你已经安装了 [GNU make](https://www.gnu.org/software/make/)（在Linux发行版上默认可用，但在OSX上不可用），只需运行：```
make MODEL=the_name_of_your_model_minus_.py

您应该将 plantuml.jar 放在与模型相同的目录中，或者设置 PLANTUML_PATH 环境变量。

为了避免安装所有依赖项（如 pandoc 或 Java），可以在容器内运行脚本：```

do this only once

export USE_DOCKER=true make image

call this after every change in your model

make

### 入门指南 - Devbox 变体

为了简化 `pytm` 的使用，可以通过 [`Devbox`](https://github.com/jetify-com/devbox) 完全隔离主机依赖项。这通常是比 OCI 容器方式开销更低且更方便的替代方案。

- 在 Linux/MacOS 上安装 Devbox：`curl -fsSL https://get.jetify.com/devbox | bash`
- 在 [Windows/WSL](https://www.jetify.com/docs/devbox/installing-devbox/index#installing-wsl2) 上安装 Devbox
- 更新 devbox 至最新版本：`devbox version update`
- 在 `~/.config/nix/nix.conf` 文件中设置你的 GitHub 访问令牌：`access-tokens = github.com=YOUR_TOKEN_HERE`
- 创建一个新的隔离 shell 环境，其中包含项目 `devbox.json` 文件中指定的所有工具和包：`devbox shell`
- 通过使用 which python 命令，显示直接在终端中输入 `python` 时将使用的 Python 可执行文件的完整路径。输出应为以下路径：`.devbox/nix/profile/default/bin/python`
- 通过运行以下命令进行测试，该命令应生成一个名为 `sample.png` 的 PNG 格式 DFD 图：`./tm.py --dfd | dot -Tpng -o sample.png`
- 退出 Devbox shell 环境：`exit`

## 使用

所有可用参数：```text
usage: tm.py [-h] [--debug] [--dfd] [--report REPORT] [--exclude EXCLUDE]
             [--seq] [--list] [--colormap] [--describe DESCRIBE]
             [--list-elements] [--json JSON] [--levels LEVELS [LEVELS ...]]
             [--stale_days STALE_DAYS]

options:
  -h, --help            show this help message and exit
  --debug               print debug messages
  --dfd                 output DFD
  --report REPORT       output report using the named template file (sample
                        template file is under docs/template.md)
  --exclude EXCLUDE     specify threat IDs to be ignored
  --seq                 output sequential diagram
  --list                list all available threats
  --colormap            color the risk in the diagram
  --describe DESCRIBE   describe the properties available for a given element
  --list-elements       list all elements which can be part of a threat model
  --json JSON           output a JSON file
  --levels LEVELS [LEVELS ...]
                        Select levels to be drawn in the threat model (int
                        separated by comma).
  --stale_days STALE_DAYS
                        checks if the delta between the TM script and the code
                        described by it is bigger than the specified value in
                        days

stale_days 参数试图确定您正在编写的模型脚本与实现被建模系统的代码之间相隔的天数。理想情况下，在大多数活跃开发的系统中，它们应该相当接近。您可以定期运行此参数来衡量项目的脉搏以及威胁模型的“新鲜度”。

当前可用的元素有：TM、Element、Server、ExternalEntity、Datastore、Actor、Process、SetOfProcesses、Dataflow、Boundary、Lambda、LLM 和 Agent。

元素的可用属性可以通过使用 --describe 后跟元素名称来列出：```text $ ./tm.py --describe Server Server class attributes: OS Operating system default: '' assumptions Assumptions about the element. These optionally allow to exclude threats with the given SIDs default factory: list controls Security controls for this element default factory: Controls data pytm.Data object(s) in incoming data flows default factory: DataSet description Description of the element default: '' findings Threats that apply to this element default factory: list handlesResources Does this asset handle resources? default: False inBoundary Trust boundary this element exists in default: None inScope Is the element in scope of the threat model default: True inputs incoming Dataflows default factory: list is_drawn default: False levels List of levels (0, 1, 2, ...) to be drawn in the model default factory: maxClassification Maximum data classification this element can handle default: <Classification.UNKNOWN: 0> minTLSVersion Minimum TLS version required default: <TLSVersion.NONE: 0> name Name of the element required onAWS Is this asset on AWS? default: False outputs outgoing Dataflows default factory: list overrides Overrides to findings, allowing to set a custom response, CVSS score or override other attributes default factory: list port Default TCP port for incoming data flows default: -1 protocol Default network protocol for incoming data flows default: '' severity Severity level of threats affecting this element default: 0 sourceFiles Location of the source code that describes this element relative to the directory of the model script default factory: list usesCache Does this server use cache? default: False usesEnvironmentVariables Does this asset use environment variables? default: False usesSessionTokens Does this server use session tokens? default: False usesVPN Does this server use VPN? default: False usesXMLParser Does this server use XML parser? default: False uuid default factory:

The *colormap* argument, used together with *dfd*, outputs a color-coded DFD where the elements are painted red, yellow or green depending on their risk level (as identified by running the rules).

## 使用 - Devbox 变体

- `devbox shell`
- `pytm` 照常使用
- `exit`

## 创建威胁模型

以下是一个示例 `tm.py` 文件，描述了一个简单的应用程序，用户登录该应用程序并在应用上发布评论。应用程序服务器将这些评论存储到数据库中。还有一个 AWS Lambda 定期清理数据库。```python

#!/usr/bin/env python3

from pytm import TM, Server, Datastore, Dataflow, Boundary, Actor, Lambda, LLM, Data, Classification, DatastoreType


tm = TM("my test tm")
tm.description = "another test tm"
tm.isOrdered = True

User_Web = Boundary("User/Web")
Web_DB = Boundary("Web/DB")

user = Actor("User")
user.inBoundary = User_Web

web = Server("Web Server")
web.OS = "CloudOS"
web.controls.isHardened = True
web.sourceFiles = ["server/web.cc"]

db = Datastore("SQL Database (*)")
db.OS = "CentOS"
db.controls.isHardened = False
db.inBoundary = Web_DB
db.type = DatastoreType.SQL
db.inScope = False
db.sourceFiles = ["model/schema.sql"]

comments = Data(
    name="Comments", 
    description="Comments in HTML or Markdown",  
    classification=Classification.PUBLIC,  
    isPII=False,
    isCredentials=False,  
    # credentialsLife=Lifetime.LONG,  
    isStored=True, 
    isSourceEncryptedAtRest=False, 
    isDestEncryptedAtRest=True 
)

results = Data(
    name="results", 
    description="Results of insert op",  
    classification=Classification.SENSITIVE,  
    isPII=False, 
    isCredentials=False,  
    # credentialsLife=Lifetime.LONG,  
    isStored=True, 
    isSourceEncryptedAtRest=False, 
    isDestEncryptedAtRest=True 
)

my_lambda = Lambda("cleanDBevery6hours")
my_lambda.controls.hasAccessControl = True
my_lambda.inBoundary = Web_DB

llm_api = LLM("AI Writing Assistant")
llm_api.isThirdParty = True
llm_api.processesPersonalData = True
llm_api.hasContentFiltering = False
llm_api.hasSystemPrompt = True
llm_api.processesUntrustedInput = True

my_lambda_to_db = Dataflow(my_lambda, db, "(&lambda;)Periodically cleans DB")
my_lambda_to_db.protocol = "SQL"
my_lambda_to_db.dstPort = 3306

user_to_web = Dataflow(user, web, "User enters comments (*)")
user_to_web.protocol = "HTTP"
user_to_web.dstPort = 80
user_to_web.data = comments

web_to_user = Dataflow(web, user, "Comments saved (*)")
web_to_user.protocol = "HTTP"

web_to_db = Dataflow(web, db, "Insert query with comments")
web_to_db.protocol = "MySQL"
web_to_db.dstPort = 3306

db_to_web = Dataflow(db, web, "Comments contents")
db_to_web.protocol = "MySQL"
db_to_web.data = results

web_to_llm = Dataflow(web, llm_api, "Chat completion request")
web_to_llm.protocol = "HTTPS"
web_to_llm.dstPort = 443

tm.process()

您还可以选择使用 pytmGPT 通过散文创建模型！

生成图表

图表输出为 Dot 和 PlantUML。

当 --dfd 参数传递给上述 tm.py 文件时，它会生成输出到标准输出，该输出被馈送到 Graphviz 的 dot 以生成数据流图：```bash

tm.py --dfd | dot -Tpng -o sample.png

生成此图：

dfd.png

为元素添加 ".levels = [1,2]" 属性将导致该元素（以及其关联的数据流，如果两个流端点位于同一DFD级别）根据命令行参数 "--levels 1 2" 显示或不显示。

以下命令生成一个序列图。```bash

tm.py --seq | java -Djava.awt.headless=true -jar plantuml.jar -tpng -pipe > seq.png

生成此图：

seq.png

创建报告

可以将图表和发现结果包含在模板中，以生成最终报告：```bash

tm.py --report docs/basic_template.md | pandoc -f markdown -t html > report.html

报告模板中使用的模板格式非常简单：```text

# Threat Model Sample
***

## System Description

{tm.description}

## Dataflow Diagram

![Level 0 DFD](https://raw.githubusercontent.com/owasp/pytm/master/dfd.png)

## Dataflows

Name|From|To |Data|Protocol|Port
----|----|---|----|--------|----
{dataflows:repeat:{{item.name}}|{{item.source.name}}|{{item.sink.name}}|{{item.data}}|{{item.protocol}}|{{item.dstPort}}
}

## Findings

{findings:repeat:* {{item.description}} on element "{{item.target}}"
}

要按照元素对发现进行分组，请使用更高级的嵌套循环：```text

Findings

{elements🔁{{item.findings:if:

{{item.findings🔁 Threat: {{{{item.id}}}} - {{{{item.description}}}}

Severity: {{{{item.severity}}}}

Mitigations: {{{{item.mitigations}}}}

References: {{{{item.references}}}}

}}}}}

循环中的所有项都必须转义，将大括号加倍，因此`{item.name}`变为`{{item.name}}`。
上面的示例使用了两个嵌套循环，因此内层循环中的项必须转义两次，这就是为什么它们使用了四个大括号。

### 覆盖

你可以覆盖发现项（与模型资产和/或数据流匹配的威胁）的属性，例如设置自定义CVSS评分和/或响应文本：```python
user_to_web = Dataflow(user, web, "User enters comments (*)", protocol="HTTP", dstPort="80")
user_to_web.overrides = [
    Finding(
        # Overflow Buffers
        threat_id="INP02",
        cvss="9.3",
        response="""**To Mitigate**: run a memory sanitizer to validate the binary""",
        severity="Very High",
    )
]

如果你正在添加一个发现项，请确保添加一个严重级别："非常高"、"高"、"中"、"低"、"非常低"。

威胁数据库

对于安全从业人员，你可以通过设置 TM.threatsFile 来提供自己的威胁文件。它应该包含像这样的条目：```json { "SID":"INP01", "target": ["Lambda","Process"], "description": "Buffer Overflow via Environment Variables", "details": "This attack pattern involves causing a buffer overflow through manipulation of environment variables. Once the attacker finds that they can modify an environment variable, they may try to overflow associated buffers. This attack leverages implicit trust often placed in environment variables.", "Likelihood Of Attack": "High", "severity": "High", "condition": "target.usesEnvironmentVariables is True and target.controls.sanitizesInput is False and target.controls.checksInputBounds is False", "prerequisites": "The application uses environment variables.An environment variable exposed to the user is vulnerable to a buffer overflow.The vulnerable environment variable uses untrusted data.Tainted data used in the environment variables is not properly validated. For instance boundary checking is not done before copying the input data to a buffer.", "mitigations": "Do not expose environment variable to the user.Do not use untrusted data in your environment variables. Use a language or compiler that performs automatic bounds checking. There are tools such as Sharefuzz [R.10.3] which is an environment variable fuzzer for Unix that support loading a shared library. You can use Sharefuzz to determine if you are exposing an environment variable vulnerable to buffer overflow.", "example": "Attack Example: Buffer Overflow in $HOME A buffer overflow in sccw allows local users to gain root access via the $HOME environmental variable. Attack Example: Buffer Overflow in TERM A buffer overflow in the rlogin program involves its consumption of the TERM environmental variable.", "references": "https://capec.mitre.org/data/definitions/10.html, CVE-1999-0906, CVE-1999-0046, http://cwe.mitre.org/data/definitions/120.html, http://cwe.mitre.org/data/definitions/119.html, http://cwe.mitre.org/data/definitions/680.html" }

`target` 字段列出了要与此威胁匹配的模型元素类。这些可以是资产，例如：Actor、Datastore、Server、Process、SetOfProcesses、ExternalEntity、Lambda、LLM、Agent 或 Element（Element 是基类，可匹配任何资产）。也可以是连接两个资产的数据流。

所有其他字段（除了 `condition`）都可用于显示，并可在模板中使用，以便在最终[报告](#report)中列出发现结果。

> **警告**
>
> `threats.json` 文件包含通过 `eval()` 运行的字符串。请确保该文件具有正确的权限，否则攻击者可能会修改字符串并导致您代表他们运行代码。

逻辑存在于 `condition` 中，其中 `target` 的成员可以进行逻辑评估。返回 true 表示该规则生成一个发现，否则不生成。条件可以比较 `target` 的属性，和/或 `target.control` 的属性，还可以调用以下方法之一：

* `target.oneOf(class, ...)`，其中 `class` 是一个或多个：Actor、Datastore、Server、Process、SetOfProcesses、ExternalEntity、Lambda、LLM、Agent 或 Dataflow，
* `target.crosses(Boundary)`，
* `target.enters(Boundary)`，
* `target.exits(Boundary)`，
* `target.inside(Boundary)`。

如果 `target` 是 Dataflow，请记住您可以访问 `target.source` 和/或 `target.sink` 以及其他属性。

对资产的条件可以通过检查 `target.input` 和 `target.output` 属性来分析所有传入和传出的数据流。例如，要仅针对有传入流量的服务器匹配威胁，请使用 `any(target.inputs)`。一个更高级的示例，匹配连接到 SQL 数据存储的元素，将是 `any(f.sink.oneOf(Datastore) and f.sink.type == DatastoreType.SQL for f in target.outputs)`。

## 从 JSON 导入

通过少量的 Python 代码，可以从 JSON 导入威胁模型（请注意示例 `tests/input.json` 中的特殊格式）。以下示例导入 tests 中的 `input.json` 示例。将以下代码保存为 `tm2.py`。```python

#!/usr/bin/env python3
# Example tm2.py contents
# Run: python tm2.py --dfd | dot -Tpng -o sample_json.png

from pytm import (
    TM,
    Actor,
    Boundary,
    Classification,
    Data,
    Dataflow,
    Datastore,
    Lambda,
    Server,
    DatastoreType,
    Assumption,
    load,
)

json_file_string = './tests/input.json'
with open(json_file_string) as input_json:
    TM.reset()
    tm = load(input_json)
    tm.process()

我们可以像之前一样调用 tm2.py，这里使用 --dfd 参数，然后将输出重定向到 Graphviz (dot):```bash

python tm2.py --dfd | dot -Tpng -o sample_json.png

## 制作幻灯片！

威胁模型完成并准备就绪后，就进入令人畏惧的演示阶段——现在 pytm 也能在这方面帮到你，它提供了一个模板，利用强大的 [RevealMD](https://github.com/webpro/reveal-md) 将威胁模型以幻灯片形式呈现！只需使用模板 `docs/revealjs.md`，你就能获得一些漂亮的、完全可配置的幻灯片，并能在浏览器中展示和分享。



https://github.com/izar/pytm/assets/368769/30218241-c7cc-4085-91e9-bbec2843f838



## 当前支持的威胁```text
INP01 - Buffer Overflow via Environment Variables
INP02 - Overflow Buffers
INP03 - Server Side Include (SSI) Injection
CR01 - Session Sidejacking
INP04 - HTTP Request Splitting
CR02 - Cross Site Tracing
INP05 - Command Line Execution through SQL Injection
INP06 - SQL Injection through SOAP Parameter Tampering
SC01 - JSON Hijacking (aka JavaScript Hijacking)
LB01 - API Manipulation
AA01 - Authentication Abuse/ByPass
DS01 - Excavation
DE01 - Interception
DE02 - Double Encoding
API01 - Exploit Test APIs
AC01 - Privilege Abuse
INP07 - Buffer Manipulation
AC02 - Shared Data Manipulation
DO01 - Flooding
HA01 - Path Traversal
AC03 - Subverting Environment Variable Values
DO02 - Excessive Allocation
DS02 - Try All Common Switches
INP08 - Format String Injection
INP09 - LDAP Injection
INP10 - Parameter Injection
INP11 - Relative Path Traversal
INP12 - Client-side Injection-induced Buffer Overflow
AC04 - XML Schema Poisoning
DO03 - XML Ping of the Death
AC05 - Content Spoofing
INP13 - Command Delimiters
INP14 - Input Data Manipulation
DE03 - Sniffing Attacks
CR03 - Dictionary-based Password Attack
API02 - Exploit Script-Based APIs
HA02 - White Box Reverse Engineering
DS03 - Footprinting
AC06 - Using Malicious Files
HA03 - Web Application Fingerprinting
SC02 - XSS Targeting Non-Script Elements
AC07 - Exploiting Incorrectly Configured Access Control Security Levels
INP15 - IMAP/SMTP Command Injection
HA04 - Reverse Engineering
SC03 - Embedding Scripts within Scripts
INP16 - PHP Remote File Inclusion
AA02 - Principal Spoof
CR04 - Session Credential Falsification through Forging
DO04 - XML Entity Expansion
DS04 - XSS Targeting Error Pages
SC04 - XSS Using Alternate Syntax
CR05 - Encryption Brute Forcing
AC08 - Manipulate Registry Information
DS05 - Lifting Sensitive Data Embedded in Cache
SC05 - Removing Important Client Functionality
INP17 - XSS Using MIME Type Mismatch
AA03 - Exploitation of Trusted Credentials
AC09 - Functionality Misuse
INP18 - Fuzzing and observing application log data/errors for application mapping
CR06 - Communication Channel Manipulation
AC10 - Exploiting Incorrectly Configured SSL
CR07 - XML Routing Detour Attacks
AA04 - Exploiting Trust in Client
CR08 - Client-Server Protocol Manipulation
INP19 - XML External Entities Blowup
INP20 - iFrame Overlay
AC11 - Session Credential Falsification through Manipulation
INP21 - DTD Injection
INP22 - XML Attribute Blowup
INP23 - File Content Injection
DO05 - XML Nested Payloads
AC12 - Privilege Escalation
AC13 - Hijacking a privileged process
AC14 - Catching exception throw/signal from privileged block
INP24 - Filter Failure through Buffer Overflow
INP25 - Resource Injection
INP26 - Code Injection
INP27 - XSS Targeting HTML Attributes
INP28 - XSS Targeting URI Placeholders
INP29 - XSS Using Doubled Characters
INP30 - XSS Using Invalid Characters
INP31 - Command Injection
INP32 - XML Injection
INP33 - Remote Code Inclusion
INP34 - SOAP Array Overflow
INP35 - Leverage Alternate Encoding
DE04 - Audit Log Manipulation
AC15 - Schema Poisoning
INP36 - HTTP Response Smuggling
INP37 - HTTP Request Smuggling
INP38 - DOM-Based XSS
AC16 - Session Credential Falsification through Prediction
INP39 - Reflected XSS
INP40 - Stored XSS
AC17 - Session Hijacking - ServerSide
AC18 - Session Hijacking - ClientSide
INP41 - Argument Injection
AC19 - Reusing Session IDs (aka Session Replay) - ServerSide
AC20 - Reusing Session IDs (aka Session Replay) - ClientSide
AC21 - Cross Site Request Forgery
DS06 - Data Leak
DR01 - Unprotected Sensitive Data
AC22 - Credentials Aging (deprecated)
AC23 - Credentials Disclosure
AC24 - Use of hardcoded credentials
LLM01 - Direct Prompt Injection
LLM02 - Indirect Prompt Injection via Retrieved Content
LLM03 - Sensitive Data Leakage to Third-Party Provider
LLM04 - Training Data Poisoning
LLM05 - Excessive Agency via Unauthorized Tool Use
LLM06 - Arbitrary Code Execution via LLM Agent
LLM07 - Jailbreaking and Safety Bypass
LLM08 - Sensitive Information Disclosure Through Output
LLM09 - Untrusted Tool Launch Configuration

pytm

最受欢迎