git-vuln-finder

git-vuln-finder logo

Encontrando posibles vulnerabilidades de software a partir de mensajes de confirmación de git. El formato de salida es un JSON con la confirmación asociada que podría contener una corrección relacionada con una vulnerabilidad de software. La búsqueda se basa en un conjunto de expresiones regulares aplicadas únicamente a los mensajes de confirmación. Si hay IDs de CVE presentes, estos se añaden automáticamente en la salida. La entrada puede ser cualquier repositorio de git o una fuente de archivo GH.

Requisitos

jq (sudo apt install jq)
Python poetry

Instalación

Usarlo como biblioteca

git-vuln-finder se puede instalar con poetry. Si no tienes poetry instalado, puedes hacer lo siguiente: curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python.

$ poetry install
$ poetry shell
$ git-vuln-finder -h

También puedes usar pip. Luego solo impórtalo:

Python 3.8.0 (default, Dec 11 2019, 21:43:13)
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from git_vuln_finder import find
>>> all_potential_vulnerabilities, all_cve_found, found = find("~/git/curl")

>>> [commit for commit, summary in all_potential_vulnerabilities.items() if summary['state'] == 'cve-assigned']
['9069838b30fb3b48af0123e39f664cea683254a5', 'facb0e4662415b5f28163e853dc6742ac5fafb3d',
... snap ...
 '8a75dbeb2305297640453029b7905ef51b87e8dd', '1dc43de0dccc2ea7da6dddb7b98f8d7dcf323914', '192c4f788d48f82c03e9cef40013f34370e90737', '2eb8dcf26cb37f09cffe26909a646e702dbcab66', 'fa1ae0abcde5df8d0b3283299e3f246bedf7692c', 'c11c30a8c8d727dcf5634fa0cc6ee0b4b77ddc3d', '75ca568fa1c19de4c5358fed246686de8467c238', 'a20daf90e358c1476a325ea665d533f7a27e3364', '042cc1f69ec0878f542667cb684378869f859911']

 >>> print(json.dumps(all_potential_vulnerabilities['9069838b30fb3b48af0123e39f664cea683254a5'], sort_keys=True, indent=4, separators=(",", ": ")))
 {
     "author": "Daniel Stenberg",
     "author-email": "[email protected]",
     "authored_date": 1567544372,
     "branches": [
         "master"
     ],
     "commit-id": "9069838b30fb3b48af0123e39f664cea683254a5",
     "committed_date": 1568009674,
     "cve": [
         "CVE-2019-5481",
         "CVE-2019-5481"
     ],
     "language": "en",
     "message": "security:read_data fix bad realloc()\n\n... that could end up a double-free\n\nCVE-2019-5481\nBug: https://curl.haxx.se/docs/CVE-2019-5481.html\n",
     "origin": "https://github.com/curl/curl.git",
     "origin-github-api": "https://api.github.com/repos///github.com/curl/curl/commits/9069838b30fb3b48af0123e39f664cea683254a5",
     "pattern-matches": [
         "double-free"
     ],
     "pattern-selected": "(?i)(double[-| ]free|buffer overflow|double free|race[-| ]condition)",
     "state": "cve-assigned",
     "stats": {
         "deletions": 4,
         "files": 1,
         "insertions": 2,
         "lines": 6
     },
     "summary": "security:read_data fix bad realloc()",
     "tags": []
 }

Usarlo como herramienta de línea de comandos

$ git clone https://github.com/cve-search/git-vuln-finder.git
$ cd https://github.com/cve-search/git-vuln-finder.git
$ pip install .
$ git-vuln-finder --help

También puedes usar pip. pipx instala scripts (disponibles en todo el sistema) proporcionados por paquetes de Python en entornos virtuales separados para aislarlos de tu sistema y entre sí.

Uso

usage: git-vuln-finder [-h] [-v] [-r R] [-o O] [-s S] [-p P] [-c] [-t] [-gh GH]

Finding potential software vulnerabilities from git commit messages.

optional arguments:
  -h, --help  show this help message and exit
  -v          increase output verbosity
  -r R        git repository to analyse
  -o O        Output format: [json]
  -s S        State of the commit found
  -p P        Matching pattern to use: [vulnpatterns, cryptopatterns, cpatterns] - the pattern 'all' is used to match all the patterns at once.
  -c          output only a list of the CVE pattern found in commit messages (disable by default)
  -t          Include tags matching a specific commit
  -gh GH      special option for gharchive, pass a file containing a PushEvent in JSON format

More info: https://github.com/cve-search/git-vuln-finder

Patrones

git-vuln-finder incluye 3 patrones predeterminados que se pueden seleccionar para encontrar las posibles vulnerabilidades descritas en los mensajes de confirmación, como:

vulnpatterns es un patrón de vulnerabilidad genérico que apunta especialmente a aplicaciones web y mensajes de confirmación de seguridad genéricos. Basado en un artículo académico.
cryptopatterns es un patrón de vulnerabilidad para errores criptográficos mencionados en mensajes de confirmación.
cpatterns es un conjunto de patrones de vulnerabilidad estándar típicos en lenguajes similares a C/C++.

Una muestra de salida parcial del repositorio git de Curl

$ git-vuln-finder -r ~/git/curl | jq .
...
 "6df916d751e72fc9a1febc07bb59c4ddd886c043": {
    "message": "loadlibrary: Only load system DLLs from the system directory\n\nInspiration provided by: Daniel Stenberg and Ray Satiro\n\nBug: https://curl.haxx.se/docs/adv_20160530.html\n\nRef: Windows DLL hijacking with curl, CVE-2016-4802\n",
    "language": "en",
    "commit-id": "6df916d751e72fc9a1febc07bb59c4ddd886c043",
    "summary": "loadlibrary: Only load system DLLs from the system directory",
    "stats": {
      "insertions": 180,
      "deletions": 8,
      "lines": 188,
      "files": 7
    },
    "author": "Steve Holme",
    "author-email": "[email protected]",
    "authored_date": 1464555460,
    "committed_date": 1464588867,
    "branches": [
      "master"
    ],
    "pattern-selected": "(?i)(denial of service |\bXXE\b|remote code execution|\bopen redirect|OSVDB|\bvuln|\bCVE\b |\bXSS\b|\bReDoS\b|\bNVD\b|malicious|x−frame−options|attack|cross site |exploit|malicious|directory traversal |\bRCE\b|\bdos\b|\bXSRF \b|\bXSS\b|clickjack|session.fixation|hijack|\badvisory|\binsecure |security |\bcross−origin\b|unauthori[z|s]ed |infinite loop)",
    "pattern-matches": [
      "hijack"
    ],
    "origin": "[email protected]:curl/curl.git",
    "origin-github-api": "https://api.github.com/repos/curl/curl/commits/6df916d751e72fc9a1febc07bb59c4ddd886c043",
    "tags": [],
    "cve": [
      "CVE-2016-4802"
    ],
    "state": "cve-assigned"
  },
  "c2b3f264cb5210f82bdc84a3b89250a611b68dd3": {
    "message": "CONNECT_ONLY: don't close connection on GSS 401/407 reponses\n\nPreviously, connections were closed immediately before the user had a\nchance to extract the socket when the proxy required Negotiate\nauthentication.\n\nThis regression was brought in with the security fix in commit\n79b9d5f1a42578f\n\nCloses #655\n",
    "language": "en",
    "commit-id": "c2b3f264cb5210f82bdc84a3b89250a611b68dd3",
    "summary": "CONNECT_ONLY: don't close connection on GSS 401/407 reponses",
    "stats": {
      "insertions": 4,
      "deletions": 2,
      "lines": 6,
      "files": 1
    },
    "author": "Marcel Raad",
    "author-email": "[email protected]",
    "authored_date": 1455523116,
    "committed_date": 1461704516,
    "branches": [
      "master"
    ],
    "pattern-selected": "(?i)(denial of service |\bXXE\b|remote code execution|\bopen redirect|OSVDB|\bvuln|\bCVE\b |\bXSS\b|\bReDoS\b|\bNVD\b|malicious|x−frame−options|attack|cross site |exploit|malicious|directory traversal |\bRCE\b|\bdos\b|\bXSRF \b|\bXSS\b|clickjack|session.fixation|hijack|\badvisory|\binsecure |security |\bcross−origin\b|unauthori[z|s]ed |infinite loop)",
    "pattern-matches": [
      "security "
    ],
    "origin": "[email protected]:curl/curl.git",
    "origin-github-api": "https://api.github.com/repos/curl/curl/commits/c2b3f264cb5210f82bdc84a3b89250a611b68dd3",
    "tags": [],
    "state": "under-review"
  },
...

Extrayendo ID(s) de CVE de los mensajes de git

  "98d132cf6a879faf0147aa83ea0c07ff326260ed": {
    "message": "Add a macro for testing assertion in both debug and production builds\n\nIf we have an assert then in a debug build we want an abort() to occur.\nIn a production build we wan
t the function to return an error.\n\nThis introduces a new macro to assist with that. The idea is to replace\nexisting use of OPENSSL_assert() with this new macro. The problem with\nOPENSSL
_assert() is that it aborts() on an assertion failure in both debug\nand production builds. It should never be a library's decision to abort a\nprocess (we don't get to decide when to kill t
he life support machine or\nthe nuclear reactor control system). Additionally if an attacker can\ncause a reachable assert to be hit then this can be a source of DoS attacks\ne.g. see CVE-20
17-3733, CVE-2015-0293, CVE-2011-4577 and CVE-2002-1568.\n\nReviewed-by: Tim Hudson <[email protected]>\n(Merged from https://github.com/openssl/openssl/pull/3496)",
    "commit-id": "98d132cf6a879faf0147aa83ea0c07ff326260ed",
    "summary": "Add a macro for testing assertion in both debug and production builds",
    "stats": {
      "insertions": 18,
      "deletions": 0,
      "lines": 18,
      "files": 1
    },
    "author": "Matt Caswell",
    "author-email": "[email protected]",
    "authored_date": 1495182637,
    "committed_date": 1495457671,
    "branches": [
      "master"
    ],
    "pattern-selected": "(?i)(denial of service |\bXXE\b|remote code execution|\bopen redirect|OSVDB|\bvuln|\bCVE\b |\bXSS\b|\bReDoS\b|\bNVD\b|malicious|x−frame−options|attack|cross site |ex
ploit|malicious|directory traversal |\bRCE\b|\bdos\b|\bXSRF \b|\bXSS\b|clickjack|session.fixation|hijack|\badvisory|\binsecure |security |\bcross−origin\b|unauthori[z|s]ed |infinite loop)",
    "pattern-matches": [
      "attack"
    ],
    "cve": [
      "CVE-2017-3733",
      "CVE-2015-0293",
      "CVE-2011-4577",
      "CVE-2002-1568"
    ],
    "state": "cve-assigned"
  }

Uso para la opción especial gharchive

$ git-vuln-finder -gh ../tests/gharchive_test.json

El valor del parámetro gh debe ser un archivo JSON que contenga un array de cada PushEvent que se desea probar.

[
   {
    "id": "19351512310",
    "type": "PushEvent",
    "actor": {
      "id": 32466128,
      "login": "DavidCruciani",
      "display_login": "DavidCruciani",
      "gravatar_id": "",
      "url": "https://api.github.com/users/DavidCruciani",
      "avatar_url": "https://avatars.githubusercontent.com/u/32466128?"
    },
    "repo": {
      "id": 424660123,
      "name": "ail-project/ail-feeder-gharchive",
      "url": "https://api.github.com/repos/ail-project/ail-feeder-gharchive"
    },
    "payload": {
      "push_id": 8628652926,
      "size": 1,
      "distinct_size": 1,
      "ref": "refs/heads/main",
      "head": "910ed71a2819546a3f3bcce1ebb9e3984a8c8d86",
      "before": "40a9ef5dc6b2add5184a0a58401bfe9058faa8df",
      "commits": [
        {
          "sha": "910ed71a2819546a3f3bcce1ebb9e3984a8c8d86",
          "author": {
            "email": "[email protected]",
            "name": "David Cruciani"
          },
          "message": "chg: [feeder] case sensitive",
          "distinct": true,
          "url": "https://api.github.com/repos/ail-project/ail-feeder-gharchive/commits/910ed71a2819546a3f3bcce1ebb9e3984a8c8d86"
        }
      ]
    },
    "public": true,
    "created_at": "2021-12-15T16:06:43Z",
    "org": {
      "id": 62389074,
      "login": "ail-project",
      "gravatar_id": "",
      "url": "https://api.github.com/orgs/ail-project",
      "avatar_url": "https://avatars.githubusercontent.com/u/62389074?"
    }
  }
]

Uso para importación

Si el objetivo es importar el módulo para usarlo, el método a llamar es find_event

from git_vuln_finder import find_event

for element in event:
    for i in range(0,len(element["payload"]["commits"])):
        all_potential_vulnerabilities, all_cve_found, found = find_event(element["payload"]["commits"][i], element)

Salida con la opción gharchive

{
    "repo_name": "LeandroFChaves/gerenciador-alunos",
    "message": "[UI] - Ajustes no css da aplica\u00e7\u00e3o\n\n- Adicionado padding para a exibi\u00e7\u00e3o do conte\u00fado das p\u00e1ginas;\n- Alinhado os bot\u00f5es de a\u00e7\u00f5es dos forms a direita da table",
    "language": "pt",
    "commit-id": "73a1c68b520853198eaac199a41d141ee96dc64d",
    "author": "LeandroFChaves",
    "author-email": "[email protected]",
    "authored_date": "2021-10-01T03:00:07Z",
    "branches": "refs/heads/master",
    "pattern-selected": "(?i)(denial of service|\\bXXE\\b|remote code execution|\\bopen redirect|OSVDB|\\bvuln|\\bCVE\\b|\\bXSS\\b|\\bReDoS\\b|\\bNVD\\b|malicious|x\u2212frame\u2212options|attack|cross site|exploit|malicious|directory traversal|\\bRCE\\b|\\bdos\\b|\\bXSRF \\b|\\bXSS\\b|clickjack|session.fixation|hijack|\\badvisory|\\binsecure|security|\\bcross\u2212origin\\b|unauthori[z|s]ed|infinite loop)",
    "pattern-matches": [
        "dos"
    ],
    "origin-github-api": "https://api.github.com/repos/LeandroFChaves/gerenciador-alunos/commits/73a1c68b520853198eaac199a41d141ee96dc64d",
    "state": "under-review"
}

Ejecutando las pruebas

$ pytest

Advertencia

git-vuln-finder es una herramienta automática que detecta posibles vulnerabilidades en los mensajes de confirmación basándose en palabras clave específicas. Si tienes la intención de utilizar la información de vulnerabilidades descubierta, como el registro de CVEs, por favor revisa las vulnerabilidades descubiertas con los autores del software antes de proceder.

Licencia y autor(es)

Este software es software libre y está licenciado bajo la versión 3 de AGPL.

Agradecimientos

Gracias a Jean-Louis Huynen por las discusiones sobre los patrones de vulnerabilidad criptográfica.
Gracias a Sebastien Tricaud por las discusiones sobre idioma nativo, mensajes de confirmación y patrones externos.
Gracias a Cedric Bonhomme por convertir git-vuln-finder en una biblioteca Python, agregar pruebas y mejorar el proceso general de instalación.
Gracias a David Cruciani por el soporte de gharchive.

Contribuciones

Aceptamos contribuciones para el software y especialmente patrones de vulnerabilidad adicionales. Cada contribuyente será añadido en el archivo AUTHORS y será propietario colectivo de este software de código abierto. Los contribuyentes reconocen el Developer Certificate of Origin.

Referencias

Notas
https://csce.ucmss.com/cr/books/2017/LFS/CSREA2017/ICA2077.pdf (usando principalmente CVE referenciados en el mensaje de confirmación) - archivo (http://archive.is/xep9o)
https://asankhaya.github.io/pdf/automated-identification-of-security-issues-from-commit-messages-and-bug-reports.pdf (2 expresiones regulares principales)

git-vuln-finder

Más Populares

git-vuln-finder

Requisitos

Instalación

Usarlo como biblioteca

Usarlo como herramienta de línea de comandos

Uso

Patrones

Una muestra de salida parcial del repositorio git de Curl

Uso para la opción especial gharchive

Uso para importación

Salida con la opción gharchive

Ejecutando las pruebas

Advertencia

Licencia y autor(es)

Agradecimientos

Contribuciones

Referencias