Syntribosは主にLinuxとMac環境で開発されており、ほとんどのUnixおよびLinuxベースのオペレーティングシステムで動作します。現時点ではWindowsはサポートしていませんが、将来変更される可能性があります。

.. _Documentation: https://docs.openstack.org/developer/syntribos/
.. _Apache license: https://github.com/openstack/syntribos/blob/master/LICENSE
.. _Launchpad project: https://launchpad.net/syntribos
.. _Blueprints: https://blueprints.launchpad.net/syntribos
.. _Bugs: https://bugs.launchpad.net/syntribos
.. _Source code: https://github.com/openstack/syntribos

============
インストール
============

Syntribosは`pipを使用してpypi <https://pypi.python.org/pypi/pip>`__から直接インストールできます。

::

   pip install syntribos

最新の変更を取り入れるには、`ソース <https://www.github.com/openstack/syntribos.git>`__から`pip <https://pypi.python.org/pypi/pip>`__を使用してsyntribosをインストールします。

リポジトリをクローンします::

   $ git clone https://github.com/openstack/syntribos.git

クローンしたリポジトリのディレクトリに移動し、pipでインストールします::

   $ cd syntribos
   $ pip install .

======================================
syntribos環境の初期化
======================================

syntribosがインストールされたら、syntribos環境を初期化する必要があります。これは手動で行うか、``init``コマンドを使用して行えます。

::

    $ syntribos init

.. Note::
    デフォルトでは、``syntribos init``は開発チームが管理する`リモートリポジトリ <https://github.com/openstack/syntribos-payloads>`_から一連のデフォルトペイロードファイルを取得します。これらのペイロードファイルはファズテストを実行するために必要です。この動作を無効にするには、``--no_downloads``フラグを指定してsyntribosを実行します。ペイロードファイルは、常時 ``syntribos download --payloads``を実行することでも取得できます。

syntribosをインストールするカスタムルートを指定するには、``--custom_root``フラグを指定します。これにより、ターミナルからの情報入力を求めるプロンプトがスキップされます。これはJenkinsジョブやユーザー入力が取得できない状況で便利です。

既に``init``コマンドを実行したが、新しい環境を最初から始めたい場合は、``--force``フラグを指定して既存のファイルを上書きできます。``--custom_root``と``--force``フラグを組み合わせて、カスタムインストールルート内のファイルを上書きすることもできます。

注意：カスタムインストールルートにsyntribosをインストールした場合、syntribosを実行する際に``--custom_root``フラグを指定する必要があります。

**例:**

::

    $ syntribos --custom_root /your/custom/path init --force
    $ syntribos --custom_root /your/custom/path run

=============
設定
=============

すべての設定ファイルには``[syntribos]``セクションが必要です。使用しているエクステンションやテスト対象に応じて、他のセクションを追加します。たとえば、組み込みのidentityエクステンションを使用している場合は、``[user]``セクションが必要です。``[logging]``および``[remote]``セクションはオプションです。

syntribos設定ファイルの基本構造は以下の通りです::

    [syntribos]
    #
    # テスト対象サービスのエンドポイントURLとバージョン
    #
    endpoint=http://localhost:5000
    # ペイロードとテンプレートのパスを設定
    templates=<location_of_templates_dir/file>
    payloads=<location_of_payloads_dir>

    [user]
    #
    # AUTH_TOKENを取得するためのユーザー認証情報とエンドポイントURL
    # このセクションはidentityエクステンションを使用する場合のみ必要です
    #
    endpoint=
    username=<yourusername>
    password=<yourpassword>

    [remote]
    #
    # オプション: リモートURIとcache_dirを明示的に定義する場合
    #
    templates_uri=<URI to a tar file of set of templates>
    payloads_uri=<URI to a tar file of set of payloads>
    cache_dir=<a local path to save the downloaded files>

    [logging]
    log_dir=<location_to_save_debug_logs>

``[syntribos]``セクションで指定するエンドポイントURLは、syntribosがテストするエンドポイントURLです。``[user]``セクションのエンドポイントURLはAUTH_TOKENを取得するために使用されます。任意のプロジェクトをテストするには、``[syntribos]``の下のエンドポイントURLをAPIを指すように更新し、必要に応じてユーザー認証情報も変更します。

リモートからのテンプレートとペイロードのダウンロード~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Payload and template files can be downloaded remotely in syntribos.
In the config file under the ``[syntribos]`` section, if the ``templates``
and ``payloads`` options are not set, by default syntribos will
download all the latest payloads and the templates for a few OpenStack
projects.

To specify a URI to download custom templates and payloads
from, use the ``[remotes]`` section in the config file.
Available options under ``[remotes]`` are ``cache_dir``, ``templates_uri``,
``payloads_uri``, and ``enable_cache``. The ``enable_cache`` option is
``True`` by default; set to ``False`` to disable caching of remote
content while syntribos is running. If the ``cache_dir`` set to a path,
syntribos will attempt to use that as a base directory to save downloaded
template and payload files.

The advantage of using these options are that you will be able to get
the latest payloads from the official repository and if you are
using syntribos to test OpenStack projects, then, in most cases you
could directly use the well defined templates available with this option.

This option also helps to easily manage different versions of templates
remotely, without the need to maintain a set of different versions offline.

Testing OpenStack keystone API

$ vi examples/configs/keystone.conf



[syntribos]
#
# As keystone is being tested in the example, enter your
#
# keystone auth endpoint url.
endpoint=http://localhost:5000
# Set payload and templates path
templates=<location_of_templates_dir/file>
payloads=<location_of_payloads_dir>

[user]
#
# User credentials
#
endpoint=http://localhost:5000
username=<yourusername>
password=<yourpassword>
# Optional, only needed if Keystone V3 API is used
#user_id=<youruserid>
# Optional, api version if required
#version=v2.0
# Optional, for getting scoped tokens
#user_id=<alt_userid>
# If user id is not known
# For V3 API
#domain_name=<name_of_the_domain>
#project_name=<name_of_the_project>
# For Keystone V2 API
#tenant_name=<name_of_the_project>

#[alt_user]
#
# Optional, Used for cross auth tests (-t AUTH)
#
#endpoint=http://localhost:5000
#username=<alt_username>
#password=<alt_password>
# Optional, for getting scoped tokens
#user_id=<alt_userid>
# If user id is not known
# For V3 API
#domain_name=<name_of_the_domain>
#project_name=<name_of_the_project>
# For Keystone V2 API
#tenant_name=<name_of_the_project>

[remote]
#
# Optional, Used to specify URLs of templates and payloads
#
#cache_dir=<a local path to save the downloaded files>
#templates_uri=https://github.com/your_project/templates.tar
#payloads_uri=https://github.com/your_project/payloads.tar
# To disable caching of these remote contents, set the following variable to False
#enable_caching=True

[logging]
#
# Logger options go here
#
log_dir=<location_to_store_log_files>
# Optional, compresses http_request_content,
# if you don't want this, set this option to False.
http_request_compression=True

$ syntribos --config-file keystone.conf run

It is possible to limit syntribos to run a specific test type using
the ``-t`` flag.

::

    $ syntribos --config-file keystone.conf -t SQL run


This will match all tests that contain ``SQL`` in their name. For example:
``SQL_INJECTION_HEADERS``, ``SQL_INJECTION_BODY``, etc.

Specifying a custom root directory

The results log is displayed at the end of every syntribos run, it can be
written to a file by using the ``-o`` flag on the command line.

The results log includes failures and errors. The ``"failures"`` key represents
tests that have failed, indicating a possible security vulnerability. The
``"errors"`` key gives us information on any unhandled exceptions, such as
connection errors, encountered on that run.

Example failure object:

::

    {
       "defect_type": "xss_strings",
       "description": "The string(s): '[\"<STYLE>@import'http://xss.rocks/xss.css';</STYLE>\"]',
       known to be commonly returned after a successful XSS attack, have been found in the
       response. This could indicate a vulnerability to XSS attacks.",
       "failure_id": 33,
       "instances": [
          {
            "confidence": "LOW",
            "param": {
              "location": "data",
              "method": "POST",
              "type": null,
              "variables": [
                "type",
                "details/name",
              ]
          },
          "severity": "LOW",
          "signals": {
             "diff_signals": [
               "LENGTH_DIFF_OVER"
             ],
             "init_signals": [
               "HTTP_CONTENT_TYPE_JSON",
               "HTTP_STATUS_CODE_2XX_201"
             ],
             "test_signals": [
               "FAILURE_KEYS_PRESENT",
               "HTTP_CONTENT_TYPE_JSON",
               "HTTP_STATUS_CODE_2XX_201",
             ]
          },
          "strings": [
            "<STYLE>@import'http://xss.rocks/xss.css';</STYLE>"
             ]
          }
       ],
       "url": "127.0.0.1/test"
    }


Error form:

::

    ERROR:
    {
      "error": "Traceback (most recent call last):\n  File \"/Users/test/syntribos/tests/fuzz/base_fuzz.py\",
       line 58, in tearDownClass\n    super(BaseFuzzTestCase, cls).tearDownClass()\n
       File \"/Users/test/syntribos/tests/base.py\", line 166, in tearDownClass\n
       raise sig.data[\"exception\"]\nReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=8080):
       Read timed out. (read timeout=10)\n",
       "test": "tearDownClass (syntribos.tests.fuzz.sql.image_data_image_data_get.template_SQL_INJECTION_HEADERS_sql-injection.txt_str21_model1)"
    }


Debug Logs
~~~~~~~~~~

Debug logs include details about HTTP requests, HTTP responses, and other
debugging information such as errors and warnings across the project. The
path where debug logs are saved by default is ``.syntribos/logs/``.
Debug logs are arranged in directories based on the timestamp in these
directories and files are named according to the templates.

For example:

::

    $ ls .syntribos/logs/
    2016-09-15_11:06:37.198412 2016-09-16_10:11:37.834892 2016-09-16_13:31:36.362584
    2016-09-15_11:34:33.271606 2016-09-16_10:38:55.820827 2016-09-16_13:36:43.151048
    2016-09-15_11:41:53.859970 2016-09-16_10:39:50.501820 2016-09-16_13:40:23.203920

::

    $ ls .syntribos/logs/2016-09-16_13:31:36.362584
    API_Versions::list_versions_template.log
    API_Versions::show_api_details_template.log
    availability_zones::get_availability_zone_detail_template.log
    availability_zones::get_availability_zone_template.log
    cells::delete_os_cells_template.log
    cells::get_os_cells_capacities_template.log
    cells::get_os_cells_data_template.log

Each log file includes some essential debugging information such as the string
representation of the request object, signals, and checks used for tests, etc.

Example request::

    ------------
    REQUEST SENT
    ------------
    request method.......: PUT
    request url..........: http://127.0.0.1/api
    request params.......:
    request headers size.: 7
    request headers......: {'Content-Length': '0', 'Accept-Encoding': 'gzip, deflate',
    'Accept': 'application/json',
    'X-Auth-Token': <uuid>, 'Connection': 'keep-alive',
    'User-Agent': 'python-requests/2.11.1', 'content-type': 'application/xml'}
    request body size....: 0
    request body.........: None

Example response::

    -----------------
    RESPONSE RECEIVED
    -----------------
    response status..: <Response [415]>
    response headers.: {'Content-Length': '70',
    'X-Compute-Request-Id': <random id>,
    'Vary': 'OpenStack-API-Version, X-OpenStack-Nova-API-Version',
    'Openstack-Api-Version': 'compute 2.1', 'Connection': 'close',
    'X-Openstack-Nova-Api-Version': '2.1', 'Date': 'Fri, 16 Sep 2016 14:15:27 GMT',
    'Content-Type': 'application/json; charset=UTF-8'}
    response time....: 0.036277
    response size....: 70
    response body....: {"badMediaType": {"message": "Unsupported Content-Type", "code": 415}}
    -------------------------------------------------------------------------------
    [2590]  :  XSS_BODY
    (<syntribos.clients.http.client.SynHTTPClient object at 0x102c65f10>, 'PUT',
    'http://127.0.0.1/api')
    {'headers': {'Accept': 'application/json', 'X-Auth-Token': <uuid> },
    'params': {}, 'sanitize': False, 'data': '', 'requestslib_kwargs': {'timeout': 10}}
    Starting new HTTP connection (1): 127.0.0.1
    "PUT http://127.0.0.1/api HTTP/1.1" 501 93

Example signals captured::

    Signals: ['HTTP_STATUS_CODE_4XX_400', 'HTTP_CONTENT_TYPE_JSON']
    Checks used: ['HTTP_STATUS_CODE', 'HTTP_CONTENT_TYPE']

Debug logs are sanitized to prevent storing secrets to log files.
Passwords and other sensitive information are marked with asterisks using a
slightly modified version of `oslo_utils.strutils.mask_password <https://docs.openstack.org/developer/oslo.utils/api/strutils.html#oslo_utils.strutils.mask_password>`__.

Debug logs also include string compression, wherein long fuzz strings are
compressed before being written to the logs. The threshold to start data
compression is set to 512 characters. Although it is not recommended to turn
off compression, it is possible by setting the variable
``"http_request_compression"``, under the logging section in the config file,
to ``False``.


=============================
Anatomy of a request template
=============================

This section describes how to write templates and how to run specific tests.
Templates are input files which have raw HTTP requests and may be
supplemented with variable data using extensions.

In general, a request template is a marked-up raw HTTP request. It's possible
for you to test your application by using raw HTTP requests as your request
templates, but syntribos allows you to mark-up your request templates for
further functionality.

A request template looks something like this:

::

    POST /users/{user1} HTTP/1.1
    Content-Type: application/json
    X-Auth-Token: CALL_EXTERNAL|syntribos.extensions.vAPI.client:get_token:[]|

    {"newpassword": "qwerty123"}

For fuzz tests, syntribos will automatically detect URL parameters, headers,
and body content as fields to fuzz. It will not automatically detect URL path
elements as fuzz fields, but they can be specified with curly braces ``{}``.

Note: The name of a template file must end with the extension ``.template``
Otherwise, syntribos will skip the file and will not attempt to parse any files
that do not adhere to this naming scheme.

Using external functions in templates

CALL_EXTERNAL|{extension dot path}:{function name}:[arguments]

X-Auth-Token: CALL_EXTERNAL|syntribos.extensions.identity.client:get_token_v3:["user"]|

"username": "CALL_EXTERNAL|syntribos.extensions.random_data.client:get_uuid:[]|"

CALL_EXTERNAL|common_utils.client.{method_name}:{comma separated parameters in square brackets}

"encoded_url": "CALL_EXTERNAL|common_utils.client:url_encode:['http://localhost:5000']|

CALL_EXTERNAL|random_data.client.{method_name}:{comma separated parameters in square brackets}

"address": "CALL_EXTERNAL|random_data.client:random_ip:[]|"

While syntribos is designed to test all fields in a request, it can also
ignore specific fields through the use of Action Fields. If you want to
fuzz against a static object ID, use the Action Field indicator as
follows:

::

    "ACTION_FIELD:id": "1a16f348-c8d5-42ec-a474-b1cdf78cf40f"

The ID provided will remain static for every test.

Meta Variable File

{
    "user_password": {
        "val": 1234
    },
    "user_name": {
        "type": config,
        "val": "user.username"
        "fuzz_types": ["ascii"]
    },
    "user_token": {
        "type": "function",
        "val": "syntribos.extensions.identity:get_scoped_token_v3",
        "args": ["user"],
        "fuzz": false
    }
}

POST /user HTTP/1.1
X-Auth-Token: |user_token|

{
    "user": {
        "username": "|user_name|",
        "password": "|user_password|"
    }
}

As mentioned above, some tests included with syntribos by default
are: LDAP injection, SQL injection, integer overflow, command injection,
XML external entity, reflected cross-site scripting,
Cross Origin Resource Sharing (CORS), SSL, Regex Denial of Service,
JSON Parser Depth Limit, and User defined.

In order to run a specific test, use the `-t, --test-types`
option and provide ``syntribos`` with a keyword, or keywords, to match from
the test files located in ``syntribos/tests/``.

For SQL injection tests, see below:

::

    $ syntribos --config-file keystone.conf -t SQL run

To run SQL injection tests against the template body only, see below:

::

    $ syntribos --config-file keystone.conf -t SQL_INJECTION_BODY run

For all tests against HTTP headers only, see below:

::

    $ syntribos --config-file keystone.conf -t HEADERS run


============
Unit testing
============

To execute unit tests automatically, navigate to the ``syntribos`` root
directory and install the test requirements.

::

    $ pip install -r test-requirements.txt

Now, run the ``unittest`` as below:

::

    $ python -m unittest discover tests/unit -p "test_*.py"

If you have configured tox you could also run the following:

::

    $ tox -e py27
    $ tox -e py35

This will run all the unit tests and give you a result output
containing the status and coverage details of each test.

=======================
Contributing Guidelines
=======================

Syntribos is an open source project and contributions are always
welcome. If you have any questions, we can be found in the
#openstack-security channel on Freenode IRC.

1. Follow all the `OpenStack Style Guidelines <https://docs.openstack.org/developer/hacking/>`__
   (e.g. PEP8, Py3 compatibility)
2. Follow `secure coding guidelines <https://security.openstack.org/#secure-development-guidelines>`__
3. Ensure all classes/functions have appropriate `docstrings <https://www.python.org/dev/peps/pep-0257/>`__
   in  `RST format <http://docutils.sourceforge.net/docs/user/rst/quickref.html>`__
4. Include appropriate unit tests for all new code(place them in the
   ``tests/unit`` folder)
5. Test any change you make using tox:

  ::

    pip install tox
    tox -e pep8
    tox -e py27
    tox -e py35
    tox -e cover

Anyone wanting to contribute to OpenStack must follow
`the OpenStack development workflow <https://docs.openstack.org/infra/manual/developers.html#development-workflow>`__

Submit all changes through the code review process in Gerrit
described above. All pull requests on Github will be closed/ignored.

File bugs on the `syntribos launchpad site <https://bugs.launchpad.net/syntribos>`__,
and not on Github. All Github issues will be closed/ignored.

Submit blueprints `here <https://blueprints.launchpad.net/syntribos>`__ for all
breaking changes, feature requests, and other unprioritized work.


.. Note:: README.rst is a file that can be generated by running
   ``python readme.py`` from the ``syntribos/scripts`` directory. When the
   README file needs to be updated; modify the corresponding rst file in
   ``syntribos/doc/source`` and have it generate by running the script.