License Supported versions https://readthedocs.org/projects/sphinx-test-reports/badge/?version=latest Travis-CI Build Status PyPI Package latest release

Sphinx-Test-Reports

Sphinx-Test-Reports shows test results inside Sphinx documentations.

Functions

  • By using test-results it shows test results of JUnit based xml reports as table. JUnit exports are supported by:
    • pytest
    • nosetest
    • Other test frameworks (including frameworks from java and co.) support also the JUnit format.
  • By using test-env it shows test environment information from tox-envreport based exports as table. (E.g. used operating system, python version, installed packages, …)

Examples

test result report

Input

my_data.xml

<testsuite name="My testsuite" tests="4" failures="1" errors="0" skips="1" time="0.572" >
    <testcase classname="test_basic_doc" name="test_doc_build_html"/>
    <testcase classname="test_empty_doc" name="test_doc_build_html"/>
    <testcase classname="test_junit_parser" name="test_init_parser">
        <failure type="KeyException"> key "name" could not be accessed in junit object </failure>
    </testcase>
    <testcase classname="test_junit_parser" name="test_call_parser">
        <skipped > Passed on the last test run </skipped>
    </testcase>
</testsuite>

my_document.rst

My Test Results
===============

.. test-results:: my_data.xml

Output

My testsuite

Tests: 4, Failures: 1, Errors: 0, Skips: 1

Time: 0.572

class name status reason

test_basic_doc

test_doc_build_html

passed

test_empty_doc

test_doc_build_html

passed

test_junit_parser

test_init_parser

failure

key “name” could not be accessed in junit object

test_junit_parser

test_call_parser

skipped

Passed on the last test run

test env report

Input

The following lines are just an excerpt of a json-file generated by tox-env-report.

tox-reports.json

{
    "py27": {
       "host": "user_abc",
       "installed_packages": [
          "arrow==0.10.0",
          "atomicwrites==1.2.1"
       ],
       "installpkg": {
          "basename": "erc_e2e-0.1.0.zip",
          "md5": "df03855c61c10ddd32f2c888856cc51e",
          "sha256": "576f9f616a18bc74404a0f241c96d9e3c75f03f091f8981495598d0c6dc25c41"
       },
       "name": "py35",
       "path": "/home/user_abc/projects/sphinx-test-reports",
       "platform": "linux",
       "python": {
          "executable": "/home/user_abc/projects/sphinx-test-reports/.tox/py35/bin/python",
          "version": "3.5.2 (default, Nov 12 2018, 13:43:14) \n[GCC 5.4.0 20160609]",
          "version_info": [
              3,
              5,
              2,
              "final",
              0
          ]
       }
    }
}

my_documents.rst

.. test-env:: ../tests/data/tox-report-short.json
   :data: name, host, installed_packages, path
   :env: py27

Output

py27

Variable Data
name py35
host user_abc
path /home/user_abc/projects/sphinx-test-reports
installed_packages
[
    "arrow==0.10.0", 
    "atomicwrites==1.2.1"
]

Motivation

Sphinx-Test-Reports was created for an automotive project, which needs to document test results and their used environment configuration in an human-readable format. The goal is to provide enough information to be able to setup an identical test environment in 20+ years.

Sphinx-Test-Reports is part of a software bundle, which was designed to fulfill the parameters of the ISO 26262 standard for safety critical software in automotive companies.

Other tools are: sphinx-needs and tox-envreport