License Supported versions https://readthedocs.org/projects/sphinx-test-reports/badge/?version=latest Travis-CI Build Status PyPI Package latest release

Sphinx-Test-Reports

Sphinx-Test-Reports shows test results inside Sphinx documentations.

It provides the following features:

  • test-file: Documents all test cases from a junit-based xml file.

  • test-suite: Documents a specific test-suite and its test-cases.

  • test-case: Documents a single test-case from a given file and suite.

  • test-report: Creates a report from a test file, including tables and more for analysis.

  • test-results: Creates a simple table of test cases inside a given file.

  • test-env: Documents the used test-environment. Based on tox-envreport.

Introduction

A single documented test-case looks like this:

Test-Case: test_add_source_parser TC_F7984
style: tr_passed
file: ../tests/data/pytest_sphinx_data.xml
suite: pytest
case: test_add_source_parser
case_name: test_add_source_parser
classname: None
time: 0.009029150009155273
result: passed

Custom Message: The result is passed with an execution time of 0.009029150009155273.

Now follows automatically generated output like system-out, messages and text of the test-case.

System-out:

# testroot: root
# builder: html
# srcdir: /tmp/add_source_parser
# outdir: /tmp/add_source_parser/_build/html
# status:
#x1B[01mRunning Sphinx v1.7+/aecb0082#x1B[39;49;00m
#x1B[01mloading pickled environment... #x1B[39;49;00mnot yet created

# warning:

Take a look into our pytest example to see the complete result of all Sphinx tests (1154 test cases!).

The objects created by Sphinx-Test-Reports are based on Sphinx-Needs. So all features for filtering, sorting and showing data is supported.

As example, here is a shorten list of tests results from the Sphinx-pytest example:

ID

Title

Result

REPORT_0EA_049F6

test_insert_similar

passed

REPORT_0EA_734D6

test_build

passed

REPORT_0EA_140FB

test_rstdim_to_latexdim

passed

REPORT_0EA_8F625

test_generate

passed

REPORT_0EA_57BA7

test_theme_api

failure

REPORT_0EA_73FAE

test_build_latex_doc[lualatex-howto]

failure

REPORT_0EA_EB670

test_texinfo

skipped

Motivation

Sphinx-Test-Reports was created for an automotive project, which needs to document test results and their used environment configuration in an human-readable format. The goal is to provide enough information to be able to setup an identical test environment in 20+ years.

Sphinx-Test-Reports is part of a software bundle, which was designed to fulfill the parameters of the ISO 26262 standard for safety critical software in automotive companies.

Other tools are: sphinx-needs and tox-envreport