Sphinx-Test-Reports¶
Sphinx-Test-Reports
shows test results inside Sphinx documentations.
It provides the following features:
test-file: Documents all test cases from a junit-based xml file.
test-suite: Documents a specific test-suite and its test-cases.
test-case: Documents a single test-case from a given file and suite.
test-report: Creates a report from a test file, including tables and more for analysis.
test-results: Creates a simple table of test cases inside a given file.
test-env: Documents the used test-environment. Based on tox-envreport.
Introduction¶
A single documented test-case looks like this:
Custom Message: The result is passed with an execution time of 0.009029150009155273. Now follows automatically generated output like system-out, messages and text of the test-case. System-out: # testroot: root
# builder: html
# srcdir: /tmp/add_source_parser
# outdir: /tmp/add_source_parser/_build/html
# status:
#x1B[01mRunning Sphinx v1.7+/aecb0082#x1B[39;49;00m
#x1B[01mloading pickled environment... #x1B[39;49;00mnot yet created
# warning:
|
Take a look into our pytest example to see the complete result of all Sphinx tests (1154 test cases!).
The objects created by Sphinx-Test-Reports
are based on
Sphinx-Needs.
So all features for filtering, sorting and showing data is supported.
As example, here is a shorten list of tests results from the Sphinx-pytest example:
ID |
Title |
Result |
---|---|---|
test_insert_similar |
passed |
|
test_rstdim_to_latexdim |
passed |
|
test_theme_api |
failure |
|
test_build |
passed |
|
test_build_latex_doc[lualatex-howto] |
failure |
|
test_generate |
passed |
|
test_texinfo |
skipped |
Sphinx-Needs Ecosystem¶
In the last years additional information and extensions have been created, which are based or related to Sphinx-Needs:

Sphinx-Needs.com
Webpage to present most important Sphinx-Needs functions and related extensions.
Good entrypoint to understand the benefits and to get an idea about the complete ecosystem of Sphinx-Needs.

Sphinx-Needs
Base extension, which provides all of its functionality under the MIT license for free.
Create, update, link, filter and present need objects like Requirements, Specifications, Bugs and much more.

Sphinx-Needs Enterprise
Synchronizes Sphinx-Needs data with external, company internal systems like CodeBeamer, Jira or Azure Boards.
Provides scripts to baseline data and make CI usage easier.

Sphinx-Test-Reports
Extension to import test results from xml files as need objects.
Created need objects can be filtered and e.g. linked to specification objects.
Further Sphinx extensions¶
During the work with Sphinx-Needs in bigger, company internal projects, other Sphinx extensions have been created to support the work in teams of the automotive industry:

Extension to collect or generate files from different sources and include them into the Sphinx source folder.
Sources like git repositories, jinja based files or symlinks are supported.

Provides a Bazel domain in Sphinx documentations and allows the automated import of Bazel files and their documentation.
Motivation¶
Sphinx-Test-Reports
was created for an automotive project, which needs to document test results and their used
environment configuration in an human-readable format.
The goal is to provide enough information to be able to setup an identical test environment in 20+ years.
Sphinx-Test-Reports
is part of a software bundle, which was designed to fulfill
the parameters of the ISO 26262 standard
for safety critical software in automotive companies.
Other tools are: sphinx-needs, sphinx-collections and tox-envreport.