test-report¶
test-report
generate a complete report including test-cases, tables and statistics for a given test-file.
.. test-report:: My Report
:id: REPORT
:file: ../tests/doc_test/utils/pytest_sphinx_data_short.xml
The following options must be set:
id: An unique id. Will be used as prefix for all created objects.
file: File path to the test file.
Optional options are:
tags: comma separated list of tags. Will be set to all created test objects.
links: comma separated list of links. Will be set to all created test objects.
Example¶
Used code:
.. test-report:: My Report
:id: REPORT
:file: ../tests/doc_test/utils/pytest_sphinx_data_short.xml
:tags: my_report, awesome
:links: SPEC_001
.. spec:: Example specification
:id: SPEC_001
Used as simple link target. See Sphinx-Needs for details.
Result:
Test file: /home/docs/checkouts/readthedocs.org/user_builds/sphinx-test-reports/checkouts/stable/docs/../tests/doc_test/utils/pytest_sphinx_data_short.xml
Template used: /home/docs/checkouts/readthedocs.org/user_builds/sphinx-test-reports/envs/stable/lib/python3.7/site-packages/sphinxcontrib/test_reports/directives/test_report_template.txt
Statistics
Test cases:
ID |
Title |
Result |
---|---|---|
test_insert_similar |
passed |
|
test_rstdim_to_latexdim |
passed |
|
test_theme_api |
failure |
|
test_build |
passed |
|
test_build_latex_doc[lualatex-howto] |
failure |
|
test_generate |
passed |
|
test_texinfo |
skipped |
Failed test cases:
ID |
Title |
Result |
---|---|---|
test_theme_api |
failure |
|
test_build_latex_doc[lualatex-howto] |
failure |
Skipped test cases:
ID |
Title |
Result |
---|---|---|
test_texinfo |
skipped |
Imported data
[‘This file contains a subset of executed tests for Sphinx.’] |
System-out: #x1B[01mRunning Sphinx v1.7+/aecb0082#x1B[39;49;00m
#x1B[01mloading pickled environment... #x1B[39;49;00mnot yet created
#x1B[01mbuilding [mo]: #x1B[39;49;00mtargets for 0 po files that are out of date
#x1B[01mbuilding [websupport]#x1B[39;49;00m: targets for 1 source files that are out of date
#x1B[01mupdating environment: #x1B[39;49;00m1 added, 0 changed, 0 removed
|
Text: app = <SphinxTestApp buildername='html'>, status = <_io.StringIO
object at 0x7ff4b6c574c8>, warning = <_io.StringIO object at 0x7ff4b6c57798>
@pytest.mark.sphinx(
confoverrides={'html_theme': 'ziptheme',
'html_theme_options.testopt': 'foo'})
def test_theme_api(app, status, warning):
E Extra items in the left set:
E 'alabaster'
E Use -v to get the full diff
tests/test_theming.py:26: AssertionError
System-out: # testroot: root
# builder: html
# srcdir: /tmp/root
# outdir: /tmp/root/_build/html
# status:
# warning:
|
Text: app = <SphinxTestApp buildername='latex'>, status = <_io.StringIO object at
0x7ff4c662a438>, warning = <_io.StringIO object at 0x7ff4c662a678>
engine = 'lualatex', docclass = 'howto'
@skip_if_requested
@skip_if_stylefiles_notfound
print(stdout)
print(stderr)
> assert False, '%s exited with return code %s' % (
app.config.latex_engine, p.returncode)
E AssertionError: lualatex exited with return code 1
E assert False
tests/test_build_latex.py:86: AssertionError
System-out: b"This is LuaTeX, Version 1.0.4 (TeX Live 2017/Debian) \n restricted system commands
enabled.\n(./lualatex/SphinxTests.tex\nLaTeX2e <2017-04-15>\n[\\directlua]:1: module 'luaotfload-main'
not found:\n\tno field package.preload['luaotfload-main']\n\t[kpse lua searcher] file not found:
#x1B[31m/tmp/root/contents.txt:10: WARNING: extensions is already assigned section numbers (nested numbered
toctree?)#x1B[39;49;00m
#x1B[31m/tmp/root/robots.txt: WARNING: document isn't included in any toctree#x1B[39;49;00m
|
Text: /home/daniel/workspace/sphinx/sphinx/tests/test_build_texinfo.py:61: <py._xmlgen.raw object at
0x7ff4c6fc0978>
Message: <Skipped instance>
System-out: # testroot: root
# builder: texinfo
# srcdir: /tmp/root
# outdir: /tmp/root/_build/texinfo
# status:
#x1B[01mRunning Sphinx v1.7+/aecb0082#x1B[39;49;00m
#x1B[01mloading pickled environment... #x1B[39;49;00mdone
|
Template¶
The following rst-code is used to generate the report.
**Test file**: {file}
**Template used**: {template_path}
**Statistics**
| Test suites: :need_count:`'{id}' in tags and type=='{suite_need}'`
| Test cases: :need_count:`'{id}' in tags and type=='{case_need}'`
| Failed test cases: :need_count:`'{id}' in tags and 'failure' == result and type=='{case_need}'`
| Skipped test cases: :need_count:`'{id}' in tags and 'skipped' == result and type=='{case_need}'`
**Test cases**:
.. needtable::
:filter: '{id}' in tags and type == '{case_need}'
:columns: id, title, result
:style_row: tr_[[copy('result')]]
**Failed test cases**:
.. needtable::
:filter: '{id}' in tags and 'failure' == result
:columns: id, title, result
:style_row: tr_[[copy('result')]]
**Skipped test cases**:
.. needtable::
:filter: '{id}' in tags and 'skipped' == result
:columns: id, title, result
:style_row: tr_[[copy('result')]]
**Imported data**
.. {file_type}:: {title}
:id: {id}{links_string}
:tags: {tags}
:file: {file}
:auto_suites:
:auto_cases:
{content}
Template
--------
The following rst-code is used to generate the report.
.. literalinclude:: ../../sphinxcontrib/test_reports/directives/test_report_template.txt
:language: rst
Used as simple link target. See Sphinx-Needs for details. |