Instead of using separate regular expressions to capture the multiple
forms pytest test summaries might take, use one generic regular
expression to match the summary line, and then process comma-separated
components of the line with more specific matches.
Three test cases needed to be updated to accommodate this refactor:
- Two tests omitted the leading '=' characters from the line to match,
even though their summary lines clearly originated from pytest. Fix by
adding some leading '=' characters for those summary lines.
- One test mistakenly counted an "xpass" as an "xfail". Autospec maps
"xpass" results to "pass", so update the test accordingly.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
More modern `pytest` versions appear to format the time value
differently... So, for the regexps that no longer match, support the
new format.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Support the output of `meson test` in addition to `ninja test`, since
the meson docs recommend running `meson test`.
Also add a new unit test to cover some of the test result summary line
differences.
Fixes#323
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The following pattern match expression:
match = re.search(r"(---\s+)?FAIL:?\s*", line)
would match both XFAIL and FAIL.
In the case of XFAIL in the line, the match would be interpreted as FAIL.
Remedy: use this instead:
match = re.search(r"(---\s+)?(?<!X)FAIL:?\s*", line).
Kudos to phmccarty.
Update autospec to be largely flake8 clean with 2 exceptions added to
the Makefile. One for Module docs that autospec is intentionally not
including as the file splits aren't real modules at this point. And
the second for import groups needing newline separation that autospec
is ignoring to avoid splitting up each autospec "module" being
imported as autospec today isn't really built on modules the python
way.
This patch takes a different approach, since it was likely that
changing the way we count the existing individual patterns could break
other logs.
So when it identifies this is a meson test, it forks off to just parse
the meson patterns.
It doesn't support the "CLR-XTEST:" prefix to group multiple tests,
mostly because I couldn't find examples of it to test.
Convert the large count.pl script to a python script. This was mainly
done for ease of testing and calling within other python modules, such
as autospec/test.py.
The python script is nowhere near perfect. Most of the conversion was
done using find and replace. However, the python script has been tested
and behaves in the same way the perl script does. The only change (other
then several regex bugs found) should be the way it is called.
python3 count.py path/to/logtoscan
and within a python module:
import count
count.parse_log('path/to/logtoscan')
William Douglas