tests/run-tests.py: Add support for tests to use unittest.

All the existing tests require a .exp file (either manually specified or
generated running the test first under CPython) that is used to check the
output of running the test under MicroPython.  The test passes if the
output matches the expected output exactly.

This has worked very well for a long time now.  But some of the newer
hardware tests (eg UART, SPI, PWM) don't really fit this model, for the
following main reasons:

- Some but not all parts of the test should be skipped on certain hardware
  targets.  With the expected-output approach, skipping tests is either all
  or nothing.

- It's often useful to output diagnostics as part of the test, which should
  not affect the result of the test (eg the diagnostics change from run to
  run, like timing values, or from target to target).

- Sometimes a test will do a complex check and then print False/True if it
  passed or not, which obscures the actual test result.

To improve upon this, this commit adds support to `run-tests.py` for a test
to use `unittest`.  It detects this by looking at the end of the output
after running the test, looking for the test summary printed by `unittest`
(or an error message saying `unittest` was not found).  If the test uses
`unittest` then it should not have a .exp file, and it's not run under
CPython.  A `unittest` based test passes or fails based on the summary
printed by `unittest`.

Note that (as long as `unittest` is installed on the target) the tests are
still fully independent and you can still run them without `run-tests.py`:
you just run it as usual, eg `mpremote run <test.py>`.  This is very useful
when creating and debugging tests.

Note also that the standard test suite testing Python semantics (eg
everything in `tests/basics/`) will probably never use unittest.  Only more
advanced tests will, and ones that are not runnable under CPython.

Signed-off-by: Damien George <damien@micropython.org>

0 files changed, 0 insertions, 0 deletions