Run tests in a forked process with a set of pipes to communicate
keep-alives and overall result. This allows us to detect when e.g.
vpp dies mid-API call causing the test to hang waiting for response
(which will never come since vpp died).
Support setting a (per test case) TIMEOUT make test option to set timeout,
with a default timeout of 120 seconds.
Example - fail the test suite if any test-case fails to finish within
300s:
make TIMEOUT=300 test
Change-Id: I0d04f26a7232968f4bf043adf5d5b508f5018717
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This change introduces a wrapper script which kills all processes in
the same process group as itself (with the exception of the script).
Using this script to run the unit tests should prevent stale processes
left behind in some cases (e.g. when test framework crashes).
Change-Id: If3b9201c06b87fa6be095721436893207d09b5e4
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Introduce faulthandler to print stack trace to stderr on python crash.
Don't disable automatic garbage collection to decrease the chances of
running out of memory.
Change-Id: I6927a5f6ea9569735d084d4ed3d258950a400d74
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Allow using custom python interpreter via 'PYTHON' env variable.
E.g. env PYTHON=pypy make test.
Get latest pip automatically to properly function in environments,
where old pip is available and a proxy is required. This allows
testing pypy in a centos vagrant-provided vm.
Change-Id: I67a658fc927303468cc67f0ac192317ca2907625
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Signed-off-by: Damjan Marion <damarion@cisco.com>
Add code which checks if vpp doesn't crash/exit immediately after
startup to aid debugging stuff like mistyped graph node name or so.
Refuse to run tests if the vpp is unable to start, complain loudly
and print vpp's stderr at critical log level if this happens to make
spotting these problems in jenkins easy.
Change-Id: I40d3fbd05c822c0534713bae6bef05ecfb0e0c1d
Signed-off-by: Klement Sekera <ksekera@cisco.com>
env EXTERN_TESTS="/path/to/extra/tests" make test
causes to run the default test set and tests collected from
test_*.py files under subtree specified in EXTERN_TESTS.
Change-Id: I58c5471dd6010730278a5b47d4318737d920bc28
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Check if vpp_papi is importable before running the tests to avoid
confusing python crashes.
Change-Id: I6adf406e353bf381d590f2ef988a1ea79b95cf37
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This starts a bash with the same environment as the test framework
uses, allowing easy debugging.
Change-Id: I956deda913b73dae5b1e1976417834ae4731f88a
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Implement plumbing to allow decorating tests as extended, e.g.:
@unittest.skipUnless(running_extended_tests(), "part of extended tests")
both methods and classes can be decorated this way.
Change make test and make test-debug to run only non-extended tests.
Introduce make test-all and make test-all-debug to run the full suite.
Run full suite as part of make verify.
Decorate most BFD tests as extended.
Change-Id: I3bc64f59e9fe238f7f767d7e043dc165d03e9dfa
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Unless overridden by COREDUMP_SIZE env variable, tell VPP to set
coredump size to unlimited, otherwise use $COREDUMP_SIZE as
the argument.
Change-Id: Ia2a6508207c66a171b33d272c820b1deb4a83e82
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Check if there are existing vpp processes before running the test suite
and refuse to run if there are. This prevents the removal of other test
suite temporary files and also makes sure that if the machine is loaded
by (zombie) vpp processes, interactive tests (like bfd) won't fail.
Change-Id: I88a74098188cb3f51966de5db19d7f80f39e51e2
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Disable automatic garbage collection and run it manually before
running each test case to minimize stalls. Improve vpp subprocess
cleanup. Reduce helper thread count to one and properly clean that
thread once it's not needed.
Change-Id: I3ea78ed9628552b5ef3ff29cc7bcf2d3fc42f2c3
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Implement fine-grained test filtering by supporting more complicated
filters beside the original file name suffix filter.
Change-Id: If5a166d08cffe8c58cc6cf174e6df861c34dbaa6
Signed-off-by: Klement Sekera <ksekera@cisco.com>
