When fixing the test summary after a test run doesn't finish properly I
introduced a bug where child process which died would leave the whole
run hanging. This patch fixed the bug while still having the correct
test summary.
Change-Id: I206b1a7dab4032d24cbc50667b8dd0bdcebb67a6
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
Logging was not being logged to log.txt
Crashed tests were not reported properly when running tests in one
process
Change-Id: Ica2d703d88351cc8a94bad86764ae4376dc79590
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
The implementation of parallel test execution in VPP Test Framework.
- VPPTestCase test methods are grouped together
- tests are running in separate processes
- VPP instances spawned by tests are assigned to different cores
- output from these processes is redirected through pipes and printed
out testcase by testcase
- TEST_JOBS env var is used to specify the number of parallel processes
- improved test summary
- a bit of code cleanup
Change-Id: I9ca93904d9fe2c3daf980500c64a8611838ae28c
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
Move the tcp unit test to the plugin
Add a bihash unit test and a "make test" program to call it
Adjust framework.py to load the plugin, which is disabled by default
Change-Id: Ic229d386a56a9d28dbd54974f231149053ca8f93
Signed-off-by: Dave Barach <dave@barachs.net>
- Enable/Disable an interface for IGMP
- improve logging
- refactor common code
- no orphaned timers
- IGMP state changes in main thread only
- Large groups split over multiple state-change reports
- SSM range configuration API.
- more tests
Change-Id: If5674f1044e7e97274a711f47807c9ba689d7b9a
Signed-off-by: Neale Ranns <nranns@cisco.com>
This change adds handling of special case when setUpClass throws. In
this case TestResults receives a mock object called _ErrorHolder.
By parsing its description, we find test class name and use it to lookup
the test class in test suite to be able to add it to the list of failures
for re-running.
Change-Id: I656f21e38aa450fc567cdcbcf6e586967f947a64
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This reverts commit a98346f664aae148d26a8e158008b773d73db96f.
Change-Id: Iee5b3a5ddff0e8fd3a30fe5973cee24de434fe12
Signed-off-by: Ole Troan <ot@cisco.com>
This reverts commit d5c60b96a3fd93916fc4af5c8d6d25625c28242e.
Change-Id: I3632b9c3f76c615aee897f28f76d094e7031e689
Signed-off-by: Ole Troan <ot@cisco.com>
This change makes ip reassembly an interface feature, while adding
concurrency support. Due to this, punt is no longer needed to test
reassembly.
Change-Id: I467669514ec33283ce935be0f1dd08f07684f0c7
Signed-off-by: Klement Sekera <ksekera@cisco.com>
- Reduce replicated code in test cases
- Configure separate namespace secrets for thru hoststack
test case to validate namespace secret functionality.
- Pass per-instance environment variables to Worker class
init function.
Change-Id: I3cd5d4538f105cbfb09671c4d761541b40714b8f
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
New option TEST_DEBUG=1 turns on test framework debugging, which
currently consists of printing difference in allocated objects/memory
and also creates reference graphs for any unfreed VppPapiProvider/VPP
objects - these take a lot of memory and thus should be freed regularly.
Change-Id: I29db0c1341009d4b5c5df9222d14f3095883fd0f
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This fixes a constant setting of random seed forgotten from testing.
Change-Id: Ie3c4db8bb2b4b73ba33de1ffc02cb563391fd31c
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This option allows one to toggle whether captured VPP stdout/stderr
output is printed inline (each line logged immediately after being
collected). Default setting is 0/no (behaviour same as before).
This allows one to see the output in case of vpp crashing during test
run even if test framework gets stuck waiting for shared memory mutex
(which crashed vpp can no longer unlock).
Complete stdout/stderr output is always printed as a block to the
logger after test finishes.
Change-Id: If3645a9b85562dba26ed9c87daa27ea5ef34f2e5
Signed-off-by: Klement Sekera <ksekera@cisco.com>
The VOM is a C++ library for use by clients/agents of VPP for programming
state. It uses the binary APIs to do so. Various other common client side
functions are also provided. Please see om.hpp for a more detailed description.
Change-Id: Ib756bfe99817093815a9e26ccf464aa5583fc523
Signed-off-by: Neale Ranns <neale.ranns@cisco.com>
Co-authored-by: Mohsin Kazmi <sykazmi@cisco.com>
- Also change default coredump configuration from
"coredump-size unlimited" to "full-coredump"
Change-Id: Iefedc2636f2d9696b7575b34e91dd7be49f601fa
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Workaround old `check' library on centos.
Disable building/running of C++ VAPI test as centos's gcc can't compile
our C++ code (vapi.hpp) due to bug.
Change-Id: If9da9c7f1bc076f7cdfb9bd3016dfe60a08afa36
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Compress files in temporary directories of failed tests and symlink
the directories under /tmp/vpp-failed-unittests location - preparation
for jenkins archivation. Automatically cleanup the directory at start
of test run.
The compression is performed only when environment variable
COMPRESS_FAILED_TEST_LOGS is set to one of "yes", "y", "1".
This is set in verify target, but left unset by default, so when
invoking make test by hand, files won't be compressed.
Change-Id: I84c8f1c6aa79aa9c0b753357022b1f195f17a283
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Run tests in a forked process with a set of pipes to communicate
keep-alives and overall result. This allows us to detect when e.g.
vpp dies mid-API call causing the test to hang waiting for response
(which will never come since vpp died).
Support setting a (per test case) TIMEOUT make test option to set timeout,
with a default timeout of 120 seconds.
Example - fail the test suite if any test-case fails to finish within
300s:
make TIMEOUT=300 test
Change-Id: I0d04f26a7232968f4bf043adf5d5b508f5018717
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Remove the word derp and replace it with a proper, more gramatically
correct message.
Change-Id: I04fd44cc67dace1a31ca48fc8ce67b246162ba79
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Introduce faulthandler to print stack trace to stderr on python crash.
Don't disable automatic garbage collection to decrease the chances of
running out of memory.
Change-Id: I6927a5f6ea9569735d084d4ed3d258950a400d74
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This code improvement allows running in pypy (and other interpreters)
without exhausting file descriptors.
Change-Id: Icb692a0fe1343c12cbbb15af6c58753420e74330
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Add code which checks if vpp doesn't crash/exit immediately after
startup to aid debugging stuff like mistyped graph node name or so.
Refuse to run tests if the vpp is unable to start, complain loudly
and print vpp's stderr at critical log level if this happens to make
spotting these problems in jenkins easy.
Change-Id: I40d3fbd05c822c0534713bae6bef05ecfb0e0c1d
Signed-off-by: Klement Sekera <ksekera@cisco.com>
- single-hop BFD: attach a delegate to the appropriate adjacency
- multi-hop BFD [not supported yet]: attach a delegate to the FIB entry.
adjacency/fib_entry state tracks the BFD session state. when the state is down the object does not contribute forwarding hence and hence dependent objects will not use it.
For example, if a route is ECMP via two adjacencies and one of them is BFD down, then only the other is used to forward (i.e. we don't drop half the traffic).
Change-Id: I0ef53e20e73b067001a132cd0a3045408811a822
Signed-off-by: Neale Ranns <nranns@cisco.com>
vpp_lite platform is not needed anymore as same efect can be
achieved with following startup.conf config:
plugins {
plugin dpdk_plugin.so { disable }
}
Change-Id: I690ea8ceb1c6e1fe32e01e7da54e9958019a93bf
Signed-off-by: Damjan Marion <damarion@cisco.com>
Implement plumbing to allow decorating tests as extended, e.g.:
@unittest.skipUnless(running_extended_tests(), "part of extended tests")
both methods and classes can be decorated this way.
Change make test and make test-debug to run only non-extended tests.
Introduce make test-all and make test-all-debug to run the full suite.
Run full suite as part of make verify.
Decorate most BFD tests as extended.
Change-Id: I3bc64f59e9fe238f7f767d7e043dc165d03e9dfa
Signed-off-by: Klement Sekera <ksekera@cisco.com>
