Reduce the incidence of:
20:04:23,606 unexpected time.sleep() result - slept for 2.187967e-03s instead of ~6.837845e-04s!
Change-Id: Ic576fda7f75e571c9019111588a6a936ee2cf5c2
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
All callers of payload_to_info were required to wrap payload with str().
Refactor to call scapy's payload.load for raw payloads or specify the
specific fieldname.
Change-Id: I1c80599d4df8dc129dbb8274733afaad406d5bcf
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
.format() doesn't understand what to do with binary values by default.
Specify that we expect coersion to string.
Change-Id: Ic4b697f70852124dc85ab231f76b7934d0d71f4d
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
This cleans up exception catching to identify oserrors. By raising the
specific exception closer to the offending call, we get additional stack history
and can add clearer error logging to assist in troubleshooting.
Change-Id: I592e4d46844b822a816485bf54910f8daed92088
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Create a common pg_send method for cases when results are just discarded.
Change-Id: I786960d2d7bbb96dcb407f6e59aa96951b7b19e7
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Subclasses of (unittest.TestResult|unittest.TextTestResult) must implement an empty constructor.
The standard python library unittests depend on the empty constructor.
Change-Id: I6d1bd2e7d6996d85a9da66cb7d40af0ed3c30e2b
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
It came to my attention that Ole added a simple test in:
https://gerrit.fd.io/r/#/c/16381/ and the framework forced him
to launch an instance of VPP to test the formatting of a mac address.
This change allows the test framework to run standard unittest.TestCases
without the need to spawn a VPP instance.
Change-Id: I56651ab27c4c6bf920081a526f168a743d643201
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
This reverts commit 919efad2671993d4c6d5a0dba8eeb99d5c60edf1.
Change-Id: I5ec4fa08a03f508dac684dc1ddbd7e6b85c49b97
Signed-off-by: Ole Troan <ot@cisco.com>
* This allows for VppTestCase to instantiated with an empty constructor.
Change-Id: Ib71d3bfb4529e113f9b73c39de9b95f12ee9cfb4
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
'raw_input' does not exist in python3.
Change-Id: I95d916db2834fab83e5e791f35103a4513e610b8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
The runnning environment is static as of module load time,
so only evalute the conditions once at module load time.
Track-by: VPP-1518
Change-Id: I73b0d17ae1ff90789e70307f168d43921829aec8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
The new string type is modelled after string in proto3.
It is always variable length.
Change-Id: I64884067e28a80072c8dac31b7c7c82d6e306051
Signed-off-by: Ole Troan <ot@cisco.com>
Signed-off-by: Michal Cmarada <mcmarada@cisco.com>
Signed-off-by: Ole Troan <ot@cisco.com>
* When the framework logs an unexpected sleep time,
display the units in the same format. Typically,
it has been returning a float and an 'e'.
ex. unexpected time.sleep() result - slept for 0.000107049942017s instead of ~4.88758087158e-05s!
* Defer logger interpolation.
Change-Id: I543ad6d41c7f263d61615341437f80973c0017de
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Add 2/3 support to binarytomac and mactobinary and move to vpp_mac.py
Change-Id: I3dc7e4a24486aee22140c781aae7e44e58935877
Signed-off-by: Ole Troan <ot@cisco.com>
There are a few tests that fail on ARM and thus we can't enable testing
in CI. Skip the failing tests until they're fixed so that we catch new
failures in CI (when we enable testing for the remaining tests).
Change-Id: Ie896ef5c449ef965029633e38d317a8d5ac26352
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
instead of using build-root, use /test/venv directory for virtualenv
similarly, don't pollute build-root with test-built binaries
Change-Id: I1e63c04037eaee718b27b34ef16c9eb0252afa53
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Split one big suite into smaller suites when not running tests in
parallel. This results in all tests being executed in one iteration.
Change-Id: I0d3d357a95d9cc596b606d5911a5819e8ffdeee5
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
This reverts commit 72f0004ac5f6e201cbe042593d76df6f2491d743.
Reason for revert: Traceback (most recent call last):
File "/home/ksekera/vpp/test/test_ipsec_ah.py", line 36, in
setUpClass
super(TemplateIpsecAh, cls).setUpClass()
File "/home/ksekera/vpp/test/template_ipsec.py", line 105, in
setUpClass
super(TemplateIpsec, cls).setUpClass()
File "/home/ksekera/vpp/test/framework.py", line 459, in setUpClass
cls.quit()
File "/home/ksekera/vpp/test/framework.py", line 475, in quit
six.input("When done debugging, press ENTER to kill the "
AttributeError: 'module' object has no attribute 'input'
Change-Id: Idf0bbfea231730b37bae5dcb4557a0f82ab1b810
Signed-off-by: Ole Troan <ot@cisco.com>
Raw input does not exist in python3 use six.input.
Change-Id: Ie461696ff869057bcc3969a7571602dde570f5b8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
When a command fails, CalledProcessError is raised. testing with except: masks other failures.
Change-Id: I7e3a6739411cb6a4c13e96dd123aff9159213fea
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
The pipe used for sending keepalive messages was being added to an
instance of KeepAliveReporter and then used by the class itself. This
worked in the past but doesn't anymore. Fix the issue by adding the pipe
to the class instead of an instance.
Change-Id: If7cdca2de23ca78448e80569b155e9e29e81ff94
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
When running tests with child processes the logs from child processes
would propagate to root logger, potentially resulting in some logs
being emitted twice. Fix this by disabling log propagation to parent
loggers in child processes.
Change-Id: I31eb265c2b7f7bceff627043956a67d6def3da2b
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
This reverts commit c32023b9fd6970ed1cac1b3c7f36233b536d9968.
Change-Id: Ic934d223b10028093b0262e28515bde3ae1ccb71
Signed-off-by: Ole Troan <ot@cisco.com>
Instance method breaking encapsulation by calling a global fn.
Change-Id: Ifde2a207951143764aed75f1b191aed0bac83704
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Traceback if .send_keep_alive is called before pipe is configured.
Traceback (most recent call last):
File "/vpp/test/test_span.py", line 27, in setUpClass
super(TestSpan, cls).setUpClass()
File "/vpp/test/framework.py", line 411, in setUpClass
cls.reporter.send_keep_alive(cls, 'setUpClass')
File "/vpp/test/framework.py", line 172, in send_keep_alive
if self.pipe is None:
File "/vpp/test/framework.py", line 160, in pipe
return self._pipe
AttributeError: 'KeepAliveReporter' object has no attribute '_pipe'
Change-Id: I561d2748441702478a84fbb4580a4d2667d70ffd
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Loosen polling loop while waiting for stats socket.
==============================================================================
Classifier IP6 UDP proto Test Case
==============================================================================
connect: No such file or directory
==============================================================================
ERROR: setUpClass (test_ip4.TestIPLoadBalance)
------------------------------------------------------------------------------
Traceback (most recent call last):
File "/vpp/test/framework.py", line 424, in setUpClass
cls.statistics = VPPStats(socketname=cls.stats_sock)
File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_stats.py", line 120, in _init_
raise IOError()
IOError
Change-Id: I80523122f96eafc41b3ebbf27bee73f6c637d781
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
When running tests in one process, only one logger was used and each
testcase added its own file handler, which resulted in logs appearing in
multiple files. Fix this by restoring the creation of new loggers
for each testcase and only reuse the stream handler from parent process.
Change-Id: I5b8471e041dc769128fddb433d33812bfcb5ecf6
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
- Fixed a bug when an error occuring in tearDownClass would not result in
test being marked as failed
- Improved test results reporting in cases when an error occurs in setUpClass
and tearDownClass
- Fixed DEBUG=core when the core is produced in setUpClass or
tearDownClass
- Reworked DEBUG=core to always be handled after all tests have been
executed
- Fixed FAILFAST=1 for parallel test runs
Change-Id: I3e9cd3b97ba6fa802fa0aa2dd7678ff82eee09ec
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
Split the stat_segment.c code from stats.c.
Rename stats.[ch] to prepare for removing (19.01?)
In addition stats.api can be removed.
Since the stats aggregation for the stat segment does not use the API, that
part is now done on the main thread. (Old stats aggregator is also left in place).
Change-Id: I9867429f4fc547b1a7ab7f88bc4f3625428d681b
Signed-off-by: Ole Troan <ot@cisco.com>
The previous version sent the whole VPPTestResult object through pipe,
which uses pickle to transfer objects. Pickle does not support sending
any arbitrary objects and was causing issues. Now just a tuple of
(test_id, result) is sent.
Change-Id: I3a3a9e6f1b9ac9b05889babfc1f7560c7ac4471c
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
There was an issue when tests crashed and weren't properly retried.
Change-Id: Id5ef828ecc9a8dc0f08c50183721db06e162e6c3
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
from vpp_papi.vpp_stats import VPPStats
s = VPPStats(socketname='/var/run/stats.sock')
c = s.ls('/if/rx')
counters = s.dump(c)
print(s.set_error_str())
Change-Id: I203ebe60b0c9ee5742aadc737c0f29051757959d
Signed-off-by: Ole Troan <ot@cisco.com>
When fixing the test summary after a test run doesn't finish properly I
introduced a bug where child process which died would leave the whole
run hanging. This patch fixed the bug while still having the correct
test summary.
Change-Id: I206b1a7dab4032d24cbc50667b8dd0bdcebb67a6
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
Logging was not being logged to log.txt
Crashed tests were not reported properly when running tests in one
process
Change-Id: Ica2d703d88351cc8a94bad86764ae4376dc79590
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
The implementation of parallel test execution in VPP Test Framework.
- VPPTestCase test methods are grouped together
- tests are running in separate processes
- VPP instances spawned by tests are assigned to different cores
- output from these processes is redirected through pipes and printed
out testcase by testcase
- TEST_JOBS env var is used to specify the number of parallel processes
- improved test summary
- a bit of code cleanup
Change-Id: I9ca93904d9fe2c3daf980500c64a8611838ae28c
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
