Some testclasses (e.g. IPSec) have methods with the same names,
this causes a race when running in parallel to save the API and
move it into the test's directory - the name of the file has
only the test method name as a discriminator.
Saving two traces from two VPP instances "succeeds", in that
it silently overwrites the file. But only one mv operation
works - the other gives an error "file not found" and fails the test.
Solution: add the PID of VPP under test to the API dump filename,
this way the filenames become guaranteed unique and the race is avoided.
Change-Id: I1bb47ed73369fea0f1bfbf907588a51e2cabfb7f
Type: test
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
A sizable number of tests call pg_start() to get the packets flowing and then
immediately expect to have the entirety of the packets gone through.
This works on powerful and unstressed hardware, but fails in beautifully random
ways under load.
This also necessitates the complicated logic of remembering the "zombie captures",
then sleeping for some time before cleaning them up....
The solution is simple: in pg_start(), start the generators, wait till they
all finish, clean up, done.
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Change-Id: I930e51b7aae39c9841d22dd905a4d13a465a672b
Type: test
Intermittently, a test would start VPP, but no testcases would execute.
This would be more probable apparent during the high load or if there
is another testcase dumping the core at that moment.
Adding the logging to the connection revealed it was the stats socket
connection erroring with error -2. Increasing the deadline
from 3 seconds to 5 minutes has eliminated this error.
Change-Id: I40bd7e642abb9e2aef0238c612e4c34781de5db2
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Type: test
Note which worker received fragment with offset zero and use this worker
to send out the reassembled packet.
Type: fix
Change-Id: I1d3cee16788db3b230682525239c0100d51dc380
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Discovered running test-debug job in CI.
- fix missing paren () around format value.
Type: test
Change-Id: Iebddd3035a435f8ad1cb1d6fa4e8e8c2d4ddaf96
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Identified in post-merge review.
Type: test
Change-Id: I46e19285479437561a43975ba9b5cb68f478736c
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Introduce a new class, that returns the truthiness of a env var.
Since an environment variable is just a string, it would normally
be true if not unset. The new class returns true when the env var is
set to a string that would be considered true.
Type: test
Depends-on: https://gerrit.fd.io/r/20484
Change-Id: I90ef010156f6fec246bde5c0e208ced1869b180f
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Reduces the vpp image virtual size by multiple gigabytes
Add a "show bihash" command which displays configured and current
virtual space in use by bihash tables.
Modify the .py test framework to call "show bihash" on test tear-down
Type: refactor
Change-Id: Ifc1b7e2c43d29bbef645f6802fa29ff8ef09940c
Signed-off-by: Dave Barach <dave@barachs.net>
No reason to waste CI cycles if we know the test will not run.
See:
17:55:11 ==============================================================================
17:55:11 VPP Object Model Test
17:55:11 ==============================================================================
17:55:11 Exception in thread Thread-4:
17:55:11 Traceback (most recent call last):
17:55:11 File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
17:55:11 self.run()
17:55:11 File "/w/workspace/vpp-beta-verify-master-ubuntu1804/test/framework.py", line 1475, in run
17:55:11 stdout=subprocess.PIPE, stderr=subprocess.PIPE)
17:55:11 File "/w/workspace/vpp-beta-verify-master-ubuntu1804/test/run/venv/local/lib/python2.7/site-packages/subprocess32.py", line 617, in __init__
17:55:11 restore_signals, start_new_session)
17:55:11 File "/w/workspace/vpp-beta-verify-master-ubuntu1804/test/run/venv/local/lib/python2.7/site-packages/subprocess32.py", line 1415, in _execute_child
17:55:11 raise child_exception_type(errno_num, err_msg)
17:55:11 OSError: [Errno 2] No such file or directory: '/w/workspace/vpp-beta-verify-master-ubuntu1804/test/build/vom_test/vom_test'
17:55:11
17:55:11 17:55:11,328 Timeout! Worker did not finish in 120s
17:55:11 run C++ VOM tests ERROR [ temp dir used by test case: /tmp/vpp-unittest-VOMTestCase-vpMcWF ]
17:55:11
Type: fix
Change-Id: I3d8252807e98a09a8abd70de8a22517151f9d786
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
'Exception: VPP is dead when setting up the test'
is more helpful with the test info.
Type: test
Change-Id: I6a262fdcf2ecb33aae17bef3d19745053a0bade8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
- Move Exception into same module as TestCase.
- Move the error reporting logic inside the error.
- Allows testing of the returncode and signal_name for tests to consume.
- Fix the signal reporting code:
VppDiedError: VPP subprocess died unexpectedly with returncode -6 [unknown].
displays as:
VppDiedError: VPP subprocess died unexpectedly with return code: -6 [SIGABRT].
Type: test
Change-Id: I8488ab318a596c9b737308829cedfb7e96e57302
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Prepend sudo to the gdb command line.
Type: test
Change-Id: I09013c3ca512e26de0c46c02f643b21064ba499c
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Catches:
----
Traceback (most recent call last):
File "/vpp/test/framework.py", line 593, in tearDown
self.logger.info(self.vapi.ppcli("api trace save %s" % api_trace))
File "/vpp/test/vpp_papi_provider.py", line 413, in ppcli
return cli + "\n" + str(self.cli(cli))
File "/vpp/test/vpp_papi_provider.py", line 402, in cli
r = self.papi.cli_inband(cmd=cli)
File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 100, in __call__
return self._func(**kwargs)
File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 414, in f
return self._call_vpp(i, msg, multipart, **kwargs)
File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 634, in _call_vpp
msg = self.transport.read()
File "/vpp/src/vpp-api/python/vpp_papi/vpp_transport_shmem.py", line 120, in read
raise VppTransportShmemIOError(rv, 'vac_read failed')
VppTransportShmemIOError: [Errno -1] vac_read failed
----
Change-Id: I767e48c4d03081eb5df6a8aa67da7e192d25e4cc
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
The current code only allowed access to the main thread error counters.
That is not so useful for a multi worker instance.
No return a vector indexed by thread of counter_t values.
Type: fix
Change-Id: Ie322c8889c0c8175e1116e71de04a2cf453b9ed7
Signed-off-by: Ole Troan <ot@cisco.com>
limit max # of fragments to 3 per packet by default
add API option to configure the limit at runtime
Change-Id: Ie4b9507bf5c6095b9a5925972b37fe0032f4f9e8
Signed-off-by: Klement Sekera <ksekera@cisco.com>
The unit and code coverage tests are boring. The rest of the patch
involves test and packet-generator infra cleanups.
Teach the "make test-xxx" family of targets to set the api test plugin
path correctly, to make "binary-api <api-message-name> <args>" debug
CLI commands work correctly in the "make test"
environment. Unfortunately involves both the top-level and test
Makefiles.
Add a minor pg cli feature, a CLI to manually set
s->sw_if_index[VLIB_TX].
Consider the case where one configures an interface with both a
device-input and an output feature. To test the output feature using
the pg, it's necessary to inject packets into the interface output
node with both b->sw_if_index[VLIB_TX] and b->sw_if_index[VLIB_RX] set
correctly. For example:
packet-generator new {
name tx
limit 15
size 128-128
interface local0 # rx: device input feature not configured on local0
tx-interface loop0 # tx: output node requires b->sw_if_index[VLIB_TX]
node loop0-output
data {
hex 0x01005e7ffffa000dead0000008000102030405060708090a0b0c0d0e0f0102030405
}
}
Fix a longstanding bug in the packet generator stream setup. Remove
kludges which set b->sw_if_index[VLIB_TX] to ~0 [in multiple places]
instead of using the stream value s->sw_if_index[VLIB_TX], and setting
THAT datum correctly.
Change-Id: I1097a18e8db73661ded6b822c1d718f7e5cf36ed
Signed-off-by: Dave Barach <dave@barachs.net>
- Wait for coredump to complete before killing VPP.
Change-Id: Ib7b96b339f796f9ef7e0eadff81d181464b8a03a
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Otherwise, gcov data vanishes without a trace.
Add a __gcov_flush() call to the unix signal handler, under #ifdef
CLIB_GCOV. Add -DCLIB_GCOV to vpp_gcov_TAG_CFLAGS.
Change-Id: I2726e671b26dfbe7fae88f46a8207bb2b5106884
Signed-off-by: Dave Barach <dave@barachs.net>
Handle the case where buffer overflows.
Then SOCK_SEQPACKET assumption that multiple API messages
are not returned by recv() is broken. Use SOCK_STREAM for
API exchanges instead.
Add support for running tests over sockets.
make test SOCKET=1
Change-Id: Ibe5fd69b1bf617de4c7ba6cce0a7c2b3f97a2821
Signed-off-by: Ole Troan <ot@cisco.com>
This change adds a consistent interface for adding test-specific show commands to
log.txt.
It also adds log markers for the execution of setUp[Class], tearDown[Class]
in the logs.
Change-Id: I7d42e396e594a59e866a7d55dac0af25548e657a
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Some API action handlers called vl_msg_ai_send_shmem()
directly. That breaks Unix domain socket API transport.
A couple (bond / vhost) also tried to send a sw_interface_event
directly, but did not send the message to all that had
registred interest. That scheme never worked correctly.
Refactored and improved the interface event code.
Change-Id: Idb90edfd8703c6ae593b36b4eeb4d3ed7da5c808
Signed-off-by: Ole Troan <ot@cisco.com>
a new dedicated BVI interface as opposed to [re]using a loopback.
benefits:
- removes ambiguity over the purpose of a loopback interface
- TX node dedicated to BVI only functions.
Change-Id: I749d6b38440d450ac5b909a28053c75ec9df946a
Signed-off-by: Neale Ranns <nranns@cisco.com>
Reduce the incidence of:
20:04:23,606 unexpected time.sleep() result - slept for 2.187967e-03s instead of ~6.837845e-04s!
Change-Id: Ic576fda7f75e571c9019111588a6a936ee2cf5c2
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
All callers of payload_to_info were required to wrap payload with str().
Refactor to call scapy's payload.load for raw payloads or specify the
specific fieldname.
Change-Id: I1c80599d4df8dc129dbb8274733afaad406d5bcf
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
.format() doesn't understand what to do with binary values by default.
Specify that we expect coersion to string.
Change-Id: Ic4b697f70852124dc85ab231f76b7934d0d71f4d
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
This cleans up exception catching to identify oserrors. By raising the
specific exception closer to the offending call, we get additional stack history
and can add clearer error logging to assist in troubleshooting.
Change-Id: I592e4d46844b822a816485bf54910f8daed92088
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Create a common pg_send method for cases when results are just discarded.
Change-Id: I786960d2d7bbb96dcb407f6e59aa96951b7b19e7
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Subclasses of (unittest.TestResult|unittest.TextTestResult) must implement an empty constructor.
The standard python library unittests depend on the empty constructor.
Change-Id: I6d1bd2e7d6996d85a9da66cb7d40af0ed3c30e2b
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
It came to my attention that Ole added a simple test in:
https://gerrit.fd.io/r/#/c/16381/ and the framework forced him
to launch an instance of VPP to test the formatting of a mac address.
This change allows the test framework to run standard unittest.TestCases
without the need to spawn a VPP instance.
Change-Id: I56651ab27c4c6bf920081a526f168a743d643201
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
* This allows for VppTestCase to instantiated with an empty constructor.
Change-Id: Ib71d3bfb4529e113f9b73c39de9b95f12ee9cfb4
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
'raw_input' does not exist in python3.
Change-Id: I95d916db2834fab83e5e791f35103a4513e610b8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
The runnning environment is static as of module load time,
so only evalute the conditions once at module load time.
Track-by: VPP-1518
Change-Id: I73b0d17ae1ff90789e70307f168d43921829aec8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
