- Make framework.py classes a subset of asfframework.py classes
- Remove all packet related code from asfframework.py
- Add test class and test case set up debug output to log
- Repatriate packet tests from asf to test directory
- Remove non-packet related code from framework.py and
inherit them from asfframework.py classes
- Clean up unused import variables
- Re-enable BFD tests on Ubuntu 22.04 and fix
intermittent test failures in echo_looped_back
testcases (where # control packets verified but
not guaranteed to be received during test)
- Re-enable Wireguard tests on Ubuntu 22.04 and fix
intermittent test failures in handshake ratelimiting
testcases and event testcase
- Run Wiregard testcase suites solo
- Improve debug output in log.txt
- Increase VCL/LDP post sleep timeout to allow iperf server
to finish cleanly.
- Fix pcap history files to be sorted by suite and testcase
and ensure order/timestamp is correct based on creation
in the testcase.
- Decode pcap files for each suite and testcase for all
errors or if configured via comandline option / env var
- Improve vpp corefile detection to allow complete corefile
generation
- Disable vm vpp interfaces testcases on debian11
- Clean up failed unittest dir when retrying failed testcases
and unify testname directory and failed linknames into
framwork functions
Type: test
Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
When custom-packaging the VPP artifacts, it can be useful to exclude
some of the core plugins from packaging/testing, for some reasons.
A removal of a plugin(s) from the worktree needs to be tracked as
a separate change, and thus is tricky from the maintenance
point of view.
This change adds the ability to "pretend they do not exist" -
plugins which are added to the comma-separated environment
variable "VPP_EXCLUDED_PLUGINS" will not be added to the build
process and not packaged.
The tests do not have the 1:1 relationship as plugins,
so they might need to be modified separately. This change
includes some of these modifications as an example.
Type: feature
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Change-Id: Id31562d00a01ced1acbb4996a633517cbd6f09d8
Drop pycodestyle for code style checking in favor of black. Black is
much faster, stable PEP8 compliant code style checker offering also
automatic formatting. It aims to be very stable and produce smallest
diffs. It's used by many small and big projects.
Running checkstyle with black takes a few seconds with a terse output.
Thus, test-checkstyle-diff is no longer necessary.
Expand scope of checkstyle to all python files in the repo, replacing
test-checkstyle with checkstyle-python.
Also, fixstyle-python is now available for automatic style formatting.
Note: python virtualenv has been consolidated in test/Makefile,
test/requirements*.txt which will eventually be moved to a central
location. This is required to simply the automated generation of
docker executor images in the CI.
Type: improvement
Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Implement command line argument parsing instead of passing arguments via
environment variables. Add script for running tests without having to
invoke make. Deprecate running tests via make.
Type: improvement
Change-Id: I2e3054a61a2ae25d460e9be00be7d7705fbf943e
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
- Generate copyright year and version
instead of using hard-coded data
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
- Relocate plugin tests for 'make test' into
src/plugins/*/test so that plugin test cases
are co-located with the plugin source code.
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I503e6a43528e14981799b735fa65674155713f67
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Catches:
----
Traceback (most recent call last):
File "/vpp/test/framework.py", line 593, in tearDown
self.logger.info(self.vapi.ppcli("api trace save %s" % api_trace))
File "/vpp/test/vpp_papi_provider.py", line 413, in ppcli
return cli + "\n" + str(self.cli(cli))
File "/vpp/test/vpp_papi_provider.py", line 402, in cli
r = self.papi.cli_inband(cmd=cli)
File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 100, in __call__
return self._func(**kwargs)
File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 414, in f
return self._call_vpp(i, msg, multipart, **kwargs)
File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 634, in _call_vpp
msg = self.transport.read()
File "/vpp/src/vpp-api/python/vpp_papi/vpp_transport_shmem.py", line 120, in read
raise VppTransportShmemIOError(rv, 'vac_read failed')
VppTransportShmemIOError: [Errno -1] vac_read failed
----
Change-Id: I767e48c4d03081eb5df6a8aa67da7e192d25e4cc
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
This change adds a consistent interface for adding test-specific show commands to
log.txt.
It also adds log markers for the execution of setUp[Class], tearDown[Class]
in the logs.
Change-Id: I7d42e396e594a59e866a7d55dac0af25548e657a
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Continuation/Part 2 of https://gerrit.fd.io/r/#/c/17092/
Change-Id: Id0122d84eaf2c05d29e5be63a594d5e528ee7c9a
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
test/vpp_l2.py:213:26: F821 undefined name 'L2_VTR_OP'
self.itf.set_vtr(L2_VTR_OP.L2_DISABLED)
Move L2_VTR_OP enum to vpp_sub_interface.py where the VTR code is found.
Change-Id: I9eb9a3a2c679813c221ce1d0c4fa8aac6076c443
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Split this work up into pieces.
Please don't add new wrappers to vpp_papi_provider.py.
Change-Id: I0f8f2afc4cd2bba07ea70ddecea2d7319f7b2e10
Signed-off-by: Ole Troan <ot@cisco.com>
.format() doesn't understand what to do with binary values by default.
Specify that we expect coersion to string.
Change-Id: Ic4b697f70852124dc85ab231f76b7934d0d71f4d
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
They mask dependencies and confuse third party tools.
Change-Id: I24ee16bfa0d746a8e8cb99891ba8b20523048b69
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Change to binascii.hexlify() for consistent bahavior.
Change-Id: Ie430cdd1ffeb6510db4aa037546e42d85992093b
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Renamed one acl testcase class which was copy-pasted and fixed one
inheritance issue.
Renamed one bihash testcase class.
Change-Id: I70d911ee7872d8d9ddba96c7f721dd099f0152f7
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
This reverts commit d5c60b96a3fd93916fc4af5c8d6d25625c28242e.
Change-Id: I3632b9c3f76c615aee897f28f76d094e7031e689
Signed-off-by: Ole Troan <ot@cisco.com>
- allow to optionally specify the specific MACIP ACL index:
'show acl-plugin macip acl [index N]'
- after showing the MACIP ACL, show the sw_if_index of
interface(s) where it is applied.
Also, add some executions of this debug commands
to the MACIP test case for easy verification.
Change-Id: I56cf8272abc20b1b2581b60d528d27a70d186b18
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Parsing the list of applied ACLs from the debug CLI output
is not a good idea, since the debug CLI can and will change.
Fix that.
Change-Id: Ia47214863f2658a54219f59cb974b5cbc536f862
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
The classifier tables layout might (and most always will) change during the MACIP ACL modification.
Furthermore, vnet_set_input_acl_intfc() is quite a picky creature - it quietly does nothing
if there is an existing inacl applied, even if the number is different, so a simple "reapply"
does not work. So, cleanly remove inacl, then reapply when the new tables are ready.
Also, fix the testcase which was supposed to test this exact behavior.
Thanks to Jon Loeliger for spotting this issue.
Change-Id: I7e4bd8023d9de7e914448bb4466c1b0ef6940f58
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
