- Make framework.py classes a subset of asfframework.py classes
- Remove all packet related code from asfframework.py
- Add test class and test case set up debug output to log
- Repatriate packet tests from asf to test directory
- Remove non-packet related code from framework.py and
inherit them from asfframework.py classes
- Clean up unused import variables
- Re-enable BFD tests on Ubuntu 22.04 and fix
intermittent test failures in echo_looped_back
testcases (where # control packets verified but
not guaranteed to be received during test)
- Re-enable Wireguard tests on Ubuntu 22.04 and fix
intermittent test failures in handshake ratelimiting
testcases and event testcase
- Run Wiregard testcase suites solo
- Improve debug output in log.txt
- Increase VCL/LDP post sleep timeout to allow iperf server
to finish cleanly.
- Fix pcap history files to be sorted by suite and testcase
and ensure order/timestamp is correct based on creation
in the testcase.
- Decode pcap files for each suite and testcase for all
errors or if configured via comandline option / env var
- Improve vpp corefile detection to allow complete corefile
generation
- Disable vm vpp interfaces testcases on debian11
- Clean up failed unittest dir when retrying failed testcases
and unify testname directory and failed linknames into
framwork functions
Type: test
Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
- Package update performed by
1. updating pip, pip-tools, setuptools
2. 'make test-refresh-deps' on ubuntu 22.04
3. fixing 'make test' and 'make docs' issues
on ubuntu 22.04
4. 'make test-refresh-deps' on ubuntu 20.04
- Add dependency for 'make test-refresh-deps'
to insure python venv is set up.
- Update of python formatter, black,
caused reformating of 41 python code
files.
Type: make
Change-Id: I7cafdf4b5189065ac57cb6b254937f6e0897a924
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Support running tests with `--tmp-dir` on a filesystem different from /tmp.
os.rename withs only within a single FS whereas shutil.move works accross
different filesystems.
Type: improvement
Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru>
Change-Id: I5371f5d75386bd2b82a75b3e6c1f2c850bc62356
Drop pycodestyle for code style checking in favor of black. Black is
much faster, stable PEP8 compliant code style checker offering also
automatic formatting. It aims to be very stable and produce smallest
diffs. It's used by many small and big projects.
Running checkstyle with black takes a few seconds with a terse output.
Thus, test-checkstyle-diff is no longer necessary.
Expand scope of checkstyle to all python files in the repo, replacing
test-checkstyle with checkstyle-python.
Also, fixstyle-python is now available for automatic style formatting.
Note: python virtualenv has been consolidated in test/Makefile,
test/requirements*.txt which will eventually be moved to a central
location. This is required to simply the automated generation of
docker executor images in the CI.
Type: improvement
Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Raise a new UnexpectedPacketErrror, when a packet is captured
unexpectedly. This pretty-prints a terse description of said packet.
Type: improvement
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Change-Id: Ibac19fc5bbd82a150fec3c90940a37af6344fd4f
Implement command line argument parsing instead of passing arguments via
environment variables. Add script for running tests without having to
invoke make. Deprecate running tests via make.
Type: improvement
Change-Id: I2e3054a61a2ae25d460e9be00be7d7705fbf943e
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Type: feature
this allows VPP to simulate linux tun devices.
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: I3adf38b49a254804370f78edd5d275d192fd00a6
This change allows one to inject multiple streams for different workers
on the same PG interface at the same time.
Type: improvement
Change-Id: I29d80369aabada261eda466e5a5d8d3518bb8bc8
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Fix of the top 11 python issues flagged as BLOCKER.
Ticket: VPP-1856
Type: fix
Change-Id: Icf4691e62f4a69d6ee196b6d6e2ab52d961b5c76
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Rather than only using time-based method of periodically checking
whether the pcap file appeared, first check that the packet generator
has stopped. To make this change fail-safe, have a 5-minute timeout
on this activity, just in case the things go terribly wrong.
Type: test
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Change-Id: Id16b2802b2de8a4cafb5d9f0a8c9ba62ec89dc32
Add support for specifying the worker thread when adding packet stream.
Type: feature
Change-Id: I8a98b91c211e60cd53e1166f9f51365394ecacfd
Signed-off-by: Klement Sekera <ksekera@cisco.com>
In a lot of places within the unit tests pg_start() starts
the capture with an already existing capture running
for the same test.
If the pcap file already exists, then it is renamed and there
is no problem.
However, there is a potential for race if the previous
pg_start() has enabled the capture, but the check for
renaming it happened just slightly earlier than the first
packet has arrived.
Then a second call to pg_start() will hit a check that
a file exists, and will cause an error. This is especially
visible when running the tests in parallel due to increased
load.
Solution: disable the capture before enabling it.
This will flush the aready running capture and eliminate the race.
The additional delay that flushing of the pcap creates has exposed
several other race conditions:
NAT tests: Some of the NAT reassembly tests
verify that the entries were added to the reassembly data structures,
but do so by comparing the quantities of entries. With the default
timeout being 2s, some of the entries might timeout,
resulting in a bogus test failure.
Solution: Bump the timeout to 20s for the affected tests.
Punt tests: nr_packets == 3 makes test intermittently fail,
nr_packets > 3 make it reliably fail, and nr_packets = 2 works
Solution: set nr_packets == 2 for the time being
IGMP tests: the leave-group calls get a spurious packet
from the time the new groups were configured
Solution: add 1 second delay before starting to delete the groups
Type: test
Change-Id: I931182a7b2860cf670e030ee7da8038f6e87356d
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
PacketGenerator streams were named based on their sw_if_index but without
knowing that, it looked like it was paired with the interface instance value.
Reencode the name to encode the data in a clearer way.
20:48:12,859 CLI: packet-generator new pcap
/tmp/vpp-unittest-TestClassifierPBR-6DVQW2/pg0_in.pcap
source pg0 name pcap0-sw_if_index-1
Change-Id: I5b82193943fee0cf060900d73aaaaecca1247105
Type: style
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Pull common code into a single method.
Type: refactor
Change-Id: Ic540d23eebbd17f838ed7a1a9dee80815a27847c
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Reduce the incidence of:
20:04:23,606 unexpected time.sleep() result - slept for 2.187967e-03s instead of ~6.837845e-04s!
Change-Id: Ic576fda7f75e571c9019111588a6a936ee2cf5c2
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
This reverts commit a98346f664aae148d26a8e158008b773d73db96f.
Change-Id: Iee5b3a5ddff0e8fd3a30fe5973cee24de434fe12
Signed-off-by: Ole Troan <ot@cisco.com>
- add/delete IPv6 VRF instances and verify results by parsing output
of ip6_fib_dump API command and by traffic
- small changes in assert_nothing_captured and get_capture to get logged
unexpected packets
Change-Id: I32207447be2df942e335aa9890ff52fb88e46597
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Disable automatic garbage collection and run it manually before
running each test case to minimize stalls. Improve vpp subprocess
cleanup. Reduce helper thread count to one and properly clean that
thread once it's not needed.
Change-Id: I3ea78ed9628552b5ef3ff29cc7bcf2d3fc42f2c3
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Under stress, it's possible to hit a race condition, when the packet
header is fully written to pcap, but not all packet data - yet.
Scapy is stupid enough to:
1. not detect and report this error, truncating the packet instead
2. continue munching more data from wrong offset
The work around is to scan the file ahead, parse the packet header,
figure out how much data we need, wait for the file to be big
enough, then restore the file position back to where it was
and finally let scapy parse the packet.
Change-Id: I9fc71d3ebdc62ecab6c90b90f177d0eaeb09b8bb
Signed-off-by: Klement Sekera <ksekera@cisco.com>
- add/delete IPv4 VRF instances and verify results by parsing output
of ip_fib_dump API command and by traffic
Change-Id: I61ed5013adca29afd00b942f65be7bf964f38d85
Signed-off-by: Jan Gelety <jgelety@cisco.com>
includes Fix for VPP-584 with API change to remove prefix length from LL programming
Change-Id: If860751c35e60255fb977f73bc33e8c2649e728e
Signed-off-by: Neale Ranns <nranns@cisco.com>
Perform accounting of expected packets based on created packet infos.
Use this accounting info to automatically expect (and verify) the
correct number of packets to be captured. Automatically retry the read
of the capture file if scapy raises an exception while doing so to
handle rare cases when capture file is read while only partially
written during busy wait. Don't fail assert_nothing_captured if only
junk packets arrived.
Change-Id: I16ec2e9410ef510d313ec16b7e13c57d0b2a63f5
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Most of the test cases are not interested in IPv6 Router Alerts,
so change the default behaviour of get_capture to filter out
these packets with the possibility of turning the filtering off,
for test cases which are interested in the RAs.
Change-Id: I0b5ee685f82c49cd32c6d6a4638eb3493d2988fc
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Introduce an API which asserts empty capture for interface.
Throw exception in old API if the capture does not exist, thus
making it clear if the test expects packets to arrive or not.
Improve performance by not doing sleeps after starting the packet
generator, rather lazily deleting captures when needed.
Fix wrong usage of packet.show() in various tests.
Change-Id: I456cb23316eef99b3f35f80344fe595c4db9a21c
Signed-off-by: Klement Sekera <ksekera@cisco.com>
This is a work-in-progress basic BFD session handling. Only
asynchronous mode is supported at the moment. Setting the session flags
doesn't work.
Change-Id: Idba27f721b5c35be5a66a6d202a63d23ff7ecf6f
Signed-off-by: Klement Sekera <ksekera@cisco.com>
