16 Commits

Author SHA1 Message Date
Ivan Ivanets
19e2c7ac01 tests: remove decorator tag_fixme_ubuntu2204
Type: test

There is no need to use the tag_fixme_ubuntu2204 decorator.
All tests that previously used it are now stable. The neighbor test
has been moved to the extended suite because it is not related to the
type of operating system.

Change-Id: Ifa0a30cbeeeaabb2bbe4d213fa61e6fbde8789ca
Signed-off-by: Ivan Ivanets <iivanets@cisco.com>
2024-12-04 16:26:21 +00:00
Dave Wallace
85ce93160f tests: fix make test python issues on ubuntu 24.04
- add patch to scapy to use latest six module instead
  of the old scapy specific six module which fails to
  import moves sub-module on python 3.12
- fix warning for deprecated legacy editable install of vpp_papi
- skip failing testcases on ubuntu-24.04

Type: test

Change-Id: Idc1a008c6e45ba69caa50c2e245012eb05effed7
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2024-08-22 07:35:40 +00:00
Klement Sekera
ca2f2e1ec9 tests: more options for decoding pcaps
Introduce "none", "all" and "failed" options for --decode-pcaps
parameter. Keep "failed" as default to be consistent with current
behaviour. Add missing documentation to test/Makefile and passthrough to
Makefile.

Rationale: running tshark binary takes about 100-150ms and if there are
thousands of pcap files, it takes minutes to decode them. This might not
be desirable if rerunning the tests repeatedly during development.

Type: improvement
Change-Id: Ie033521d51d18b9d499b9bc40fe6eff21c94622d
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
2024-07-15 18:58:07 +00:00
Andrew Yourtchenko
9987d470a6 tests: disable failing tests on Ubuntu 22.04
Also rework the logic so the skipping of marked Ubuntu 22.04 occurs at framework level

Leave debian11 special cases as-is.

Type: fix
Change-Id: I481eb32cd1a0860935482e9f930ced409da653c9
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2024-07-03 12:43:57 +02:00
Naveen Joy
70335e8e50 tests: memif interface tests using libmemif
Type: test

Change-Id: I711dfe65ad542a45acd484f0b4e3e6ade9576f66
Signed-off-by: Naveen Joy <najoy@cisco.com>
2023-12-21 00:53:44 +00:00
Dmitry Valter
953fbd2b24 tests: fix default node variant in tests
Pass a correct form of node variant config in tests

Type: fix
Signed-off-by: d-valter@yandex-team.ru
Change-Id: I8cdc240b18a1664e57a5814d6cd644891c99f515
Fixes: 8800f732f868bf54da8adba05e38bd2477895ca5
2023-11-28 20:48:31 +00:00
Dave Wallace
87a8826d17 tests: refactor pcap file deletion to improve robustness
Type: test

Change-Id: I504c079126bd8b33c5e217a1b9086788a8c778e5
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-11-28 03:34:49 +00:00
Dave Wallace
1b86f0f477 tests: retry unlinking pcap files on os error
Type: test

Change-Id: I270798ed68f04bd3974dd39c44e85dad5fa02de0
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-11-14 17:23:49 +00:00
Dave Wallace
8800f732f8 tests: refactor asf framework code
- Make framework.py classes a subset of asfframework.py classes
- Remove all packet related code from asfframework.py
- Add test class and test case set up debug output to log
- Repatriate packet tests from asf to test directory
- Remove non-packet related code from framework.py and
  inherit them from asfframework.py classes
- Clean up unused import variables
- Re-enable BFD tests on Ubuntu 22.04 and fix
  intermittent test failures in echo_looped_back
  testcases (where # control packets verified but
  not guaranteed to be received during test)
- Re-enable Wireguard tests on Ubuntu 22.04 and fix
  intermittent test failures in handshake ratelimiting
  testcases and event testcase
- Run Wiregard testcase suites solo
- Improve debug output in log.txt
- Increase VCL/LDP post sleep timeout to allow iperf server
  to finish cleanly.
- Fix pcap history files to be sorted by suite and testcase
  and ensure order/timestamp is correct based on creation
  in the testcase.
- Decode pcap files for each suite and testcase for all
  errors or if configured via comandline option / env var
- Improve vpp corefile detection to allow complete corefile
  generation
- Disable vm vpp interfaces testcases on debian11
- Clean up failed unittest dir when retrying failed testcases
  and unify testname directory and failed linknames into
  framwork functions

Type: test

Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-11-03 05:06:43 +00:00
Klement Sekera
c5fa5391ed tests: more descriptive error message
Type: improvement
Change-Id: Icf8a5dc711e1e11ec919b515d7af2f487a1c04c2
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-08-17 17:24:56 -04:00
Dave Wallace
7b8b465269 tests docs: update python3 venv packages
- Package update performed by
  1. updating pip, pip-tools, setuptools
  2. 'make test-refresh-deps' on ubuntu 22.04
  3. fixing 'make test' and 'make docs' issues
     on ubuntu 22.04
  4. 'make test-refresh-deps' on ubuntu 20.04
- Add dependency for 'make test-refresh-deps'
  to insure python venv is set up.
- Update of python formatter, black,
  caused reformating of 41 python code
  files.

Type: make

Change-Id: I7cafdf4b5189065ac57cb6b254937f6e0897a924
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-08-16 19:53:12 +00:00
Klement Sekera
47f3527108 tests: support for expected failures
- Add support for @unittest.expectedFailure decorator.

Type: improvement

Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Change-Id: I761751cda505e962225dc680b97c1fffa96f5176
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-05-20 02:34:14 +00:00
Klement Sekera
0157885517 tests: enhance counter comparison error message
- Make error message more human readable.

Type: improvement

Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Change-Id: Iefc276b3a85ff82b927028a72bb91ed87ebd04ba
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-05-20 02:33:57 +00:00
Klement Sekera
d3e0d104ad tests: refactor extra_vpp_punt_config
Rename extra_vpp_punt_config to a more generic name extra_vpp_config to
better fit its purpose. It's fit for general use and already used that
way by quic and vcl tests anyway.

Type: refactor
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Change-Id: Ib0a5789b0dbb3a8c3cae654dea4e32ac5e56dd41
2023-05-19 21:29:39 +00:00
Dmitry Valter
71d02aa631 tests: support tmp-dir on different filesystem
Support running tests with `--tmp-dir` on a filesystem different from /tmp.
os.rename withs only within a single FS whereas shutil.move works accross
different filesystems.

Type: improvement
Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru>
Change-Id: I5371f5d75386bd2b82a75b3e6c1f2c850bc62356
2023-02-14 01:26:01 +00:00
Pratikshya Prasai
657bdf781a tests: initial asf framework refactoring for 'make test'
Type: refactor

Change-Id: I41455b759a5d302ad5c4247c13634c471e7d49a8
Signed-off-by: Pratikshya Prasai <pratikshyaprasai2112@gmail.com>
Signed-off-by: Saima Yunus <yunus.saima.234@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-11-09 21:22:33 -05:00