Check and skip VPP_EXCLUDED_PLUGINS tests for most of plugins.
Type: improvement
Signed-off-by: Dmitry Valter <d-valter@yandex-team.com>
Change-Id: I23fd3666729251c639aa8da72a676058e3f5bb4e
Also rework the logic so the skipping of marked Ubuntu 22.04 occurs at framework level
Leave debian11 special cases as-is.
Type: fix
Change-Id: I481eb32cd1a0860935482e9f930ced409da653c9
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
When the NIC does not support mac filter, we rely on ethernet-input
node to do the destination mac check, ie, when the interface is in L3,
the mac address for the packet must be the mac address of the
interface where the packet arrives. This works fine in ethernet-input
node when all packets in the frame might have different interfaces, ie,
ETH_INPUT_FRAME_F_SINGLE_SW_IF_ID is not set in the frame. However,
when all packets are having the same interface,
ETH_INPUT_FRAME_F_SINGLE_SW_IF_ID is set, ethernet-input node goes
through the optimized routine eth_input_single_int -> eth_input_process_frame.
That is where dmac check has a bug when all packets in the frame are
either, ip4, ip6, or mpls without vlan tags. Because without vlan tags,
the code handles all packets in fast path and ignores dmac check.
With vlan tags, the code goes to slow path where dmac check is handled
properly.
The fix is to check if we have a bad dmac in the fast path and force the
code to go to slow path which will handle dmac check properly.
Also do a wholesale correction on all the testcases which do not use
the proper dmac when sending L3 packets.
Type: fix
Change-Id: I73153a805cecdc24c4eefcc781676de04737ae2c
Signed-off-by: Steven Luong <sluong@cisco.com>
IPFIX buffers are stored on a per worker thread basis. Currently, the
flush callbacks will flush only buffers stored for the main thread. And
buffers for worker threads will not be sent until their size reach the
path MTU configured for the exporter. So if traffic is constant, the
problem will unlikely to be visible. Buffers will be sent once they
reach the maximum size. However, if traffic stops at some point and
flush is triggered in order to make the plugin send all currently
buffered data, this will not happen. And collectors will not receive
that data. The plugin will keep the remaining data until traffic starts
again, the buffers reach the maximum size, and be sent.
With this fix, flush buffers for worker threads and for the main thread
when the flush callbacks are triggered.
This will allow to remove @tag_fixme_vpp_workers from the unit tests
that don't set timers. The tests that set timers will still be failing
for other multi-worker related problems.
Type: fix
Change-Id: I9a7d9cef8ddbec7ee68c79309e48e7bc0953d488
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Currently, when IPFIX records generation is enabled for an interface in
the TX direction, some rewritten traffic is being sent from that
interface, and the Ethernet header's location has changed due to
rewriting, generated TX flows will contain fields with wrong and zero
values. For example, that can be observed when traffic is rewritten from
a subinterface to a hardware interface (i.e. when tags are removed). A
TX flow generated in this case will have wrong L2 fields because of an
incorrectly located Ethernet header. And zero L3/L4 fields because the
Ethernet type will match neither IP4 nor IP6.
The same code is executed to generate flows for both input and output
features. And the same mechanism is applied to identify the Ethernet
header in the buffer's data. However, such general code usually works
with the buffer's data conditionally based on the direction. For most
input features, the buffer's current_data will likely point to the IP
header. For most output features, the buffer's current_data will likely
point to the Ethernet header.
With this fix:
- Keep relying on ethernet_buffer_get_header() to locate the Ethernet
header for input features. And start using vlib_buffer_get_current()
to locate the Ethernet header for output features. The function will
account for the Ethernet header's position change in the buffer's
data if there is rewriting.
- After fixing Ethernet header determination in the buffer's data,
L3/L4 fields will contain non-zero but still incorrect data. That is
because IP header determination needs to be fixed too. It currently
relies on the fact that the Ethernet header is always located at the
beginning of the buffer's data and that l2_hdr_sz can be used as an
IP header offset. However, this may not be the case after rewriting.
So start calculating the actual offset of the IP header in the
buffer's data.
- Add a unit test to cover the case.
Type: fix
Change-Id: Icf3f9e6518912d06dff0d5aa48e103b3dc94edb7
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
As a result of recent fixes, all currently stored flows of an interface
are deleted when the feature is being disabled for the interface. This
includes stopping the timer and freeing the flow entries for further
reuse. The problem is that meta information is not cleared in the flow
entries being deleted. For example, packet delta count will keep its
value. The next flow that gets one of these pool entries will already
have a non-zero packet count. So the counting of packets will start from
a non-zero value. And incorrect packet delta count will be exported for
that flow.
With this fix, clear meta information too when clearing interface state.
Also, update the corresponding test to cover this case.
Type: fix
Change-Id: I9a73b3958adfd1676e66b0ed50f1478920671cca
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
- Make framework.py classes a subset of asfframework.py classes
- Remove all packet related code from asfframework.py
- Add test class and test case set up debug output to log
- Repatriate packet tests from asf to test directory
- Remove non-packet related code from framework.py and
inherit them from asfframework.py classes
- Clean up unused import variables
- Re-enable BFD tests on Ubuntu 22.04 and fix
intermittent test failures in echo_looped_back
testcases (where # control packets verified but
not guaranteed to be received during test)
- Re-enable Wireguard tests on Ubuntu 22.04 and fix
intermittent test failures in handshake ratelimiting
testcases and event testcase
- Run Wiregard testcase suites solo
- Improve debug output in log.txt
- Increase VCL/LDP post sleep timeout to allow iperf server
to finish cleanly.
- Fix pcap history files to be sorted by suite and testcase
and ensure order/timestamp is correct based on creation
in the testcase.
- Decode pcap files for each suite and testcase for all
errors or if configured via comandline option / env var
- Improve vpp corefile detection to allow complete corefile
generation
- Disable vm vpp interfaces testcases on debian11
- Clean up failed unittest dir when retrying failed testcases
and unify testname directory and failed linknames into
framwork functions
Type: test
Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Currently, TCP flags of a flow entry don't get reset once the flow is
exported (unlike other meta information about a flow - packet delta
count and octet delta count). So TCP flags are accumulated as long as
the flow is active. When the flow expires, it is exported the last time,
and its pool entry is freed for further reuse. The next flow that gets
this pool entry will already have non-zero TCP flags. If it's a TCP
flow, the flags will keep being accumulated. This might look fine when
exported. If it's a non-TCP flow, that will definitely look erroneous.
With this fix, reset TCP flags once the flow is exported. Also, cover
the reuse case with tests.
Type: fix
Change-Id: I5f8560afffcfe107909117d3d063e8a69793437e
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Currently, when L2 and L4 recording is enabled on the L2 datapath, the
L2 template will contain L4 fields and L2 flows will be exported with
those fields always set to zero.
With this fix, when L4 recording is enabled, add L4 fields to templates
other than the L2 template (i.e. to the IP4, IP6, L2_IP4, and L2_IP6
templates). And export L2 flows without L4 fields. Also, cover that case
in the tests.
Type: fix
Change-Id: Id5ed8b99af5634fb9d5c6e695203344782fdac01
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
When IPFIX flow record generation is enabled on an interface and the
active timer is set, flows will be saved and then exported according to
the active and passive timers. If then disable the feature on the
interface, the flow entries currently saved will remain in the state
tables. They will gradually expire and be exported. The problem is that
the template for them has already been removed. And they will be sent
with zero template ID which will make them unreadable.
A similar problem will occur if feature settings are "changed" on the
interface - i.e. disable the feature and re-enable it with different
settings (e.g. set a different datapath). The remaining flows that
correspond to the previous feature settings will be eventually sent
either with zero template ID or with template ID that corresponds to the
current feature settings on the interface (and look like garbage data).
With this fix, flush the current buffers before template removal and
clear the remaining flows of the interface during feature disabling.
Type: fix
Change-Id: I1e57db06adfdd3a02fed1a6a89b5418f85a35e16
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Currently, L2 flows are exported using L2_IP6 template if L3 or L4
recording is enabled on L2 datapath. That occurs because during feature
enable, L2 template is added and its ID is not saved immediately. Then
L2_IP4 and L2_IP6 templates are added overwriting "template_id" each
time. And in the end, the current value of "template_id" is saved for L2
template. The problem is that "template_id" at that point contains the
ID of L2_IP6 template.
With this fix, save the template ID immediately after adding a template
for all variants (datapaths). Also, cover the case with a test.
Type: fix
Change-Id: Id27288043b3b8f0e89e77f45ae9a01fa7439e20e
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Type: improvement
With this change:
- add dump/details messages to obtain interfaces for which IPFIX flow
record generation is enabled;
- add get message to obtain parameters;
- add a new message to set parameters with validation present and to
correspond with get/set naming;
- add tests for get/set parameters and dump/details interfaces.
Change-Id: I09f6ec990171ac8bcb9d2f5c92629803b8ab6c28
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Type: feature
Currently, the plugin supports only IPFIX flow record generation for
outbound packets.
With this change:
- add a new API message for enabling the feature on an interface that
accepts direction (rx, tx, both);
- update existing debug command for feature enabling to accept
direction;
- update existing debug command for showing currently enabled feature
on interfaces to display direction;
- update templates to include a direction field;
- generate flow records on the specified direction and data path;
- report direction in flow data;
- update tests to use the new API;
- add tests for inbound flows.
Change-Id: I121fd904b38408641036ebeea848df7a4e5e0b30
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Drop pycodestyle for code style checking in favor of black. Black is
much faster, stable PEP8 compliant code style checker offering also
automatic formatting. It aims to be very stable and produce smallest
diffs. It's used by many small and big projects.
Running checkstyle with black takes a few seconds with a terse output.
Thus, test-checkstyle-diff is no longer necessary.
Expand scope of checkstyle to all python files in the repo, replacing
test-checkstyle with checkstyle-python.
Also, fixstyle-python is now available for automatic style formatting.
Note: python virtualenv has been consolidated in test/Makefile,
test/requirements*.txt which will eventually be moved to a central
location. This is required to simply the automated generation of
docker executor images in the CI.
Type: improvement
Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Implement command line argument parsing instead of passing arguments via
environment variables. Add script for running tests without having to
invoke make. Deprecate running tests via make.
Type: improvement
Change-Id: I2e3054a61a2ae25d460e9be00be7d7705fbf943e
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
- Generate copyright year and version
instead of using hard-coded data
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
- Relocate plugin tests for 'make test' into
src/plugins/*/test so that plugin test cases
are co-located with the plugin source code.
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I503e6a43528e14981799b735fa65674155713f67
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
-Migrate tests to api and cleanup:
'# FIXME this should be an API call' in tests.
Type: feature
Change-Id: I715f9f8acc0f2af9c974ea221b2aea46692a6218
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
TypeError: not all arguments converted during string formatting
Change-Id: I0b4b27e37ba47e737aeb639c9787f8bc08a92dce
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Split this work up into pieces.
Please don't add new wrappers to vpp_papi_provider.py.
Change-Id: I0f8f2afc4cd2bba07ea70ddecea2d7319f7b2e10
Signed-off-by: Ole Troan <ot@cisco.com>
The runnning environment is static as of module load time,
so only evalute the conditions once at module load time.
Track-by: VPP-1518
Change-Id: I73b0d17ae1ff90789e70307f168d43921829aec8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Change to binascii.hexlify() for consistent bahavior.
Change-Id: Ie430cdd1ffeb6510db4aa037546e42d85992093b
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
- fixed problem with tcp_flag
- changed flowtimestamp into NTP format
Change-Id: I4ef05d6c69c5c078a0c80d59c5ccb0c85b924ba6
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I67839281623721bf42f0a918a53356143d9dc78a
Signed-off-by: Ole Troan <ot@cisco.com>
Signed-off-by: Pavel Kotucek <pkotucek@cisco.com>
Signed-off-by: Ole Troan <ot@cisco.com>
