Type: test
There is no need to use the tag_fixme_ubuntu2204 decorator.
All tests that previously used it are now stable. The neighbor test
has been moved to the extended suite because it is not related to the
type of operating system.
Change-Id: Ifa0a30cbeeeaabb2bbe4d213fa61e6fbde8789ca
Signed-off-by: Ivan Ivanets <iivanets@cisco.com>
Type: test
Activate tests that works successfully on ubuntu 22.04 but were
problematic on later version and were excluded by
@tag_fixme_ubuntu2204
Change-Id: I93c4626a120ce8b475e9c5ca933a5544d6337933
Signed-off-by: Ivan Ivanets <iivanets@cisco.com>
Check and skip VPP_EXCLUDED_PLUGINS tests for most of plugins.
Type: improvement
Signed-off-by: Dmitry Valter <d-valter@yandex-team.com>
Change-Id: I23fd3666729251c639aa8da72a676058e3f5bb4e
In some cases we do not need multicast support. Making it optional helps
scaling to high number of VRFs, by reducing the control plane operations
and memory consumption.
Type: improvement
Change-Id: Ib34ed3fe2806e2f4624981da4e4a3c49c69f70be
Signed-off-by: Benoît Ganne <bganne@cisco.com>
Also rework the logic so the skipping of marked Ubuntu 22.04 occurs at framework level
Leave debian11 special cases as-is.
Type: fix
Change-Id: I481eb32cd1a0860935482e9f930ced409da653c9
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
- Make framework.py classes a subset of asfframework.py classes
- Remove all packet related code from asfframework.py
- Add test class and test case set up debug output to log
- Repatriate packet tests from asf to test directory
- Remove non-packet related code from framework.py and
inherit them from asfframework.py classes
- Clean up unused import variables
- Re-enable BFD tests on Ubuntu 22.04 and fix
intermittent test failures in echo_looped_back
testcases (where # control packets verified but
not guaranteed to be received during test)
- Re-enable Wireguard tests on Ubuntu 22.04 and fix
intermittent test failures in handshake ratelimiting
testcases and event testcase
- Run Wiregard testcase suites solo
- Improve debug output in log.txt
- Increase VCL/LDP post sleep timeout to allow iperf server
to finish cleanly.
- Fix pcap history files to be sorted by suite and testcase
and ensure order/timestamp is correct based on creation
in the testcase.
- Decode pcap files for each suite and testcase for all
errors or if configured via comandline option / env var
- Improve vpp corefile detection to allow complete corefile
generation
- Disable vm vpp interfaces testcases on debian11
- Clean up failed unittest dir when retrying failed testcases
and unify testname directory and failed linknames into
framwork functions
Type: test
Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Use client address hash to pick the first outside address
instead of just address high octet, becasue it may denegerate
into stable 10/172/192, depending on nat address count.
Fix outside address distribution test to acually test the
distribution, not the algo, so previous distribution will
fail with 65 nat addresses and 100 clients:
FAIL: Outside address distribution based on source address
Traceback (most recent call last):
File ".../test/test_nat44_ed.py", line 2048, in test_outside_address_distribution
msg="Bad outside address distribution")
AssertionError: 156.25 not less than 0.33 : Bad outside address distribution
Type: improvement
Change-Id: I604b1294422f20d211db5614c47559557a78a193
Signed-off-by: Vladislav Grishenko <themiron@yandex-team.ru>
- Package update performed by
1. updating pip, pip-tools, setuptools
2. 'make test-refresh-deps' on ubuntu 22.04
3. fixing 'make test' and 'make docs' issues
on ubuntu 22.04
4. 'make test-refresh-deps' on ubuntu 20.04
- Add dependency for 'make test-refresh-deps'
to insure python venv is set up.
- Update of python formatter, black,
caused reformating of 41 python code
files.
Type: make
Change-Id: I7cafdf4b5189065ac57cb6b254937f6e0897a924
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Nat session is allocated before the port allocation. During port allocation
candidate address+port are set to o2i 6-tuple and tested against the flow hash.
If insertion fails, the port is busy and rejected. When all N attempts are
unsuccessful, "out-of-ports" error is recorded and the session is to be
deleted.
During session deletion o2i and i2o tuples are deleted from the flow hash.
In case of "out-of-ports" i2o tuple is not valid, however o2i is and it refers
to **some other** session that's known to be allocated.
By backing match tuple up session should be invalidated well enough not to
collide with any valid one.
Type: fix
Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru>
Change-Id: Id30be6f26ecce7a5a63135fb971bb65ce318af82
After deleting a sw interface with nat44 features, the next created
sw interface will get the same sw_index reused and therefore will
erroneously have the same nat features enabled.
Type: fix
Change-Id: I1d84f842ab7ab2a757668ae1a111efe67e1e924d
Signed-off-by: Vladislav Grishenko <themiron@yandex-team.ru>
The number of available dynamic ports is set to (0xffff - 1024) =
64511, which is not divisable by the pow2 number of workers - the
only integer divisors are 31 and 2081.
So, total dynamic port range of all workers will be less than it:
1 wrk: n = (port_per_thread = 64511/1)*1 = 64511 + 1025 = 65536
2 wrk: n = (port_per_thread = 64511/2)*2 = 64510 + 1025 = 65535
4 wrk: n = (port_per_thread = 64511/4)*4 = 64508 + 1025 = 65533
8 wrk: n = (port_per_thread = 64511/8)*8 = 64504 + 1025 = 65529
...
As seen, with multiple workers there are unused trailing ports for every
nat pool address and that is the reason of out-of-bound index in the
worker array on out2in path due (port - 1024) / port_per_thread math.
This was fixed in 5c9f9968de63fa627b4a72b344df36cdc686d18a, so packets
to unused ports will go to existing worker and dropped there.
Per RFC 6335 https://www.rfc-editor.org/rfc/rfc6335#section-6:
6. Port Number Ranges
o the System Ports, also known as the Well Known Ports, from 0-1023
(assigned by IANA)
o the User Ports, also known as the Registered Ports, from 1024-
49151 (assigned by IANA)
o the Dynamic Ports, also known as the Private or Ephemeral Ports,
from 49152-65535 (never assigned)
According that let's allocate dynamic ports from 1024 and have full port
range with a wide range of the workers number - 64 integer divisors in
total, including pow2 ones:
1 wrk: n = (port_per_thread = 64512/1)*1 = 64512 + 1024 = 65536
2 wrk: n = (port_per_thread = 64512/2)*2 = 64512 + 1024 = 65536
3 wrk: n = (port_per_thread = 64512/3)*3 = 64512 + 1024 = 65536
4 wrk: n = (port_per_thread = 64512/4)*4 = 64512 + 1024 = 65536
5 wrk: n = (port_per_thread = 64512/5)*5 = 64510 + 1024 = 65534
6 wrk: n = (port_per_thread = 64512/6)*6 = 64512 + 1024 = 65536
7 wrk: n = (port_per_thread = 64512/7)*7 = 64512 + 1024 = 65536
8 wrk: n = (port_per_thread = 64512/8)*8 = 64512 + 1024 = 65536
...
Modulo from 5c9f9968de63fa627b4a72b344df36cdc686d18a is still required
when the numbers of workers is not the integer divisor of 64512.
Type: fix
Fixes: 5c9f9968de63fa627b4a72b344df36cdc686d18a
Change-Id: I9edaea07e58ff4888812b0d86cbf41a3784b189e
Signed-off-by: Vladislav Grishenko <themiron@yandex-team.ru>
Drop pycodestyle for code style checking in favor of black. Black is
much faster, stable PEP8 compliant code style checker offering also
automatic formatting. It aims to be very stable and produce smallest
diffs. It's used by many small and big projects.
Running checkstyle with black takes a few seconds with a terse output.
Thus, test-checkstyle-diff is no longer necessary.
Expand scope of checkstyle to all python files in the repo, replacing
test-checkstyle with checkstyle-python.
Also, fixstyle-python is now available for automatic style formatting.
Note: python virtualenv has been consolidated in test/Makefile,
test/requirements*.txt which will eventually be moved to a central
location. This is required to simply the automated generation of
docker executor images in the CI.
Type: improvement
Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
The RFC7857 state machine introduced in 56c492a is a trade-off.
It tries to retain sessions as much as possible and also offers
some protection against spurious RST by re-establishing sessions if data
is received after the RST. From experience in the wild, this algorithm is
a little too liberal, as it leaves too many spurious established sessions
in the session table.
E.g. a oberserved pattern is:
client server
<- FIN, ACK
ACK ->
ACK ->
RST, ACK ->
With the current state machine this would leave the session in established state.
These proposed changes do:
- require 3-way handshake to establish session.
(current requires only to see SYNs from both sides)
- RST will move session to transitory without recovery if data is sent after
- Only a single FIN is needed to move to transitory
Fixes: 56c492aa0502751de2dd9d890096a82c5f04776d
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I92e593e00b2efe48d04997642d85bd59e0eaa2ea
Signed-off-by: Ole Troan <ot@cisco.com>
Set deprecated option on unsupported API calls.
Cleaned up API calls with deprecated option. Removed
in progress option from long term used API calls.
Removed obsolete/unused nodes, functions, variables.
Fixed set frame queue nelts function. Calling API
would incorrectly not fail even though frame queue nelts
can only be set before first call nat44_plugin_enable.
Moved all formatting functions to _format.c file.
Type: refactor
Change-Id: I3ca16e0568f8d7eee3a27c3620ca36164833a7e4
Signed-off-by: Filip Varga <fivarga@cisco.com>
Add missing translation of ICMP inner IP layer.
Change responsible test so that it actually tests something.
Type: fix
Fixes: 4881cb4c6f
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Change-Id: Id3a6f12a7308d81b1cdf9815f857221fab2f24d9
Type: improvement
For error conditions, such as TTL expired, dest unreach, etc, Rate limit the sending of ICMP error messages.
The rate limiting is done based on src,dst IP address of the received packet.
the rate limit has been chosen, somewhat arbitrarily, to be 1e-3. This is the same limit as the ARP throttling.
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: I4a0b791cde8c941a9bf37de6aa5da56779d3cef4
Add support for inline packet counter verification to send_and_* functions.
Diff dictionary is a dictionary of dictionaries of interesting stats:
diff_dictionary =
{
"err" : { '/error/counter1' : 4, },
sw_if_index1 : { '/stat/segment/counter1' : 5,
'/stat/segment/counter2' : 6,
},
sw_if_index2 : { '/stat/segment/counter1' : 7,
},
}
It describes a per sw-if-index diffset, where each key is stat segment
path and value is the expected change for that counter for sw-if-index.
Special case string "err" is used for error counters.
This then allows more precise packet counter verification by first
defining a "zero" dictionary, e.g. for ED NAT:
cls.no_diff = StatsDiff({
pg.sw_if_index: {
'/nat44-ed/in2out/fastpath/tcp': 0,
'/nat44-ed/in2out/fastpath/udp': 0,
'/nat44-ed/in2out/fastpath/icmp': 0,
'/nat44-ed/in2out/fastpath/drops': 0,
'/nat44-ed/in2out/slowpath/tcp': 0,
'/nat44-ed/in2out/slowpath/udp': 0,
'/nat44-ed/in2out/slowpath/icmp': 0,
'/nat44-ed/in2out/slowpath/drops': 0,
'/nat44-ed/in2out/fastpath/tcp': 0,
'/nat44-ed/in2out/fastpath/udp': 0,
'/nat44-ed/in2out/fastpath/icmp': 0,
'/nat44-ed/in2out/fastpath/drops': 0,
'/nat44-ed/in2out/slowpath/tcp': 0,
'/nat44-ed/in2out/slowpath/udp': 0,
'/nat44-ed/in2out/slowpath/icmp': 0,
'/nat44-ed/in2out/slowpath/drops': 0,
}
for pg in cls.pg_interfaces
})
and then to specify only changed counters directly when calling
one of send_and_* functions:
self.send_and_assert_no_replies(
self.pg0, pkts, msg="i2o pkts",
stats_diff=self.no_diff | {
"err": {
'/err/nat44-ed-in2out-slowpath/out of ports': len(pkts),
},
self.pg0.sw_if_index: {
'/nat44-ed/in2out/slowpath/drops': len(pkts),
},
}
)
operator | is overloaded by StatsDiff class to perform a deep merge operation,
so in above case, dictionaries for "err" and self.pg0.sw_if_index do not
overwrite whole sub-dictionaries, rather the contents are merged,
assuring that all the remaining counters are verified to be zero.
Type: improvement
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Change-Id: I2b87f7bd58a7d4b34ee72344e2f871b2f372e2d9
Implement proper state machine based on above RFCs. ACKs to SYNs/FINs
are no longer required/tracked. This is more friendly to peers and
accounts for lost packets and retransmits.
This change also means that all traffic is translated and forwarded
while in transitory timeout, which helps delivering e.g. retransmitted
FINs, FINACKs and other messages.
Also support reopening a session in transitory timeout after seeing both
FINs by seeing both SYNs again. This helps quick connection
reestablishment if the peers want to.
Type: improvement
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Signed-off-by: Miklos Tirpak <miklos.tirpak@gmail.com>
Change-Id: Ibf521c79463472db97e593bfa02b32b4a06dfd2a
NAT pool address overlaps with pg4 network 10.0.0.1/24
this is not desirable because of upcoming change [1]. This
change uncovered configuration issue with hairpinning test
that would fail because of already configured pg4 interface
that shares subnet with nat pool address. Packets would
incorectly end up in bad pg interface causing the test to fail.
This patch changes nat pool address to 10.0.10.3 that doesn't
belong to any of the subnets configured on pg interfaces in nat
tests.
[1] https://gerrit.fd.io/r/c/vpp/+/34441
Type: test
Change-Id: I49e63dacbf0847116adbcf1954ff5defd833a657
Signed-off-by: Filip Varga <fivarga@cisco.com>
Fixed bad use of macros for autoendian API calls
and updated tests for the new API. Removed sw_if_index
check macro because of ntol conversion. Changed
REPLY_MACRO to REPLY_MACRO_END to fix ntohl conversions.
Type: fix
Change-Id: I878a07b3f80fe03179feab60f0abc662f408a2c8
Signed-off-by: Filip Varga <fivarga@cisco.com>
Put static mappings in flow hash, drop existing hash tables used for
static mappings. Drop refcount variables and use hash table as a single
point of truth. Allow creating a static mapping conflicting with dynamic
mapping, which will take precedence after dynamic mapping is freed, so
that the existing flow can finish transferring data.
Type: fix
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: Idfde8efabc09971be38921d4b0ca5ccf4e9fe412
Put static mappings in flow hash, drop existing hash tables used for
static mappings. Drop refcount variables and use hash table as a single
point of truth. Allow creating a static mapping conflicting with dynamic
mapping, which will take precedence after dynamic mapping is freed, so
that the existing flow can finish transferring data.
Type: fix
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: Ieeba691d83a83887d0a0baccd5f3832f66126096
Type: fix
With endpoint dependent NAT44, When there are multiple workers we look
for a flow which matches the packet in order to figure out which worker
should handle the packet. If the packet is an ICMP error, it may be
associated with an existing flow by inspecting the L3/L4 headers that
are included in the message payload.
This was not being done for in2out packets in
nat44_ed_get_in2out_worker_index(), so some packets which were related
to an open session were not being associated with that session and were
being passed to a different thread than the one where the session was
created. Later on, when the packet was processed by the fast path in2out
node, the L3/L4 headers in the payload are inspected and the fast path
node finds the existing session. Since that session is owned by a
different thread than the one the packet is being processed by, the
in2out fast path node can potentially access the wrong session and/or
memory adjacent to the session pool. This can cause a SEGV.
Make nat44_ed_get_in2out_worker_index() look at the inner headers when
processing an ICMP error. THis is already done in
nat44_ed_get_out2in_worker_index() and in the fast path in2out node.
Change-Id: Icdc1abebcbce452ee7be7cb23fc563e09bf575f2
Signed-off-by: Matthew Smith <mgsmith@netgate.com>
Type: fix
The NAT plugin stores packet counters and a counter represents a vector
indexed by interface index. When an interface is assigned a NAT role,
the counters are validated to be long enough for the given interface
index.
When a packet traverses NAT in2out and output feature is disabled, the
appropriate counters are updated by the RX interface index. In this
case, translation happens on the inside interface and its index was
ensured to be valid in all of the counters during NAT role assignment.
When a packet traverses NAT in2out and output feature is enabled, the
appropriate counters are updated by the RX interface index too. In this
case, translation happens on the outside interface and the packet could
be received on any interface, even with no NAT role assigned. If that's
the case and its index is greater than the greatest index validated in
the counters, a new counter value will be written to memory that does
not belong to the counter. As a result, a crash will occur at some
point.
With this change, use TX interface index to update the counters when
output feature is enabled. TX interface is an actual interface where
translation happens and its index is always valid in the counters.
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Change-Id: I53a52af949fe96419e1b5fef4134ab4062198f51
Handle case where extra data is present in buffer which is not part of
IP/ICMP headers.
Type: fix
Fixes: 05b5a5b3b4b04823776feed6403b5a99b2e06d76
Change-Id: Icfef811470056d38c60fc45cc302139ed7594385
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Verify that headers are not truncated and that checksums are valid.
Correct checksum computation in translation code.
Type: fix
Change-Id: I6acfcec4661411f83c86b15aafac90cd4538c0b5
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Refactored & fixed NAT44ED configuration
functions used for handling interfaces and
nodes.
Type: refactor
Signed-off-by: Filip Varga <fivarga@cisco.com>
Change-Id: I6fbbb7f0fe35d572675997745d53290152987424
TTL was already decremented in ip4-rewrite so it's okay if it's 1.
Type: fix
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: I587dc343737c15247eb62837a06d5e44c0d11acc
Replace VRF test testing multiple scenarios into more simpler tests to
improve readability and ease of debugging.
Type: refactor
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: Ibaad5328c73b401f52c5fe513dc0ed68ff3e3374
- Generate copyright year and version
instead of using hard-coded data
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
