Type: improvement
Ethernet frames on the wire are a minimum of 64 bytes, so use the length in the UDP header to determine if the ESP payload is one bytes of the special SPI, rather than the buffer's size (which will include the ethernet header's padding).
In the case of drop advance the packet back to the IP header so the ipx-drop node sees a sane packet.
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: Ic3b75487919f0c77507d6f725bd11202bc5afee8
- cleanup data structures, functions and cli
- remove option to return static html. For similar results, use
http static server
- rename to http_cli.c as it better describes what the app does
Type: refactor
Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I502e7566fba4376c68fbe41de9e45079a159e864
Type: improvement
When an IPSec interface is first constructed, the end node of the feature arc is not changed, which means it is interface-output.
This means that traffic directed into adjacencies on the link, that do not have protection (w/ an SA), drop like this:
...
00:00:01:111710: ip4-midchain
tx_sw_if_index 4 dpo-idx 24 : ipv4 via 0.0.0.0 ipsec0: mtu:9000 next:6 flags:[]
stacked-on:
[@1]: dpo-drop ip4 flow hash: 0x00000000
00000000: 4500005c000100003f01cb8cac100202010101010800ecf40000000058585858
00000020: 58585858585858585858585858585858585858585858585858585858
00:00:01:111829: local0-output
ipsec0
00000000: 4500005c000100003f01cb8cac100202010101010800ecf40000000058585858
00000020: 5858585858585858585858585858585858585858585858585858585858585858
00000040: 58585858585858585858585858585858585858585858585858585858c2cf08c0
00000060: 2a2c103cd0126bd8b03c4ec20ce2bd02dd77b3e3a4f49664
00:00:01:112017: error-drop
rx:pg1
00:00:01:112034: drop
local0-output: interface is down
although that's a drop, no packets should go to local0, and we want all IPvX packets to go through ipX-drop.
This change sets the interface's end-arc node to the appropriate drop node when the interface is created, and when the last protection is removed.
The resulting drop is:
...
00:00:01:111504: ip4-midchain
tx_sw_if_index 4 dpo-idx 24 : ipv4 via 0.0.0.0 ipsec0: mtu:9000 next:0 flags:[]
stacked-on:
[@1]: dpo-drop ip4 flow hash: 0x00000000
00000000: 4500005c000100003f01cb8cac100202010101010800ecf40000000058585858
00000020: 58585858585858585858585858585858585858585858585858585858
00:00:01:111533: ip4-drop
ICMP: 172.16.2.2 -> 1.1.1.1
tos 0x00, ttl 63, length 92, checksum 0xcb8c dscp CS0 ecn NON_ECN
fragment id 0x0001
ICMP echo_request checksum 0xecf4 id 0
00:00:01:111620: error-drop
rx:pg1
00:00:01:111640: drop
null-node: blackholed packets
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: I7e7de23c541d9f1210a05e6984a688f1f821a155
Adding flow cache support to improve outbound IPv4/IPSec SPD lookup
performance. Details about flow cache:
Mechanism:
1. First packet of a flow will undergo linear search in SPD
table. Once a policy match is found, a new entry will be added
into the flow cache. From 2nd packet onwards, the policy lookup
will happen in flow cache.
2. The flow cache is implemented using bihash without collision
handling. This will avoid the logic to age out or recycle the old
flows in flow cache. Whenever a collision occurs, old entry will
be overwritten by the new entry. Worst case is when all the 256
packets in a batch result in collision and fall back to linear
search. Average and best case will be O(1).
3. The size of flow cache is fixed and decided based on the number
of flows to be supported. The default is set to 1 million flows.
This can be made as a configurable option as a next step.
4. Whenever a SPD rule is added/deleted by the control plane, the
flow cache entries will be completely deleted (reset) in the
control plane. The assumption here is that SPD rule add/del is not
a frequent operation from control plane. Flow cache reset is done,
by putting the data plane in fall back mode, to bypass flow cache
and do linear search till the SPD rule add/delete operation is
complete. Once the rule is successfully added/deleted, the data
plane will be allowed to make use of the flow cache. The flow
cache will be reset only after flushing out the inflight packets
from all the worker cores using
vlib_worker_wait_one_loop().
Details about bihash usage:
1. A new bihash template (16_8) is added to support IPv4 5 tuple.
BIHASH_KVP_PER_PAGE and BIHASH_KVP_AT_BUCKET_LEVEL are set
to 1 in the new template. It means only one KVP is supported
per bucket.
2. Collision handling is avoided by calling
BV (clib_bihash_add_or_overwrite_stale) function.
Through the stale callback function pointer, the KVP entry
will be overwritten during collision.
3. Flow cache reset is done using
BV (clib_bihash_foreach_key_value_pair) function.
Through the callback function pointer, the KVP value is reset
to ~0ULL.
MRR performance numbers with 1 core, 1 ESP Tunnel, null-encrypt,
64B for different SPD policy matching indices:
SPD Policy index : 1 10 100 1000
Throughput : MPPS/MPPS MPPS/MPPS MPPS/MPPS KPPS/MPPS
(Baseline/Optimized)
ARM Neoverse N1 : 5.2/4.84 4.55/4.84 2.11/4.84 329.5/4.84
ARM TX2 : 2.81/2.6 2.51/2.6 1.27/2.6 176.62/2.6
INTEL SKX : 4.93/4.48 4.29/4.46 2.05/4.48 336.79/4.47
Next Steps:
Following can be made as a configurable option through startup
conf at IPSec level:
1. Enable/Disable Flow cache.
2. Bihash configuration like number of buckets and memory size.
3. Dual/Quad loop unroll can be applied around bihash to further
improve the performance.
4. The same flow cache logic can be applied for IPv6 as well as in
IPSec inbound direction. A deeper and wider flow cache using
bihash_40_8 can replace existing bihash_16_8, to make it
common for both IPv4 and IPv6 in both outbound and
inbound directions.
Following changes are made based on the review comments:
1. ON/OFF flow cache through startup conf. Default: OFF
2. Flow cache stale entry detection using epoch counter.
3. Avoid host order endianness conversion during flow cache
lookup.
4. Move IPSec startup conf to a common file.
5. Added SPD flow cache unit test case
6. Replaced bihash with vectors to implement flow cache.
7. ipsec_add_del_policy API is not mpsafe. Cleaned up
inflight packets check in control plane.
Type: improvement
Signed-off-by: mgovind <govindarajan.Mohandoss@arm.com>
Signed-off-by: Zachary Leaf <zachary.leaf@arm.com>
Tested-by: Jieqiang Wang <jieqiang.wang@arm.com>
Change-Id: I62b4d6625fbc6caf292427a5d2046aa5672b2006
Type: feature
Gaps in the sequence numbers received on an SA indicate packets that were lost.
Gaps are identified using the anti-replay window that records the sequences seen.
Publish the number of lost packets in the stats segment at /net/ipsec/sa/lost
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: I8af1c09b7b25a705e18bf82e1623b3ce19e5a74d
- Upgrade python package requirements for test & docs
- Clean up docs generation warnings
- Consolidate python requirements for docs in test
requirements specs.
- Upgrade pip
Type: make
Change-Id: I74a3924b43ed93d15b32ec9f6fc41ed1ba95b69b
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Type: fix
two problems;
1 - just because anti-reply is not enabled doesn't mean the high sequence
number should not be used.
- fix, there needs to be some means to detect a wrapped packet, so we
use a window size of 2^30.
2 - The SA object was used as a scratch pad for the high-sequence
number used during decryption. That means that once the batch has been
processed the high-sequence number used is lost. This means it is not
possible to distinguish this case:
if (seq < IPSEC_SA_ANTI_REPLAY_WINDOW_LOWER_BOUND (tl))
{
...
if (post_decrypt)
{
if (hi_seq_used == sa->seq_hi)
/* the high sequence number used to succesfully decrypt this
* packet is the same as the last-sequnence number of the SA.
* that means this packet did not cause a wrap.
* this packet is thus out of window and should be dropped */
return 1;
else
/* The packet decrypted with a different high sequence number
* to the SA, that means it is the wrap packet and should be
* accepted */
return 0;
}
- fix: don't use the SA as a scratch pad, use the 'packet_data' - the
same place that is used as the scratch pad for the low sequence number.
other consequences:
- An SA doesn't have seq and last_seq, it has only seq; the sequence
numnber of the last packet tx'd or rx'd.
- there's 64bits of space available on the SA's first cache line. move
the AES CTR mode IV there.
- test the ESN/AR combinations to catch the bugs this fixes. This
doubles the amount of tests, but without AR on they only run for 2
seconds. In the AR tests, the time taken to wait for packets that won't
arrive is dropped from 1 to 0.2 seconds thus reducing the runtime of
these tests from 10-15 to about 5 sceonds.
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: Iaac78905289a272dc01930d70decd8109cf5e7a5
This module implement Python access to the VPP statistics segment. It
accesses the data structures directly in shared memory.
VPP uses optimistic locking, so data structures may change underneath
us while we are reading. Data is copied out and it's important to
spend as little time as possible "holding the lock".
Counters are stored in VPP as a two dimensional array.
Index by thread and index (typically sw_if_index).
Simple counters count only packets, Combined counters count packets
and octets.
Counters can be accessed in either dimension.
stat['/if/rx'] - returns 2D lists
stat['/if/rx'][0] - returns counters for all interfaces for thread 0
stat['/if/rx'][0][1] - returns counter for interface 1 on thread 0
stat['/if/rx'][0][1]['packets'] - returns the packet counter
for interface 1 on thread 0
stat['/if/rx'][:, 1] - returns the counters for interface 1 on all threads
stat['/if/rx'][:, 1].packets() - returns the packet counters for
interface 1 on all threads
stat['/if/rx'][:, 1].sum_packets() - returns the sum of packet counters for
interface 1 on all threads
stat['/if/rx-miss'][:, 1].sum() - returns the sum of packet counters for
interface 1 on all threads for simple counters
Type: refactor
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I1fe7f7c7d11378d06be8276db5e1900ecdb8f515
Signed-off-by: Ole Troan <ot@cisco.com>
VppTestCase now has vpp_worker_count property set to number of workers.
This can be overriden by child classes. Also overriden by
VPP_WORKER_CONFIG variable for legacy reasons.
Type: improvement
Change-Id: Ic328bacb9003ddf9e92815767653bd362aa7f086
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Make the ipsec[46]-tun-input nodes siblings of device-input so that
input features can be enabled on them. Register ipsec-tun for feature
updates. When a feature is enabled on the device-input arc and the
ifindex is an IPSec tunnel, change the end node of the arc for that
ifindex to be the appropriate ESP decrypt node. Set a flag on the
tunnel to indicate that the feature arc should be started for packets
input on the tunnel.
Test input policing on ESP IPSec tunnels.
Type: improvement
Signed-off-by: Brian Russell <brian@graphiant.com>
Change-Id: I3b9f047e5e737f3ea4c58fc82cd3c15700b6f9f7
support
Type: feature
attmpet 2. this includes changes in ah_encrypt that don't use
uninitialised memory when doing tunnel mode fixups.
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: Ie3cb776f5c415c93b8a5ee22f22586fd0181110d
This reverts commit c7eaa711f3.
Reason for revert: The jenkins job named 'vpp-merge-master-ubuntu1804-x86_64' had 2 IPv6 AH tests fail after the change was merged. Those 2 tests also failed the next time that job ran after an unrelated change was merged.
Change-Id: I0e2c3ee895114029066c82624e79807af575b6c0
Signed-off-by: Matthew Smith <mgsmith@netgate.com>
Type: fix
it's been 2 releases since it was marked deprecated.
Signed-off-by: Neale Ranns <nranns@cisco.com>
Change-Id: I0eba7ed607826ed0d00e7d2d8f9b27d09e8e9a6e
Type: feature
- use tunnel_encap_decap_flags to control the copying of DSCP/ECN/etc
during IPSEC tunnel mode encap.
- use DSCP value to have fixed encap value.
Signed-off-by: Neale Ranns <nranns@cisco.com>
Change-Id: If4f51fd4c1dcbb0422aac9bd078e5c14af5bf11f
Not all ESP crypto algorithms require padding/alignment to be the same
as AES block/IV size. CCM, CTR and GCM all have no padding/alignment
requirements, and the RFCs indicate that no padding (beyond ESPs 4 octet
alignment requirement) should be used unless TFC (traffic flow
confidentiality) has been requested.
CTR: https://tools.ietf.org/html/rfc3686#section-3.2
GCM: https://tools.ietf.org/html/rfc4106#section-3.2
CCM: https://tools.ietf.org/html/rfc4309#section-3.2
- VPP is incorrectly using the IV/AES block size to pad CTR and GCM.
These modes do not require padding (beyond ESPs 4 octet requirement), as
a result packets will have unnecessary padding, which will waste
bandwidth at least and possibly fail certain network configurations that
have finely tuned MTU configurations at worst.
Fix this as well as changing the field names from ".*block_size" to
".*block_align" to better represent their actual (and only) use. Rename
"block_sz" in esp_encrypt to "esp_align" and set it correctly as well.
test: ipsec: Add unit-test to test for RFC correct padding/alignment
test: patch scapy to not incorrectly pad ccm, ctr, gcm modes as well
- Scapy is also incorrectly using the AES block size of 16 to pad CCM,
CTR, and GCM cipher modes. A bug report has been opened with the
and acknowledged with the upstream scapy project as well:
https://github.com/secdev/scapy/issues/2322
Ticket: VPP-1928
Type: fix
Signed-off-by: Christian Hopps <chopps@labn.net>
Change-Id: Iaa4d6a325a2e99fdcb2c375a3395bcfe7947770e
Fix of the top 11 python issues flagged as BLOCKER.
Ticket: VPP-1856
Type: fix
Change-Id: Icf4691e62f4a69d6ee196b6d6e2ab52d961b5c76
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
Type: fix
1 - big packets; chained buffers and those without enoguh space to add
ESP header
2 - IPv6 extension headers in packets that are encrypted/decrypted
3 - Interface protection with SAs that have null algorithms
Signed-off-by: Neale Ranns <nranns@cisco.com>
Change-Id: Ie330861fb06a9b248d9dcd5c730e21326ac8e973
APIs for dedicated IPSec tunnels will remain in this release and are
used to programme the IPIP tunnel protect. APIs will be removed in a
future release.
see:
https://wiki.fd.io/view/VPP/IPSec
Type: feature
Change-Id: I0f01f597946fdd15dfa5cae3643104d5a9c83089
Signed-off-by: Neale Ranns <nranns@cisco.com>
Type: fix
Ticket: VPP-1756
the block-size was set to 0 resulting in incorrect placement of the ESP
footer.
add tests for NULL encrypt + integ.
Change-Id: I8ab3afda8e68f9ff649540cba3f2cac68f12bbba
Signed-off-by: Neale Ranns <nranns@cisco.com>
Type: fix
Several Fixes:
1 - Anti-replay did not work with GCM becuase it overwrote the sequence
number in the ESP header. To fix i added the seq num to the per-packet
data so it is preserved
2 - The high sequence number was not byte swapped during ESP encrypt.
3 - openssl engine was the only one to return FAIL_DECRYPT for bad GCM
the others return BAD_HMAC. removed the former
4 - improved tracing to show the low and high seq numbers
5 - documented the anti-replay window checks
6 - fixed scapy patch for ESN support for GCM
7 - tests for anti-reply (w/ and w/o ESN) for each crypto algo
Change-Id: Id65d96b6d1d4dd821b2ab557e87468fff6d70e5b
Signed-off-by: Neale Ranns <nranns@cisco.com>
The current code only allowed access to the main thread error counters.
That is not so useful for a multi worker instance.
No return a vector indexed by thread of counter_t values.
Type: fix
Change-Id: Ie322c8889c0c8175e1116e71de04a2cf453b9ed7
Signed-off-by: Ole Troan <ot@cisco.com>
* Configure tests to raise exception if cli_inband fails.
* Fix failing tests.
* Add filename detail to pcap.stat clib_error_return for debugging.
Note: this change identifies spurious issues with packet-generator such as:
CliFailedCommandError: packet-generator capture: pcap file
'/tmp/vpp-unittest-Test6RD-v09RPA/pg0_out.pcap' does not exist.
These issues resolve themselves on remaining test passes.
Change-Id: Iecbd09daee954d892306d11baff3864a43c5b603
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
This change adds a consistent interface for adding test-specific show commands to
log.txt.
It also adds log markers for the execution of setUp[Class], tearDown[Class]
in the logs.
Change-Id: I7d42e396e594a59e866a7d55dac0af25548e657a
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
refactor the IPSEC tests a bit so we can parameterise
the setup.
Change-Id: I777e5eb8f29ca1dce3dd273ebd05dae5846790af
Signed-off-by: Neale Ranns <nranns@cisco.com>
