VPP debug image with worker threads hit assert on adding IP route with traffic (VPP-892)

When stacking DPOs the VLIB graph is also updated to add the edge between the nodes, if this edge does not yet exist. This addition should be done with the workers stopped.

Change-Id: I327e4d7d26f0b23eb280f17e4619ff2093ff7940
Signed-off-by: Neale Ranns <nranns@cisco.com>
(cherry picked from commit c02bd03ddf)

.gitreview

@@ -2,3 +2,4 @@
 host=gerrit.fd.io
 port=29418
 project=vpp
+defaultbranch=stable/1704

MAINTAINERS

@@ -57,7 +57,7 @@ M:	Damjan Marion <damarion@cisco.com>
 F:	src/vnet/devices/
 
 VNET Device Drivers - DPDK Crypto
-M:	Sergio Gonzales Monroy <sergio.gonzalez.monroy@intel.com>
+M:	Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
 F:	src/devices/dpdk/ipsec/
 
 VNET Feature Arcs
@@ -81,7 +81,7 @@ M:	Pablo Camarillo <pcamaril@cisco.com>
 F:	src/vnet/sr/
 
 VNET IPSec
-M:	Sergio Gonzales Monroy <sergio.gonzalez.monroy@intel.com>
+M:	Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
 M:	Matus Fabian <matfabia@cisco.com>
 F:	src/vnet/ipsec/
 

Makefile

@@ -40,7 +40,7 @@ DEB_DEPENDS  = curl build-essential autoconf automake bison libssl-dev ccache
 DEB_DEPENDS += debhelper dkms git libtool libganglia1-dev libapr1-dev dh-systemd
 DEB_DEPENDS += libconfuse-dev git-review exuberant-ctags cscope pkg-config
 DEB_DEPENDS += lcov chrpath autoconf nasm
-DEB_DEPENDS += python-dev python-virtualenv python-pip libffi6
+DEB_DEPENDS += python-all python-dev python-virtualenv python-pip libffi6
 ifeq ($(OS_VERSION_ID),14.04)
 	DEB_DEPENDS += openjdk-8-jdk-headless
 else

RELEASE.md

@@ -1,12 +1,105 @@
 # Release Notes    {#release_notes}
 
 * @subpage release_notes_1704
+* @subpage release_notes_17011
 * @subpage release_notes_1701
 * @subpage release_notes_1609
 * @subpage release_notes_1606
 
 @page release_notes_1704 Release notes for VPP 17.04
 
+More than 500 commits since the 1701 release.
+
+## Features
+- Infrastructure
+  - make test improvements
+  - vnet: add device-input threadplacement infra
+  - 64 bit per-thread counters
+  - process restart cli
+  - High performance timer wheels
+  - Plugin infrastructure improvements
+    - Support for .default_disabled, .version_required
+  - Added MAINTAINERS file
+
+- Host stack
+  - TCP stack (experimental)
+  - DHCPv4 / DHCPv6 relay multi-destination
+  - DHCPv4 option 82
+  - ND proxy
+  - Attached hosts
+  - Consolidated DHCPv4 and DHCPv6 implementation
+
+- Interfaces
+  - DPDK 17.02 (retire support for DPDK 16.07)
+  - Add memif - packet memory interface for intra-host communication
+  - vhost: support interrupt mode
+  - DPDK as plugin (retired vpp_lite)
+  - DPDPK input optimizations
+  - Loopback interface allocation scheme
+
+- Network features
+  - IP Multicast FIB
+
+  - Bridging
+    - Learning on local interfaces
+    - Flushing of MACs from the L2 FIB
+
+  - SNAT
+    - CGN (Deterministic and dynamic)
+    - CGN configurable port allocation algorithm
+    - ICMP support
+    - Tentant VRF id for SNAT outside addresses
+    - Session dump / User dump
+    - Port allocation per protocol
+
+  - Security groups
+    - Routed interface support
+    - L2+L3 unified processing node
+    - Improve fragment handling
+
+  - Segement routing v6
+    - SR policies with weighted SID lists
+    - Binding SID
+    - SR steering policies
+    - SR Local SIDs
+    - Framework to expand local SIDs w/plugins
+    - Documentation
+
+  - IOAM
+    - UDP Pinger w/path fault isolation
+    - IOAM as type 2 metadata in NSH
+    - IAOM raw IPFIX collector and analyzer
+    - Anycast active server selection
+    - Documentation
+    - SRv6 Local SID
+    - IP6 HBH header and SR header co-existence
+    - Active probe
+
+  - LISP
+    - Statistics collection
+    - Generalize encap for overlay transport (vxlan-gpe support)
+    - Improve data plane speed
+
+  - GPE
+    - CLI
+    - NSH added to encap/decap path
+    - Renamed LISP GPE API to GPE
+
+  - MPLS
+    - Performance improvements (quad loop)
+
+  - BFD
+    - Command line interface
+    - Echo function
+    - Remote demand mode
+    - SHA1 authentication
+
+  - IPsec
+    - IKEv2 initiator features
+
+  - VXLAN
+    - unify IP4/IP6 control plane handling
+
 ## API changes
 
 - Python API: To avoid conflicts between VPP API messages names and
@@ -23,23 +116,112 @@
   For backwards compatibility VPP API methods are left in the main
   name space (VPP), but will be removed from 17.07.
 
+  - Python API: Change from cPython to CFFI.
+
 - create_loopback message to be replaced with create_loopback_instance
   create_loopback will be removed from 17.07.
   https://gerrit.fd.io/r/#/c/5572/
 
-@todo Release 17.04 needs release notes.
+## Known issues
+
+For the full list of issues please reffer to fd.io [JIRA](https://jira.fd.io).
+
+## Issues fixed
+
+For the full list of fixed issues please reffer to:
+- fd.io [JIRA](https://jira.fd.io)
+- git [commit log](https://git.fd.io/vpp/log/?h=stable/1704)
+
+@page release_notes_17011 Release notes for VPP 17.01.1
+
+This is bug fix release.
+
+For the full list of fixed issues please reffer to:
+- fd.io [JIRA](https://jira.fd.io)
+- git [commit log](https://git.fd.io/vpp/log/?h=stable/1701)
 
 @page release_notes_1701 Release notes for VPP 17.01
 
 @note This release was for a while known as 16.12.
-@todo Release 17.01 needs release notes. It will show up here soon...
 
 ## Features
 
+- [Integrated November 2016 DPDK release](http://www.dpdk.org/doc/guides/rel_notes/release_16_11.html)
+
+- Complete rework of Forwarding Information Base (FIB)
+
+- Performance Improvements
+  - Improvements in DPDK input and output nodes
+  - Improvements in L2 path
+  - Improvmeents in IPv4 lookup node
+
+- Feature Arcs Improvements
+  - Consolidation of the code
+  - New feature arcs
+    - device-input
+    - interface-output
+
+- DPDK Cryptodev Support
+  - Software and Hardware Crypto Support
+
+- DPDK HQoS support
+
+- Simple Port Analyzer (SPAN)
+
+- Bidirectional Forwarding Detection
+  - Basic implementation
+
+- IPFIX Improvements
+
+- L2 GRE over IPSec tunnels
+
+- Link Layer Discovery Protocol (LLDP)
+
+- Vhost-user Improvements
+  - Performance Improvements
+  - Multiqueue
+  - Reconnect
+
+- LISP Enhancements
+  - Source/Dest control plane support
+  - L2 over LISP and GRE
+  - Map-Register/Map-Notify/RLOC-probing support
+  - L2 API improvements, overall code hardening
+
+- Plugins:
+  - New: ACL
+  - New: Flow per Packet
+  - Improved: SNAT
+    - Mutlithreading
+    - Flow export
+
+- Doxygen Enhancements
+
+- Luajit API bindings
+
+- API Refactoring
+  - file split
+  - message signatures
+
+- Python and Scapy based unit testing infrastructure
+  - Infrastructure
+  - Various tests
+
+- Packet Generator improvements
+
+- TUN/TAP jumbo frames support
+
+- Other various bug fixes and improvements
+
 ## Known issues
 
+For the full list of issues please reffer to fd.io [JIRA](https://jira.fd.io).
+
 ## Issues fixed
 
+For the full list of fixed issues please reffer to:
+- fd.io [JIRA](https://jira.fd.io)
+- git [commit log](https://git.fd.io/vpp/log/?h=stable/1701)
 
 @page release_notes_1609 Release notes for VPP 16.09
 

build-data/platforms.mk

@@ -59,10 +59,6 @@ install-deb: $(patsubst %,%-find-source,$(ROOT_PACKAGES))
 	./scripts/find-vpp-api-java-contents $(INSTALL_PREFIX)$(ARCH)	\
 	 deb/debian/vpp-api-java.install ;				\
 									\
-	: vpp-api-python package ;					\
-	./scripts/find-vpp-api-python-contents $(INSTALL_PREFIX)$(ARCH)	\
-	 deb/debian/vpp-api-python.install ;				\
-									\
 	: bin package needs startup config ; 				\
 	echo ../../src/vpp/conf/startup.conf /etc/vpp 			\
 	   >> deb/debian/vpp.install ;					\

build-root/deb/debian/control

@@ -2,7 +2,7 @@ Source: vpp
 Section: net
 Priority: extra
 Maintainer: Cisco OpenVPP Packaging Team <bogus.address@cisco.com>
-Build-Depends: debhelper (>= 9), dh-systemd, dh-python, chrpath
+Build-Depends: debhelper (>= 9), dh-systemd, dh-python, chrpath, python-all
 Standards-Version: 3.9.4
 
 Package: vpp

build-root/deb/debian/rules

@@ -16,9 +16,15 @@ include /usr/share/dpkg/default.mk
 # package maintainers to append LDFLAGS
 #export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed
 
+export PYBUILD_NAME = vpp-api-python
+export PYBUILD_DIR = ../../src/vpp-api/python
+export PYBUILD_DESTDIR_python2=debian/vpp-api-python/
+export PYBUILD_DISABLE_python2=test
+export PYBUILD_SYSTEM=distutils
+
 # main packaging script based on dh7 syntax
 %:
-	dh $@ --with systemd,python2
+	dh $@ --with systemd,python2 --buildsystem=pybuild
 
 override_dh_install:
 	dh_install --exclude .git

build-root/deb/debian/vpp-api-python.postinst

@@ -1,5 +0,0 @@
-#!/bin/sh -e
-
-# after installing python-api files
-python2_sitedir=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-easy_install --install-dir=$python2_sitedir  -z $python2_sitedir/vpp_papi/vpp_papi-*.egg

build-root/deb/debian/vpp-api-python.prerm

@@ -1,8 +0,0 @@
-#!/bin/sh -e
-
-# before removing python-api files
-python2_sitedir=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-easy_install --install-dir=$python2_sitedir -mxNq vpp_papi
-
-# the egg has been copied during install
-rm $python2_sitedir/vpp_papi-*.egg

build-root/rpm/vpp.spec

@@ -89,7 +89,7 @@ This package contains the java bindings for the vpp api
 %package api-python
 Summary: VPP api python bindings
 Group: Development/Libraries
-Requires: vpp = %{_version}-%{_release}, vpp-lib = %{_version}-%{_release}, python-setuptools libffi-devel
+Requires: vpp = %{_version}-%{_release}, vpp-lib = %{_version}-%{_release}, python-setuptools libffi-devel python-cffi
 
 %description api-python
 This package contains the python bindings for the vpp api

build-root/scripts/csit-test-branch

@@ -1,2 +1,2 @@
 #!/bin/sh
-echo oper-170313
+echo rls1704

doxygen/Makefile

@@ -53,7 +53,8 @@ DOXY_SRC_DIRECTORIES = \
 	$(DOXY_SRC)/vlibsocket \
 	$(DOXY_SRC)/vnet \
 	$(DOXY_SRC)/vpp \
-	$(DOXY_SRC)/vpp-api
+	$(DOXY_SRC)/vpp-api \
+	$(DOXY_SRC)/examples
 
 # Input directories and files
 DOXY_INPUT ?= \
@@ -72,9 +73,8 @@ DOXY_INPUT := $(subst $(WS_ROOT)/,,$(DOXY_INPUT))
 # These must be left-anchored paths for the regexp below to work.
 DOXY_EXCLUDE ?= \
 	$(DOXY_SRC)/vlib/vlib/buffer.c \
-	$(DOXY_SRC)/vlib/example \
 	$(DOXY_SRC)/vpp-api/lua \
-	plugins/sample-plugin
+	$(DOXY_SRC)/examples/sample-plugin
 
 # Generate a regexp for filenames to exclude
 DOXY_EXCLUDE_REGEXP = ($(subst .,\.,$(shell echo '$(strip $(DOXY_EXCLUDE))' | sed -e 's/ /|/g')))

doxygen/user_doc.md

@@ -4,13 +4,14 @@ User Documentation    {#user_doc}
 Several modules provide operational, dataplane-user focused documentation.
 
 - [GUI guided user demo](https://wiki.fd.io/view/VPP_Sandbox/vpp-userdemo)
-- @subpage qos_doc
-- @subpage ipsec_gre_doc
-- @subpage dpdk_crypto_ipsec_doc
-- @subpage map_doc
-- @subpage lldp_doc
-- @subpage ioam_plugin_doc
-- @subpage lb_plugin_doc
-- @subpage flowperpkt_plugin_doc
-- @subpage span_doc
 - @subpage bfd_doc
+- @subpage ioam_plugin_doc
+- @subpage ipsec_gre_doc
+- @subpage lb_plugin_doc
+- @subpage lldp_doc
+- @subpage map_doc
+- @subpage dpdk_crypto_ipsec_doc
+- @subpage flowperpkt_plugin_doc
+- @subpage qos_doc
+- @subpage span_doc
+- @subpage srv6_doc

extras/scripts/lf-release.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+VPP_DIR=`dirname $0`
+VER=$($VPP_DIR/version)
+MAJOR=$(echo $VER | sed -r 's/(.{5}).*/\1/')
+MAJOR_NODOT=$(echo $MAJOR | sed -e "s/\.//")
+FULL=$(echo $VER | sed -e "s/-release//")
+
+sed -e "s/\${major_release_nodot}/$MAJOR_NODOT/g" \
+    -e "s/\${major_release}/$MAJOR/g" \
+    -e "s/\${full_release}/$FULL/g" \
+    $VPP_DIR/lf-release.txt

extras/scripts/lf-release.txt

@@ -0,0 +1,47 @@
+To: helpdesk@fd.io
+Subject: Publish vpp release artifacts
+
+vpp has completed release ${full_release}.
+
+Please copy:
+
+https://nexus.fd.io/content/repositories/fd.io.stable.${major_release_nodot}.centos7/io/fd/vpp/*/${full_release}-release.x86_64/*.rpm
+
+to
+
+https://nexus.fd.io/content/repositories/fd.io.centos7/io/fd/vpp/*/${major_release}-release.x86_64/*.rpm
+
+Please note: the groupId will be io.fd.vpp, the maven version will be ${major_release}-release.x86_64, the artifactId will be the '*' between io/fd/vpp/ and ${major_release}-release.x86_64
+
+Please copy:
+
+https://nexus.fd.io/content/repositories/fd.io.stable.${major_release_nodot}.ubuntu.trusty.main/io/fd/vpp/*/${full_release}_amd64/*.deb 
+
+to
+
+https://nexus.fd.io/content/repositories/fd.io.ubuntu.trusty.main/io/fd/vpp/*/${major_release}_amd64/*.deb 
+
+Please note: the groupId will be io.fd.vpp, the maven version will be ${major_release}_amd64, the artifactId will be the '*' between io/fd/vpp/ and ${major_release}_amd64
+
+Please copy:
+
+https://nexus.fd.io/content/repositories/fd.io.stable.${major_release_nodot}.ubuntu.xenial.main/io/fd/vpp/*/${full_release}_amd64/*.deb 
+
+to
+
+https://nexus.fd.io/content/repositories/fd.io.ubuntu.xenial.main/io/fd/vpp/*/${major_release}_amd64/*.deb
+
+Please note: the groupId will be io.fd.vpp, the maven version will be ${major_release}_amd64, the artifactId will be the '*' between io/fd/vpp/ and ${major_release}_amd64
+
+Please copy
+
+https://nexus.fd.io/content/repositories/fd.io.snapshot/io/fd/vpp/*/${full_release}-SNAPSHOT/*.jar 
+where you select the jar file with the highest build number
+
+to
+
+https://nexus.fd.io/content/repositories/fd.io.release/io/fd/vpp/*/${full_release}/*.jar 
+
+Please note: the groupId will be io.fd.vpp, the maven version will be ${full_release}, the artifactId will be the '*' between io/fd/vpp/ and ${full_release}
+
+When performing all of these copies, please make sure to *not* copy the .pom files.

src/configure.ac

@@ -1,4 +1,4 @@
-AC_INIT([vpp], [17.04], [vpp-dev@fd.io])
+AC_INIT([vpp], [17.04.2], [vpp-dev@fd.io])
 LT_INIT
 AC_CONFIG_AUX_DIR([.])
 AM_INIT_AUTOMAKE([subdir-objects])
@@ -154,7 +154,6 @@ PLUGIN_ENABLED(lb)
 PLUGIN_ENABLED(memif)
 PLUGIN_ENABLED(sixrd)
 PLUGIN_ENABLED(snat)
-PLUGIN_DISABLED(srv6sample)
 
 ###############################################################################
 # Dependency checks

src/examples/srv6-sample-localsid/srv6_sample_localsid_doc.md

@@ -1,12 +1,4 @@
-# SRv6 Sample LocalSID documentation    {#srv6_plugin_doc}
+# Sample SRv6 LocalSID documentation    {#srv6_plugin_doc}
-
-## Disclaimer
-
-This is a memo intended to contain documentation for the sample SRv6 LocalSID behavior plugin
-Everything that is not directly obvious should come here.
-For any feedback on content that should be explained please mailto:pcamaril@cisco.com
- 
-This plugin refers to Segment Routing. Please read the SR documentation first.
 
 ## Introduction
 

src/plugins/Makefile.am

@@ -69,10 +69,6 @@ if ENABLE_SNAT_PLUGIN
 include snat.am
 endif
 
-if ENABLE_SRV6SAMPLE_PLUGIN
-include sample_srv6_localsid.am
-endif
-
 include ../suffix-rules.mk
 
 # Remove *.la files

src/plugins/acl.am

@@ -22,6 +22,7 @@ acl_plugin_la_SOURCES =				\
 	acl/l2sess.c				\
 	acl/l2sess_node.c			\
 	acl/l2sess.h				\
+	acl/manual_fns.h			\
 	acl/acl_plugin.api.h
 
 API_FILES += acl/acl.api
@@ -29,8 +30,9 @@ API_FILES += acl/acl.api
 nobase_apiinclude_HEADERS +=			\
   acl/acl_all_api_h.h				\
   acl/acl_msg_enum.h				\
+  acl/manual_fns.h				\
   acl/acl.api.h
 
-acl_test_plugin_la_SOURCES = acl/acl_test.c acl/acl_plugin.api.h
+acl_test_plugin_la_SOURCES = acl/acl_test.c acl/acl_plugin.api.h acl/acl_all_api.h
 
 # vi:syntax=automake

src/plugins/acl/acl.api

@@ -60,7 +60,7 @@ define acl_plugin_get_version_reply
     @param tcp_flags_value - if proto==6, mask to AND the TCP flags in the packet with
 */
 
-typeonly manual_print manual_endian define acl_rule
+typeonly manual_print define acl_rule
 {
   u8 is_permit;
   u8 is_ipv6;
@@ -104,7 +104,7 @@ typeonly manual_print manual_endian define acl_rule
     @param src_ip_prefix_len - Source prefix length
 */
 
-typeonly manual_print manual_endian define macip_acl_rule
+typeonly manual_print define macip_acl_rule
 {
   u8 is_permit;
   u8 is_ipv6;
@@ -161,7 +161,7 @@ define acl_add_replace_reply
     @param acl_index - ACL index to delete
 */
 
-define acl_del
+manual_print define acl_del
 {
   u32 client_index;
   u32 context;
@@ -190,7 +190,7 @@ define acl_del_reply
     @param acl_index - index of ACL for the operation
 */
 
-define acl_interface_add_del
+manual_print define acl_interface_add_del
 {
   u32 client_index;
   u32 context;
@@ -224,7 +224,7 @@ define acl_interface_add_del_reply
     @param acls - vector of ACL indices
 */
 
-manual_endian define acl_interface_set_acl_list
+manual_print define acl_interface_set_acl_list
 {
   u32 client_index;
   u32 context;
@@ -266,7 +266,7 @@ define acl_dump
     @param r - Array of rules within this ACL
 */
 
-manual_print manual_endian define acl_details
+manual_endian manual_print define acl_details
 {
   u32 context;
   u32 acl_index;
@@ -296,7 +296,7 @@ define acl_interface_list_dump
     @param acls - the vector of ACL indices
 */
 
-manual_endian define acl_interface_list_details
+define acl_interface_list_details
 {
   u32 context;
   u32 sw_if_index;
@@ -313,7 +313,7 @@ manual_endian define acl_interface_list_details
     @param r - vector of MACIP ACL rules
 */
 
-manual_print manual_endian define macip_acl_add
+manual_endian manual_print define macip_acl_add
 {
   u32 client_index;
   u32 context;
@@ -341,7 +341,7 @@ define macip_acl_add_reply
     @param acl_index - MACIP ACL index to delete
 */
 
-define macip_acl_del
+manual_print define macip_acl_del
 {
   u32 client_index;
   u32 context;
@@ -367,7 +367,7 @@ define macip_acl_del_reply
     @param acl_index - MACIP ACL index
 */
 
-define macip_acl_interface_add_del
+manual_print define macip_acl_interface_add_del
 {
   u32 client_index;
   u32 context;
@@ -409,7 +409,7 @@ define macip_acl_dump
     @param r - rules comprising this ACL
 */
 
-manual_print manual_endian define macip_acl_details
+manual_endian manual_print define macip_acl_details
 {
   u32 context;
   u32 acl_index;

src/plugins/acl/acl.c

@@ -1767,8 +1767,6 @@ vl_api_macip_acl_interface_get_t_handler (vl_api_macip_acl_interface_get_t *
   vl_msg_api_send_shmem (q, (u8 *) & rmp);
 }
 
-
-
 /* Set up the API message handling tables */
 static clib_error_t *
 acl_plugin_api_hookup (vlib_main_t * vm)
@@ -1968,6 +1966,10 @@ acl_sw_interface_add_del (vnet_main_t * vnm, u32 sw_if_index, u32 is_add)
   if (0 == is_add) {
     vlib_process_signal_event (am->vlib_main, am->fa_cleaner_node_index,
                                ACL_FA_CLEANER_DELETE_BY_SW_IF_INDEX, sw_if_index);
+    /* also unapply any ACLs in case the users did not do so. */
+    macip_acl_interface_del_acl(am, sw_if_index);
+    acl_interface_reset_inout_acls (sw_if_index, 0);
+    acl_interface_reset_inout_acls (sw_if_index, 1);
   }
   return 0;
 }
@@ -2010,6 +2012,11 @@ acl_set_aclplugin_fn (vlib_main_t * vm,
     }
     goto done;
   }
+  if (unformat (input, "l4-match-nonfirst-fragment %u", &val))
+    {
+      am->l4_match_nonfirst_fragment = (val != 0);
+      goto done;
+    }
   if (unformat (input, "session")) {
     if (unformat (input, "clear")) {
       acl_main_t *am = &acl_main;
@@ -2120,10 +2127,15 @@ acl_show_aclplugin_fn (vlib_main_t * vm,
         u64 n_dels = sw_if_index < vec_len(am->fa_session_dels_by_sw_if_index) ? am->fa_session_dels_by_sw_if_index[sw_if_index] : 0;
         out0 = format(out0, "sw_if_index %d: add %lu - del %lu = %lu\n", sw_if_index, n_adds, n_dels, n_adds - n_dels);
       }));
+      out0 = format(out0, "\n\nConn cleaner thread counters:\n");
+#define _(cnt, desc) out0 = format(out0, "             %20lu: %s\n", am->cnt, desc);
+      foreach_fa_cleaner_counter;
+#undef _
       vlib_cli_output(vm, "\n\n%s\n\n", out0);
       vlib_cli_output(vm, "Sessions per interval: min %lu max %lu increment: %f ms current: %f ms",
               am->fa_min_deleted_sessions_per_interval, am->fa_max_deleted_sessions_per_interval,
               am->fa_cleaner_wait_time_increment * 1000.0, ((f64)am->fa_current_cleaner_timer_wait_interval) * 1000.0/(f64)vm->clib_time.clocks_per_second);
+
       vec_free(out0);
     }
   return error;
@@ -2190,10 +2202,20 @@ acl_init (vlib_main_t * vm)
   am->fa_max_deleted_sessions_per_interval = ACL_FA_DEFAULT_MAX_DELETED_SESSIONS_PER_INTERVAL;
   am->fa_cleaner_wait_time_increment = ACL_FA_DEFAULT_CLEANER_WAIT_TIME_INCREMENT;
 
+  am->fa_cleaner_cnt_delete_by_sw_index = 0;
+  am->fa_cleaner_cnt_delete_by_sw_index_ok = 0;
+  am->fa_cleaner_cnt_unknown_event = 0;
+  am->fa_cleaner_cnt_deleted_sessions = 0;
+  am->fa_cleaner_cnt_timer_restarted = 0;
+  am->fa_cleaner_cnt_wait_with_timeout = 0;
+
+
 #define _(N, v, s) am->fa_ipv6_known_eh_bitmap = clib_bitmap_set(am->fa_ipv6_known_eh_bitmap, v, 1);
   foreach_acl_eh
 #undef _
 
+  am->l4_match_nonfirst_fragment = 1;
+
   return error;
 }
 

src/plugins/acl/acl.h

@@ -181,6 +181,9 @@ typedef struct {
   /* EH values that we can skip over */
   uword *fa_ipv6_known_eh_bitmap;
 
+  /* whether to match L4 ACEs with ports on the non-initial fragment */
+  int l4_match_nonfirst_fragment;
+
   /* conn table per-interface conn table parameters */
   u32 fa_conn_table_hash_num_buckets;
   uword fa_conn_table_hash_memory_size;
@@ -209,6 +212,22 @@ typedef struct {
   u32 fa_conn_list_head[ACL_N_TIMEOUTS];
   u32 fa_conn_list_tail[ACL_N_TIMEOUTS];
 
+  /* Counters for the cleaner thread */
+
+#define foreach_fa_cleaner_counter                                         \
+  _(fa_cleaner_cnt_delete_by_sw_index, "delete_by_sw_index events")        \
+  _(fa_cleaner_cnt_delete_by_sw_index_ok, "delete_by_sw_index handled ok") \
+  _(fa_cleaner_cnt_unknown_event, "unknown events received")               \
+  _(fa_cleaner_cnt_deleted_sessions, "sessions deleted")                   \
+  _(fa_cleaner_cnt_timer_restarted, "session idle timers restarted")       \
+  _(fa_cleaner_cnt_wait_with_timeout, "event wait with timeout called")    \
+  _(fa_cleaner_cnt_wait_without_timeout, "event wait w/o timeout called")  \
+  _(fa_cleaner_cnt_event_cycles, "total event cycles")                     \
+  _(fa_cleaner_cnt_already_deleted, "try to delete already deleted conn")  \
+/* end of counters */
+#define _(id, desc) u32 id;
+  foreach_fa_cleaner_counter
+#undef _
 
   /* convenience */
   vlib_main_t * vlib_main;
@@ -219,6 +238,7 @@ typedef struct {
    _(HOPBYHOP , 0  , "IPv6ExtHdrHopByHop")                      \
    _(ROUTING  , 43 , "IPv6ExtHdrRouting")                       \
    _(DESTOPT  , 60 , "IPv6ExtHdrDestOpt")                       \
+   _(FRAGMENT , 44 , "IPv6ExtHdrFragment")                      \
    _(MOBILITY , 135, "Mobility Header")                         \
    _(HIP      , 139, "Experimental use Host Identity Protocol") \
    _(SHIM6    , 140, "Shim6 Protocol")                          \
@@ -231,7 +251,6 @@ typedef struct {
  Also, Fragment header needs special processing.
 
    _(NONEXT   , 59 , "NoNextHdr")                               \
-   _(FRAGMENT , 44 , "IPv6ExtHdrFragment")                      \
 
 
 ESP is hiding its internal format, so no point in trying to go past it.

src/plugins/acl/acl_test.c

@@ -164,14 +164,14 @@ vl_api_acl_rule_t_pretty_format (u8 *out, vl_api_acl_rule_t * a)
   inet_ntop(af, a->src_ip_addr, (void *)src, sizeof(src));
   inet_ntop(af, a->dst_ip_addr, (void *)dst, sizeof(dst));
 
-  out = format(out, "%s action %d src %s/%d dst %s/%d proto %d sport %d-%d dport %d-%d tcpflags %d %d",
+  out = format(out, "%s action %d src %s/%d dst %s/%d proto %d sport %d-%d dport %d-%d tcpflags %d mask %d",
                      a->is_ipv6 ? "ipv6" : "ipv4", a->is_permit,
                      src, a->src_ip_prefix_len,
                      dst, a->dst_ip_prefix_len,
                      a->proto,
                      a->srcport_or_icmptype_first, a->srcport_or_icmptype_last,
 	             a->dstport_or_icmpcode_first, a->dstport_or_icmpcode_last,
-                     a->tcp_flags_mask, a->tcp_flags_value);
+                     a->tcp_flags_value, a->tcp_flags_mask);
   return(out);
 }
 
@@ -326,6 +326,7 @@ static int api_acl_add_replace (vat_main_t * vam)
     vl_api_acl_rule_t *rules = 0;
     int rule_idx = 0;
     int n_rules = 0;
+    int n_rules_override = -1;
     u32 proto = 0;
     u32 port1 = 0;
     u32 port2 = 0;
@@ -363,6 +364,10 @@ static int api_acl_add_replace (vat_main_t * vam)
             vec_validate_acl_rules(rules, rule_idx);
             rules[rule_idx].is_permit = 1;
           }
+        else if (unformat (i, "count %d", &n_rules_override))
+          {
+            /* we will use this later */
+          }
         else if (unformat (i, "action %d", &action))
           {
             vec_validate_acl_rules(rules, rule_idx);
@@ -430,6 +435,12 @@ static int api_acl_add_replace (vat_main_t * vam)
             rules[rule_idx].tcp_flags_value = tcpflags;
             rules[rule_idx].tcp_flags_mask = tcpmask;
           }
+        else if (unformat (i, "tcpflags %d mask %d", &tcpflags, &tcpmask))
+          {
+            vec_validate_acl_rules(rules, rule_idx);
+            rules[rule_idx].tcp_flags_value = tcpflags;
+            rules[rule_idx].tcp_flags_mask = tcpmask;
+          }
         else if (unformat (i, "proto %d", &proto))
           {
             vec_validate_acl_rules(rules, rule_idx);
@@ -455,6 +466,9 @@ static int api_acl_add_replace (vat_main_t * vam)
     else
       n_rules = 0;
 
+    if (n_rules_override >= 0)
+      n_rules = n_rules_override;
+
     msg_size += n_rules*sizeof(rules[0]);
 
     mp = vl_msg_api_alloc_as_if_client(msg_size);
@@ -812,6 +826,7 @@ static int api_macip_acl_add (vat_main_t * vam)
     vl_api_macip_acl_rule_t *rules = 0;
     int rule_idx = 0;
     int n_rules = 0;
+    int n_rules_override = -1;
     u32 src_prefix_length = 0;
     u32 action = 0;
     ip4_address_t src_v4address;
@@ -843,6 +858,10 @@ static int api_macip_acl_add (vat_main_t * vam)
             vec_validate_macip_acl_rules(rules, rule_idx);
             rules[rule_idx].is_permit = 0;
           }
+        else if (unformat (i, "count %d", &n_rules_override))
+          {
+            /* we will use this later */
+          }
         else if (unformat (i, "action %d", &action))
           {
             vec_validate_macip_acl_rules(rules, rule_idx);
@@ -856,6 +875,10 @@ static int api_macip_acl_add (vat_main_t * vam)
             rules[rule_idx].src_ip_prefix_len = src_prefix_length;
             rules[rule_idx].is_ipv6 = 0;
           }
+        else if (unformat (i, "src"))
+          {
+            /* Everything in MACIP is "source" but allow this verbosity */
+          }
         else if (unformat (i, "ip %U/%d",
          unformat_ip6_address, &src_v6address, &src_prefix_length))
           {
@@ -897,6 +920,9 @@ static int api_macip_acl_add (vat_main_t * vam)
     else
       n_rules = 0;
 
+    if (n_rules_override >= 0)
+      n_rules = n_rules_override;
+
     msg_size += n_rules*sizeof(rules[0]);
 
     mp = vl_msg_api_alloc_as_if_client(msg_size);