It turns out that for scalar sizes 0..24, frames are always the same
size. That range includes all current use-cases - and then some - so
get rid of the hash table. Old code preserved under #ifdef
VLIB_SUPPORTS_ARBITRARY_SCALAR_SIZES.
Change-Id: Ic005c7143c9639f77d1a0fadd2fc0e90dccb68c1
Signed-off-by: Dave Barach <dbarach@cisco.com>
Add a new command to dump vlib graph as graphviz/dot file
Change-Id: I43fc072cff8153ac500e5fbc6641a3705c2e995e
Signed-off-by: Benoît Ganne <bganne@cisco.com>
This patch introduces following changes:
- deprecated free lists which are not used and not compatible
with external buffer managers (i.e. DPDK)
- introduces native support for per-numa buffer pools
- significantly improves performance of buffer alloc and free
Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7
Signed-off-by: Damjan Marion <damarion@cisco.com>
Add missing pre-input node runtime fork and refork code.
unix-epoll-input runs on all threads; each instance needs its own
runtime stats.
Change-Id: I16b02e42d0c95f863161176c4bb9f9917bef809d
Signed-off-by: Dave Barach <dave@barachs.net>
As a FUD reduction measure, this patch implements 2-way parallel
counter collection. Synthetic stat component counter pairs run at the
same time. Running two counters (of any kind) at the same time
naturally reduces the aggregate time required by an approximate
factor-of-2, depending on whether an even or odd number of stats have
been requested.
I don't completely buy the argument that computing synthetic stats
such as instructions-per-clock will be inaccurate if component counter
values are collected sequentially. Given uniform traffic pattern, it
must make no difference.
As the collection interval increases, the difference between serial
and parallel component counter collection will approach zero, see also
the Central Limit theorem.
Change-Id: I36ebdcf125e8882cca8a1929ec58f17fba1ad8f1
Signed-off-by: Dave Barach <dave@barachs.net>
VPP graph dispatch trace record description:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Major Version | Minor Version | NStrings | ProtoHint |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Buffer index (big endian) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ VPP graph node name ... ... | NULL octet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Buffer Metadata ... ... | NULL octet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Buffer Opaque ... ... | NULL octet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Buffer Opaque 2 ... ... | NULL octet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| VPP ASCII packet trace (if NStrings > 4) | NULL octet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet data (up to 16K) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Graph dispatch records comprise a version stamp, an indication of how
many NULL-terminated strings will follow the record header, and a
protocol hint.
The buffer index allows downstream consumers of these data to easily
filter/track single packets as they traverse the forwarding
graph. FWIW, the 32-bit buffer index is stored in big endian format.
As of this writing, major version = 1, minor version = 0. Nstrings
will be either 4 or 5.
Here is the current set of protocol hints:
typedef enum
{
VLIB_NODE_PROTO_HINT_NONE = 0,
VLIB_NODE_PROTO_HINT_ETHERNET,
VLIB_NODE_PROTO_HINT_IP4,
VLIB_NODE_PROTO_HINT_IP6,
VLIB_NODE_PROTO_HINT_TCP,
VLIB_NODE_PROTO_HINT_UDP,
VLIB_NODE_N_PROTO_HINTS,
} vlib_node_proto_hint_t;
Example: VLIB_NODE_PROTO_HINT_IP6 means that the first octet of packet
data SHOULD be 0x60, and should begin an ipv6 packet header.
Change-Id: Idf310bad80cc0e4207394c80f18db5f77c378741
Signed-off-by: Dave Barach <dave@barachs.net>
To facilitate dispatch trajectory tracing, vlib_buffer_t decoding, etc.
through Wireshark
Change-Id: I31356b9fa1f40cba8830aaf10a86a9fbb7546438
Signed-off-by: Dave Barach <dave@barachs.net>
Typically we have scalar_size == 0, so it doesn't matter
but vlib_frame_args was providing pointer to scalar frame
data, not vector data. To avoid future confusion function
is renamed to vlib_frame_scalar_args(...)
Change-Id: I48b75523b46d487feea24f3f3cb10c528dde516f
Signed-off-by: Damjan Marion <damarion@cisco.com>
Those flags have local significance and they can be used for
sending hints to the next node.
Change-Id: Ic2596ee81c64cd16f96344365370e8fcdc465354
Signed-off-by: Damjan Marion <damarion@cisco.com>
Reduces the chance of tripping over vectors in flight, especially in
single-core cases.
Change-Id: I132cdd3689f8e634f9a983af57219503817b8560
Signed-off-by: Dave Barach <dave@barachs.net>
It should be possible to use vlib without the vlibmemory library, etc.
Change-Id: Ic2316b93d7dbb728fb4ff42a3ca8b0d747c9425e
Signed-off-by: Dave Barach <dave@barachs.net>
Add a session process node that handles main thread tx and retransmit in
order to avoid having a polling input node.
Change-Id: I3357e987c023a84b533b32793e37ab4204420f64
Signed-off-by: Florin Coras <fcoras@cisco.com>
- buffer_main is no longer part of vlib_main_t
- pool of free lists is still part of vlib_main_t
- mheap is not used anymore for buffer allocation
- simple bitmap bassed buffer alloc scheme is introduced
Change-Id: I3e1e6d00e2c8122293ed0a741245eb841315a1ff
Signed-off-by: Damjan Marion <damarion@cisco.com>
This patch teaches worer threads to sleep and to be waken up by
kernel if there is activity on file desctiptors assigned to that thread.
It also adds counters to epoll file descriptors and new
debug cli 'show unix file'.
Change-Id: Iaf67869f4aa88ff5b0a08982e1c08474013107c4
Signed-off-by: Damjan Marion <damarion@cisco.com>
Queue RPC calls and send them from the main dispatch loop. As things stood,
if the vpp main input queue filled, worker threads could enter a
barrier-sync spin-wait in the middle of processing a frame. If thread
0 decided to recreate worker thread data structures, the worker thread(s)
could easily crash.
Legislate the problem out of existence by enqueueing RPC messages only
from the main dispatch loop. At that point, doing a barrier-sync wait
is perfectly OK.
Change-Id: I18da3e44bb1f29a63fe5f30cf11de732ecfd5bf7
Signed-off-by: Dave Barach <dave@barachs.net>
Use a proper u16 * vector to capture node indices, since vpp w/
plugins now exceeds 255 graph nodes
Change-Id: Ic48cad676fa3a6116413ddf08c083dd9660783f1
Signed-off-by: Dave Barach <dave@barachs.net>
This patch adds supprot support for multiple numa-aware physmem regions.
Change-Id: I5c69a6f4da33c8ee21bdb8604d52fd2886f2327e
Signed-off-by: Damjan Marion <damarion@cisco.com>
- Remove frame handoff support machinery. We haven't used it in a long
time.
- Configuration support for the local endpoints bihash table
- Drop lookup failure packets in tcp46_syn_sent
Change-Id: Icd51e6785f74661c741e76fac23d21c4cc998d17
Signed-off-by: Dave Barach <dave@barachs.net>
Currently, buffer index is calculated as a offset to the physmem
region shifted by log2_cacheline size.
When DPDK is used we "hack" physmem data with information taken from
dpdk mempool. This makes physmem code not usable with DPDK.
This change makes buffer memory start and size independent of physmem
basically allowing physmem to be used when DPDK plugin is loaded.
Change-Id: Ieb399d398f147583b9baab467152a352d58c9c31
Signed-off-by: Damjan Marion <damarion@cisco.com>
The main interior graph-node dispatch loop had a longstanding dangling
vector element reference:
for (i = 0; i < _vec_len (nm->pending_frames); i++)
cpu_time_now = dispatch_pending_node (vm, nm->pending_frames + i,
cpu_time_now);
Passing a pointer to a vector element (nm->pending_frames + i) has
considerable comedic potential if there's any chance that the vector
could expand.
dispatch_pending_node() calls dispatch_node(), and indirectly any
interior graph node dispatch function. If that node happens to expand
nm->pending_frames by filling in a new frame, nm->pending_frames can
expand.
After calling the node dispatch function, dispatch_node() does the
following:
nf = vec_elt_at_index (nm->next_frames, p->next_frame_index);
If nm->pending_frames expands during dispatch function execution, p is
a dangling reference to freed memory.
By luck, the TCP stack managed to allocate a fresh frame which
included "old-p," which caused p->next_frame_index to be filled with
the new-frame poison pattern 0xfefefefe.
This has been broken from day 1, summer 2007, first use of the
third-generation vector processing library.
Change-Id: Ideb6363bb060c4e8bf9b901882c318bd83853121
Signed-off-by: Dave Barach <dave@barachs.net>
and add adaptive mode support to receive queue
- Migrate vhost to use device infra which does the interface/queue to worker
thread assignment.
- Retire vhost thread CLI and corresponding code which assigns interface/queue
to worker thread. set interface placement should be used instead to customize
the interface/queue to worker thread assignment.
- Retire vhost interrupt/polling option when creating vhost-user interface.
Instead, set interface rx-mode should be used.
- Add code in vnet_device_input_unassign_thread to change the node state
to interrupt if the last polling interface has left the worker thread for the
device of the corresponding interface/queue.
- Add adaptive mode support. The node state is set to interrupt initially.
When the scheduler detects a burst of traffic, it switches the input node to
polling. Then we inform the device that we don't need interrupt notification.
When the traffic subsides, the scheduler switches the input node back to
interrupt. Then we immediately tell the driver that we want interrupt
notification again.
- Remove some duplicate code in vlib/main.c
Change-Id: Id19bb1b9e50e6521c6464f470f5825c26924d3a8
Signed-off-by: Steven <sluong@cisco.com>
Off by default. Enable via cmdline "... vlib { elog-post-mortem-dump }
..."
Change-Id: I2056b9de9b37475f2bfeeb5404da838f1b42645a
Signed-off-by: Dave Barach <dave@barachs.net>
This patch deprecates stack-based thread identification,
Also removes requirement that thread stacks are adjacent.
Finally, possibly annoying for some folks, it renames
all occurences of cpu_index and cpu_number with thread
index. Using word "cpu" is misleading here as thread can
be migrated ti different CPU, and also it is not related
to linux cpu index.
Change-Id: I68cdaf661e701d2336fc953dcb9978d10a70f7c1
Signed-off-by: Damjan Marion <damarion@cisco.com>
Fix a decade-old ridiculous qsort function bug. Managed to subtract
floating-point numbers as if they were integers, leading to
manufactured time-paradoxes. That completely confuses g2, leading to
the summary disappearance of entire tracks' worth of data at high zoom
levels.
Add a manual alignment tweak parameter to elog_merge, users can
dial-out time paradoxes caused by NTP-grade clock synchronization.
The event-logger has a precision of O(100ns), whereas NTP
synchronization is O(1ms).
Change-Id: I69dedabaa314f69f9df74ec9ee66e21e6c87f703
Signed-off-by: Dave Barach <dave@barachs.net>
Worker main loop is now shared code with main thread
main loop so no need to export functions anymore.
Change-Id: I99ee2eee981c1b88ca31d20eabeb6c21d030a34d
Signed-off-by: Damjan Marion <damarion@cisco.com>
vhost currently supports only polling mode. This patch is to add
interrupt mode. When the interface is configured for interrupt
mode, our input node does not get called unless there is a packet
in the vring.
If a particular CPU has one interface configured for polling mode
and another in interrupt, the input node is set to polling for
that CPU.
This diffs also includes two crashes in vlib's dispatch_node. One is
included in https://gerrit.fd.io/r/#/c/5516. The other crash is in
the ASSERT. The ASSERT can become true when the caller of
dispatch_node is in a loop. The first call converted the node
to polling. The second call thereafter will hit the ASSERT.
Change-Id: If17b6d48b20d7d8605c6a161459828637173cd32
Signed-off-by: Steven <sluong@cisco.com>
