Commit Graph

75 Commits

Author SHA1 Message Date
Dave Barach
d84ba85c00 TCP horizontal scaling
- Remove frame handoff support machinery. We haven't used it in a long
  time.
- Configuration support for the local endpoints bihash table
- Drop lookup failure packets in tcp46_syn_sent

Change-Id: Icd51e6785f74661c741e76fac23d21c4cc998d17
Signed-off-by: Dave Barach <dave@barachs.net>
2017-08-25 22:04:28 +00:00
Damjan Marion
57d963f88b Make VPP runtime directory configurable
New startup config command:

unix {
	runtime-dir /run/vpp
}

Also, adds recursive mkdir funtion for use in deifferent places
like cli-config socket path and dpdk hugepage directory path.

Change-Id: I1446ceab9c220c25804e73a743a3ebb383450124
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-08-24 19:49:09 +00:00
Florin Coras
b2215d6b0d Fix tcp multi buffer segments retransmission
- Fix tcp/udp sw checksum computation
- Fix allocation of multi buffer tcp segments for retransmits
- Send FIN only if/when tx fifo is empty

Change-Id: I2e43a14b87a72c9e547b4339b9a51811cf5732c4
Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-08-11 16:03:19 +00:00
Florin Coras
66b11318a1 Fix tcp tx buffer allocation
- Make tcp output buffer allocation macro an inline function
- Use per ip version per thread tx frames for retransmits and timer
  events
- Fix / parameterize tcp data structure preallocation
- Add a couple of gdb-callable show commands
- Fix local endpoint cleanup

Change-Id: I67b47b7570aa14cb4634b6fd93c57cd2eacbfa29
Signed-off-by: Florin Coras <fcoras@cisco.com>
Signed-off-by: Dave Barach <dave@barachs.net>
2017-08-02 01:49:39 -07:00
Damjan Marion
6b0f589283 Thread safe internal buffer manager
Change-Id: I45845b952aa42a854e1c2c396b85f905de987020
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-27 10:50:34 +00:00
JingLiuZTE
30af5da752 VPP-905: Wrong define used in function start_workers.
Change-Id: I6a5faebb63e9360cebfcfb1bc3f3c0eb6b15e937
Signed-off-by: JingLiuZTE <liu.jing5@zte.com.cn>
2017-07-26 14:41:25 +00:00
Dave Wallace
d756b35032 Fix unlinking of /dev/shm files.
- api-segment prefix not used when unlinking shm files
- unlink root region on exit if no clients referenced
- stale reference to freed segment name
- don't add fake client to /db unless CLIB_DEBUG > 2
- turn off the gmond plugin
- clean up unused vars in vpp/api

Change-Id: I66451fcfd6ee64a12466c2d6c209050e3cdb74b7
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Signed-off-by: Dave Barach <dave@barachs.net>
2017-07-17 21:00:52 -04:00
Damjan Marion
072401e809 Introduce l{2,3,4}_hdr_offset fields in the buffer metadata
To save space in the first cacheline following is changed:

- total_length_not_including_first_buffer moved to the 2nd cacheline.
This field is used only when VLIB_BUFFER_TOTAL_LENGTH_VALID and
VLIB_BUFFER_NEXT_PRESENT are both set.

- free_list_index is now stored in 4bits inside flags, which
allows up to 16 free lists. In case we need more we can store index
in the 2nd cachelin

Change-Id: Ic8521350819391af470d31d3fa1013e67ecb7681
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-14 15:39:24 +00:00
Igor Mikhailov (imichail)
19e9d954bd VPP-895 multi-thread: fix vpp crash on show runtime
In multi-threaded model (e.g. 1 main and 1 worker threads),
after an ethernet interface is deleted (e.g. vhost-user interface),
'show runtime' command produces garbled output and sometimes
leads to vpp crash.

The reason is because vlib_node_rename() frees and reallocates node's
'n->name' vector, however the change is not propagated into copies
of the node on worker threads.

Change-Id: Ibf22422913b7f2df22f70f3b2fe8dafd34c1dd06
Signed-off-by: Igor Mikhailov (imichail) <imichail@cisco.com>
(cherry picked from commit 02989064e4)
2017-07-13 22:17:53 +00:00
Chris Luke
a2522f6fd5 dpdk: fix dpdk_buffer_pool_create name
- vnet_buffer_pool_create should probably be named
  dpdk_buffer_pool_create since that is what it does.
- Its prototype should also be in a DPDK plugin header, not in
  vlib/buffer_funcs.h, since the implementation is in the plugin
  and nobody else should be calling it.

Change-Id: I7ba259afa4b888bc94f3ad257305e286b41e7370
Signed-off-by: Chris Luke <chrisy@flirble.org>
2017-07-13 19:24:43 +00:00
Damjan Marion
04a7f05e91 vlib: store buffer memory information in the buffer_main
Currently, buffer index is calculated as a offset to the physmem
region shifted by log2_cacheline size.

When DPDK is used we "hack" physmem data with information taken from
dpdk mempool. This makes physmem code not usable with DPDK.

This change makes buffer memory start and size independent of physmem
basically allowing physmem to be used when DPDK plugin is loaded.

Change-Id: Ieb399d398f147583b9baab467152a352d58c9c31
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-10 19:35:57 +00:00
Damjan Marion
191a1a407e vlib: fix issues with PCI handling code
- PCI devices not properly discovered
- vlib_pci_bus_master_enable () not working

Change-Id: I7433ab1b19b890b8900635b43037b9a2017a1921
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-10 15:42:27 +02:00
Chris Luke
475674ee5a unix: mkdir VPP_RUN_DIR before opening a socket in it
Change https://gerrit.fd.io/r/#/c/7230/ added a Unix domain
CLI socket in the default startup.conf; however unless you
had previously run VPP with the DPDK plugin enabled the
directory that it is created in. /run/vpp, would not exist
and startup would fail. This directory is typically hosted
in a tmpfs ramdisk and is thus ephemeral.

This patch adds a function that attempts to mkdir VPP_RUN_DIR
and uses it in both the DPDK plugin and the CLI code if the
CLI socket is to be created in that directory.

Change-Id: Ibbf925819099dce2b5eb0fa238b9edca1036d6fd
Signed-off-by: Chris Luke <chrisy@flirble.org>
2017-07-06 13:26:23 +00:00
Chris Luke
7447d07719 Buffer name inconsistently used a cstring/vec (VPP-901)
Spotted in the output of CLI command "show buffers", the name field
sometimes had trailing garbage, the hall sign of a string not being
terminated. In this case it was being inconsistently used as a cstring
or a vec.

- CLI printf needs %v to print the vec srring
- vlib_buffer_create_free_list_helper tried to use
  clib_mem_is_heap_object() to detect a vec object, wheras it should
  use clib_mem_is_vec()

Change-Id: Ib8b242a0c5a18924b8af7e8e1432784eebcf572c
Signed-off-by: Chris Luke <chrisy@flirble.org>
2017-07-05 17:23:41 +00:00
Neale Ranns
bb620d74b2 VPP debug image with worker threads hit assert on adding IP route with traffic (VPP-892)
When stacking DPOs the VLIB graph is also updated to add the edge between the nodes, if this edge does not yet exist. This addition should be done with the workers stopped.

Change-Id: I327e4d7d26f0b23eb280f17e4619ff2093ff7940
Signed-off-by: Neale Ranns <nranns@cisco.com>
(cherry picked from commit c02bd03ddf)
2017-06-30 13:55:23 +00:00
Dave Barach
5c20a0131a switch vlib process model to tw_timer_template timer impl
Change-Id: I36bb47faea55a6fea7af7ee58d87d8f6dd28f93d
Signed-off-by: Dave Barach <dave@barachs.net>
2017-06-28 19:12:10 +00:00
Damjan Marion
f661638811 Add option to create clib_socket with group write permissions
Also allow group write as default for CLI socket connections.

Change-Id: I6af1f277f70581358cd9241bf0f5cb0752fe250f
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-06-21 15:22:40 +00:00
Damjan Marion
a54230d4e7 Add knob to specify effective group id (gid) for VPP process
Change-Id: Icf9bd4abda058fb380f1a25d5fe3917ffb38b1c4
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-06-21 15:22:06 +00:00
Florin Coras
93992a9048 Implement sack based tcp loss recovery (RFC 6675)
- refactor existing congestion control code (RFC 6582/5681). Handling of ack
  feedback now consists of: ack parsing, cc event detection, event handling,
  congestion control update
- extend sack scoreboard to support sack based retransmissions
- basic implementation of Eifel detection algorithm (RFC 3522) for
  detecting spurious retransmissions
- actually initialize the per-thread frame freelist hash tables
- increase worker stack size to 2mb
- fix session queue node out-of-buffer handling
  - ensure that the local buffer cache vec_len matches reality
  - avoid 2x spurious event requeues when short of buffers
  - count out-of-buffer events
- make the builtin server thread-safe
- fix bihash template threading issue: need to paint -1 across uninitialized
  working_copy_length vector elements (via rebase from master)

Change-Id: I646cb9f1add9a67d08f4a87badbcb117980ebfc4
Signed-off-by: Florin Coras <fcoras@cisco.com>
Signed-off-by: Dave Barach <dbarach@cisco.com>
2017-06-09 16:47:19 +00:00
Dave Barach
a62699954a VPP-873: fix vector expansion bug in dispatch_pending_node
The main interior graph-node dispatch loop had a longstanding dangling
vector element reference:

for (i = 0; i < _vec_len (nm->pending_frames); i++)
   cpu_time_now = dispatch_pending_node (vm, nm->pending_frames + i,
                               cpu_time_now);

Passing a pointer to a vector element (nm->pending_frames + i) has
considerable comedic potential if there's any chance that the vector
could expand.

dispatch_pending_node() calls dispatch_node(), and indirectly any
interior graph node dispatch function. If that node happens to expand
nm->pending_frames by filling in a new frame, nm->pending_frames can
expand.

After calling the node dispatch function, dispatch_node() does the
following:

  nf = vec_elt_at_index (nm->next_frames, p->next_frame_index);

If nm->pending_frames expands during dispatch function execution, p is
a dangling reference to freed memory.

By luck, the TCP stack managed to allocate a fresh frame which
included "old-p," which caused p->next_frame_index to be filled with
the new-frame poison pattern 0xfefefefe.

This has been broken from day 1, summer 2007, first use of the
third-generation vector processing library.

Change-Id: Ideb6363bb060c4e8bf9b901882c318bd83853121
Signed-off-by: Dave Barach <dave@barachs.net>
2017-06-07 13:35:04 +00:00
Damjan Marion
6e8ab16cf3 vlib: add screen-256color CLI terminal type
Change-Id: Ia78e69e5e8ed18020314aef321b94ac37037799b
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-06-06 00:35:56 +00:00
Damjan Marion
a9a54c15e6 vlib: add unix_file_del_by_index function
Change-Id: I9eca5f9d1c1ae62d5ba5fb36f2f97434dbaf334e
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-06-05 23:40:05 +00:00
Damjan Marion
a7f7457593 vlib: use driver_override in sysfs for binding pci devs to vfio/uio drivers
Change-Id: I262e455792fd95d286ee3ebc0049e2352ae5899f
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-05-24 19:42:12 +00:00
Yoann Desmouceaux
4227eefcd4 Fix tab-completion coverity issue
Change-Id: I051d015e7eee621dbef273b2c57449ea4c44b768
Signed-off-by: Yoann Desmouceaux <ydesmouc@cisco.com>
2017-05-24 16:24:23 +00:00
Yoann Desmouceaux
3060e07b71 Add TAB-based auto-completion to the CLI
Hitting tab:
 - in the middle of a uniquely defined subcommand will expand the subcommand
 - in the middle of a non-uniquely defined (or empty) subcommand will display
   all possible subcommands, and possibly expand to the lowest common prefix

Change-Id: Ib858eefdb0353cd2c3aad472799d15cd537455a0
Signed-off-by: Yoann Desmouceaux <ydesmouc@cisco.com>
2017-05-23 17:55:11 +00:00
Steven
f3b53643e8 vhost: migrate to use device infra for worker thread assignment, rx-mode.
and add adaptive mode support to receive queue

- Migrate vhost to use device infra which does the interface/queue to worker
thread assignment.

- Retire vhost thread CLI and corresponding code which assigns interface/queue
to worker thread. set interface placement should be used instead to customize
the interface/queue to worker thread assignment.

- Retire vhost interrupt/polling option when creating vhost-user interface.
Instead, set interface rx-mode should be used.

- Add code in vnet_device_input_unassign_thread to change the node state
to interrupt if the last polling interface has left the worker thread for the
device of the corresponding interface/queue.

- Add adaptive mode support. The node state is set to interrupt initially.
When the scheduler detects a burst of traffic, it switches the input node to
polling. Then we inform the device that we don't need interrupt notification.
When the traffic subsides, the scheduler switches the input node back to
interrupt. Then we immediately tell the driver that we want interrupt
notification again.

- Remove some duplicate code in vlib/main.c

Change-Id: Id19bb1b9e50e6521c6464f470f5825c26924d3a8
Signed-off-by: Steven <sluong@cisco.com>
2017-05-22 16:15:38 -07:00
Dave Barach
814813103b VPP-845: add configurable elog post-mortem dump
Off by default. Enable via cmdline "... vlib { elog-post-mortem-dump }
..."

Change-Id: I2056b9de9b37475f2bfeeb5404da838f1b42645a
Signed-off-by: Dave Barach <dave@barachs.net>
2017-05-16 09:09:48 -04:00
Damjan Marion
f55f9b851f completelly deprecate os_get_cpu_number, replace new occurences
Change-Id: I82c663bc0866c6c68ba354104b0bb059387f4b9d
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-05-10 22:01:15 +00:00
Damjan Marion
7bee80c823 Fix remaining 32-bit compile issues
Change-Id: I9664214652229b663c3e3ba7406b4ede96bfb123
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-05-09 17:52:43 +00:00
Damjan Marion
3c785e09a0 vlib: do not unbind devices already bound to vfio-pci
Change-Id: I739bed5b9d9504d18ee88206e29ebc4ba1b47d28
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-05-08 18:37:54 +02:00
Filip Tehlar
816f437d94 Fix vnet unit tests
Change-Id: Ibe55e4399c6b78d83268d7c49ed498cab7bfdb43
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2017-05-03 15:19:23 +00:00
Damjan Marion
c9fc77c510 Exit if plugin load attempt fail
This happens mainly if plugin contains unresolved symbol.
Such situation typically leads on crash a bit later so it is better
to exit immediatelly and display meaningful error message.

Change-Id: I4abd9a9089a4863400bf609e8d3fd7cebab92913
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-26 00:11:35 +00:00
Damjan Marion
36c1308b35 Fix structure alignment with 32-bit pointers
Change-Id: I740de6c0f12dab452b4349e3bf89ff976a6268c0
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-25 00:13:53 +00:00
Damjan Marion
9c2b9f413c vlib: add cli command to dump all CLI commands
To be used for bash completion

Change-Id: I8f4702f24c2b7e223945e00a1b3560dec6ef39fd
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-21 14:49:28 +00:00
Pierre Pfister
ecba903bcf Fix typo in minimal epoll polling time
epoll was supposed to not sleep when timeout
is less than 1ms, but a typo made it not sleep
any time the requested timeout is lower than
1000 seconds (in practice, never...).

This patch replaces "1e3" with "1e-3", which
represents 1ms.

Change-Id: I731851b27a6bf6ab8e41586e017e94b962b09bf3
Signed-off-by: Pierre Pfister <ppfister@cisco.com>
(cherry picked from commit ec06222ae1)
2017-04-11 09:50:34 +00:00
Damjan Marion
153646e89c Common device-input interrupt infra
Change-Id: I23b588eb56a3f5690158449a1f9bc8053cd3d251
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-10 12:47:33 +00:00
Damjan Marion
586afd762b Use thread local storage for thread index
This patch deprecates stack-based thread identification,
Also removes requirement that thread stacks are adjacent.

Finally, possibly annoying for some folks, it renames
all occurences of cpu_index and cpu_number with thread
index. Using word "cpu" is misleading here as thread can
be migrated ti different CPU, and also it is not related
to linux cpu index.

Change-Id: I68cdaf661e701d2336fc953dcb9978d10a70f7c1
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-06 11:31:39 +00:00
Damjan Marion
2c2b640712 vlib: make vlib_node_interrupt_pending(...) thread safe
Change-Id: I24577bd32ae23fbe8515cc8d960eab5448ce3b5c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-04 14:40:22 +00:00
Dave Barach
903fd513e3 Clean up event log merge code
Fix a decade-old ridiculous qsort function bug. Managed to subtract
floating-point numbers as if they were integers, leading to
manufactured time-paradoxes. That completely confuses g2, leading to
the summary disappearance of entire tracks' worth of data at high zoom
levels.

Add a manual alignment tweak parameter to elog_merge, users can
dial-out time paradoxes caused by NTP-grade clock synchronization.
The event-logger has a precision of O(100ns), whereas NTP
synchronization is O(1ms).

Change-Id: I69dedabaa314f69f9df74ec9ee66e21e6c87f703
Signed-off-by: Dave Barach <dave@barachs.net>
2017-04-01 17:22:31 +00:00
Damjan Marion
ce359db3b6 vlib: extend foreach_vlib_main macro to assert if workers are not parked
Change-Id: I6ff7b65a400734a47bc0a7d03faf86ef1cf4f8c8
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-31 12:21:47 +00:00
Damjan Marion
9a332e1639 vlib: inline dispatch_node(...) (again)
Worker main loop is now shared code with main thread
main loop so no need to export functions anymore.

Change-Id: I99ee2eee981c1b88ca31d20eabeb6c21d030a34d
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-28 14:26:07 +00:00
Damjan Marion
1bfb0ddace vlib: add description field in plugin registration
Change-Id: I88b322a5d602f3d6d3310e971479180a89430e0e
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-22 13:43:14 +00:00
Steven
7312cc7785 vhost: support interrupt mode
vhost currently supports only polling mode. This patch is to add
interrupt mode. When the interface is configured for interrupt
mode, our input node does not get called unless there is a packet
in the vring.

If a particular CPU has one interface configured for polling mode
and another in interrupt, the input node is set to polling for
that CPU.

This diffs also includes two crashes in vlib's dispatch_node. One is
included in https://gerrit.fd.io/r/#/c/5516. The other crash is in
the ASSERT. The ASSERT can become true when the caller of
dispatch_node is in a loop. The first call converted the node
to polling. The second call thereafter will hit the ASSERT.

Change-Id: If17b6d48b20d7d8605c6a161459828637173cd32
Signed-off-by: Steven <sluong@cisco.com>
2017-03-22 08:54:33 +00:00
Damjan Marion
eb743fad56 vnet: add device-input threadplacement infra
This change adds two new debug CLI command:

- "show interface placmenet" to display which
thread (main or worker) is responsible for processing
interface rx queue

vpp# show interface placement
Thread 0 (vpp_main):
  node af-packet-input:
    host-vpp1 queue 0
Thread 1 (vpp_wk_0):
  node af-packet-input:
    host-virbr0 queue 0
Thread 2 (vpp_wk_1):
  node af-packet-input:
    host-vpp2 queue 0
    host-lxcbr0 queue 0

- "set interface placmenet" to assign thread (main or worker)
which process specific interface rx queue

vpp# set interface placement host-vpp1 queue 0 main

Change-Id: Id4dd00cf2b05e10fae2125ac7cb4411b446c5e9c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-20 16:34:15 +01:00
Neale Ranns
1bd01099a6 64 bit per-thread counters
after:
TenGigabitEthernet5/0/1-output   active             107522        17375708               0          7.22e0          161.60
TenGigabitEthernet5/0/1-tx       active             107522        17375708               0          6.93e1          161.60
ip4-input-no-checksum            active             107522        17375708               0          2.52e1          161.60
ip4-lookup                       active             107522        17375708               0          3.10e1          161.60
ip4-rewrite                      active             107522        17375708               0          2.52e1          161.60

before
TenGigabitEthernet5/0/1-output   active             433575       110995200               0          6.95e0          256.00
TenGigabitEthernet5/0/1-tx       active             433575       110995200               0          7.14e1          256.00
ip4-input-no-checksum            active             433575       110995200               0          2.66e1          256.00
ip4-lookup                       active             433575       110995200               0          3.29e1          256.00
ip4-rewrite                      active             433575       110995200               0          2.59e1          256.00

Change-Id: I46405bd22189f48a39f06e3443bb7e13f410b539
Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-17 19:45:36 +00:00
Steven
6aa75af241 vlib: fix potential crash in dispatch_node ELOG_DATA call
dispatch_node may be invoked from vlib main or worker threads. The call to
ELOG_DATA in dispatch_node passes the parameter &vm->elog_main. It works fine
when dispatch_node is invoked from the main thread. It does bad thing when it
is invoked from the worker thread.

While we are at it, make two additional enhancements to the same area.
1. Use ELOG_TRACK_DATA instead of ELOG_DATA to enhance g2 viewer presentation.
2. Since ELOG_DATA is in the data path, it could get very chatty. Make the call
to ELOG_TRACK_DATA conditional compile.

Change-Id: I80ca0eea10bc1e5d0d5549f9844dd9a34dbb65a2
Signed-off-by: Steven <sluong@cisco.com>
2017-03-17 12:57:53 +00:00
Damjan Marion
b6f93a1d1a vlib: additional runtime_data checks
Change-Id: I9b6ed9741fae89bdefa6f601398eb63a21155069
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-16 20:54:16 +00:00
Damjan Marion
e9f929b52d vlib: make runtime_data thread-local
Change-Id: I4aa3e7e42fb81211de1aed07dc7befee87a1e18b
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-16 13:06:27 +00:00
Neale Ranns
9d676afbb7 No vector allocation during buffer copy
Change-Id: I7e8556af833ca0e00fadc96dcd2077ff1104541b
Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-15 16:19:11 +00:00
Damjan Marion
20e272c8fc vlib: poll pre_input nodes only on main thread
Change-Id: I61464fd1610a9754693f31edd72f9fa1b6926511
Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-14 12:03:07 +00:00