793 Commits

Author SHA1 Message Date
dfa77dc606 vlib: process node scheduler rework
This commit allow use od cooperative multitasking with multiple
descheduling reasons (i.e. event wait and suspend) inside the same
process node. In previus code remote node will wake up process node
by sending event evein if process node is waiting in
vlib_process_ssupend().

This change also allowed new vlib_process_yield() API which deschedules
current process and it puts it into the end of queue.

Change-Id: I846e5a99b4ea1809eb80895f6ffe0ef0b2fd21ae
Type: improvement
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-09-26 08:33:47 +00:00
3b5a013103 vlib: introduce lazy next node initialization
This change allow a node registration A to name a next node B that does not
exist yet at registration time. When node B is effectively created,
vlib_node_main_lazy_next_update need to be called so that the vlib graph
is updated accordingly. To enable this behavior, node A needs to bear
the new VLIB_NODE_FLAG_ALLOW_LAZY_NEXT_NODES.

Change-Id: I561d3a0de19a0b7bd1045760a2ba8e27d27caa9a
Type: improvement
Signed-off-by: Mohammed Hawari <mohammed@hawari.fr>
2024-09-12 10:11:30 +00:00
a8766ce2d3 vlib: fix gcc 13.2.0 build failure on ubuntu 24.04
Type: fix

Change-Id: I007d73d28633677395d4eb81534dbe2307b1a96f
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2024-09-11 14:34:34 +00:00
b4808de247 buffers: support disabling allocation per numa domain
Currently, the per-numa buffer allocation doesn't support skipping the
allocation altogether for a particular NUMA domain - if it's set to
zero, it instead uses the default allocation.

Therefore, this changes the behavior so that the default is only used
if no per-numa allocation is specified, and an allocation of zero
is honored by not allocating a buffer at all.

Type: improvement
Change-Id: Ie9133b179a88eb884aa3da921e65292daea0df2d
Signed-off-by: Lukas Stockner <lstockner@genesiscloud.com>
2024-08-29 08:40:34 +00:00
26a6b19a0e vlib: add 'exit' as alias to 'quit'
Type: improvement
Change-Id: I099739a246dbdb6fdca136d06bc8e4b3f493a839
Signed-off-by: Maxime Peim <mpeim@cisco.com>
2024-08-06 13:38:27 +00:00
7f163b682a misc: replace index() with strchr()
Type: improvement
Change-Id: I471e514ebef0b4c1f86067115b2ebe5a5517c6fb
Signed-off-by: Matus Fabian <matfabia@cisco.com>
2024-07-22 17:07:02 +00:00
7561c45bcc vlib: add "save memory-trace" debug CLI
Save memory traces of the currently traced heap in JSON format to file
which can be used as machine-readable data for memory leak diagnose.

Type: improvement

Change-Id: I277f5be5838510e907c4dd7a8a4e9a883cb67bc3
Signed-off-by: Matus Fabian <matfabia@cisco.com>
2024-07-22 15:50:22 +00:00
fa500e9854 vlib: fix gdb_show_traces()
When using gdb_show_traces() in debug mode, don't assert if workers are
not parked, as it is typically called from gdb.

Type: fix

Change-Id: Iabf175d96dc152da4d1abfbce9ccc9020d0b5d61
Signed-off-by: Benoît Ganne <bganne@cisco.com>
2024-06-10 13:12:52 +00:00
63ff7a6325 buffers: support per-numa-domain buffer size
Currently, buffers-per-numa is used for all NUMA domains, and there's
no way to allocate different amounts for each domain.

Therefore, this adds a per-NUMA-domain buffer config section, which
currently has a single option to set the number of buffers.

If it's not specified or set to zero, the code falls back to
buffers-per-numa as before.

Type: improvement
Change-Id: If35d7a9eff6f8d1d78063ea7873dbf50780d0ec3
Signed-off-by: Lukas Stockner <lstockner@genesiscloud.com>
2024-06-05 12:28:27 +00:00
9d5be6ae2a vlib: clean up r2 plugin registration relocator
Thanks, Eliot!

Type: fix
Change-Id: I3737f62d08d5cd2db803af86285f9a2e456bab72
Signed-off-by: Dave Barach <vpp@barachs.net>
2024-06-01 08:51:14 -04:00
78925604e9 vlib: stack trace and signal handler improvements
- use libunwrap which seems to be industry standard
 - display traceback on console if running interactive or with syslog
   disabled (color output unless nocolor specified)
 - print hexdump of offending code
 - print library filename for each stack frame

Type: improvement
Change-Id: I61d3056251b87076be0578ccda300aa311c222ef
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-05-29 09:50:18 +00:00
0b0468cb0f vlib: improvement to automatic core pinning
Type: feature

If 'main-core' is not specified, the main thread is pinned
by default to the cpu it is running on during initialization.
This change does not impact manual core-pinning, which
requires the 'main-core' argument
e.g. 'cpu {main-core x corelist-workers n}.

Change-Id: I874034591bf0acf4d71b231dfbbb0f6de8fe6060
Signed-off-by: hsandid <halsandi@cisco.com>
2024-05-24 08:21:14 +00:00
5444973bd0 vlib: prevent some signals from being executed on workers
Before this commit, SIGINT, SIGHUP and SIGTERM could be executed on the
workers. Since those signals don't stop execution, it meant that atexit
handlers (like the `vl_unmap_shmem`) could run while the main thread was
still running, which can cause race conditions. To avoid that, we
prevent workers from handling those signals.

Type: fix
Signed-off-by: Guillaume Solignac <gsoligna@cisco.com>
Change-Id: I27a87d96a027d7423ced881a614427af4ab0f969
2024-05-22 12:52:31 +00:00
05c625f1f6 vlib: avoid pci scan without registrations
Type: improvement
Signed-off-by: Vladimir Zhigulin <scripath96@gmail.com>
Change-Id: Id1d78a36cec10a01fbf266b016ba364519e5cb66
2024-05-13 15:03:18 +00:00
ee98c9dd8e vlib: fix use of RTLD_DEEPBIND for musl
RTLD_DEEPBIND only exists in glibc. If it isn't defined we can just
ignore it.

Type: fix
Change-Id: I5b2d65b2462020808635b057d5f7665fb75a2444
Signed-off-by: Pierre Pfister <ppfister@cisco.com>
Signed-off-by: Eric Sun <esun@meraki.com>
2024-05-06 11:55:39 +00:00
687707d0b8 vlib: fix missing integer init
Some compilers were not clever enough to figure out these inits
weren't needed. Hence adding them.

Type: fix
Signed-off-by: Guillaume Solignac <gsoligna@cisco.com>
Change-Id: I50c4e978bfd2ef15ece7a29e517bb8ddba12f960
Signed-off-by: Pierre Pfister <ppfister@cisco.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2024-05-06 11:53:32 +00:00
4aecd4869c vlib: revert automatic core pinning changes
This reverts commit 71c32a8989.

Type: fix

Reason for revert: vnet pinning is not considered in this patch.
This causes keywords 'workers' and 'skip-cores' to be broken,
as well as keyword 'main-core auto' introduced in this patch.
If this patch is ever reconsidered, it must account for vnet
pinning fix in commit https://gerrit.fd.io/r/c/vpp/+/40711.

Change-Id: I1f3154a6c7e830b100f824375aa00e95b192f7f3
Signed-off-by: hsandid <halsandi@cisco.com>
2024-05-06 11:52:35 +00:00
e625d0e7af vlib: Add FreeBSD specific platform files
Type: improvement
Change-Id: I0739959084d7cfd8309793f54c92c046b94d7a02
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-05-06 11:38:56 +00:00
52bd5376e4 pci: fix missing limits.h
When building with musl, PATH_MAX is not available if we don't add its header
to pci.c.

Type: fix
Signed-off-by: Guillaume Solignac <gsoligna@cisco.com>
Change-Id: I3d4e7cf5754d7c1d480834d7cd038d5bda46fd2f
Signed-off-by: Pierre Pfister <ppfister@cisco.com>
2024-04-29 08:35:20 +00:00
d346b8852a vlib: Add FreeBSD thread specific header and calls
Type: improvement
Change-Id: Ia84683936da02968991ea717316a834a5da1a517
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-04-24 16:40:24 +00:00
b2a357bc1f vlib: Add a skeleton pci interface for FreeBSD
Implement a subset of the pci interface with unimplemented methods
marked with a comment. This is enough functionality to allow dpdk to
attach to pci devices and forward traffic on FreeBSD.

Type: improvement
Change-Id: I0d87778cd92ae332a6802c11083d3d99988678a1
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-04-24 12:31:24 +00:00
ab47993a00 vppinfra: Add method for getting current executable name
Add a unix method for getting the current executable name. This is
implemented to match the readlink api for existing calls.

Type: improvement
Change-Id: Id06a55892d09d0b305a56b55a424f53ffb685a72
Signed-off-by: Tom Jones <thj@freebsd.org>
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-04-24 10:30:20 +00:00
36d099a5cd vlib: Place linux pci headers in a linux include block
Type: improvement
Change-Id: I6b30ffd1eabfa9544d548082b7e38b8fe7224c07
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-04-22 19:09:13 +00:00
50e7ea7b14 vlib: fix vlib_buffer_free_inline() AVX-512 overflow
'queue' is declared on the stack and must be big enough to handle
queue_size + max processed elements per loops.
When AVX-512 was added, this additional max was not increased from 4 to
8 accordingly.

Type: fix
Fixes: 767a9ab4b1

Change-Id: I15f2f930fc316f92da2e4642f41852d856c06a48
Signed-off-by: Benoît Ganne <bganne@cisco.com>
2024-04-10 14:04:51 +00:00
eb2a4189f5 vlib: fix counter_index check
Ensure counter index is valid before using it to lookup
the counter.

Type: fix
Change-Id: I423c7a6aa6b65f6367b18d8e99cf40f52e06b416
Signed-off-by: lijinhui <lijh_7@chinatelecom.cn>
2024-04-01 17:57:58 +00:00
71c32a8989 vlib: improve automatic core pinning
Type: feature

Auto core pinning now fetches vpp cpu affinity list
using pthread api. This enables us to do core-pinning in
environments where the host cpu list does not necessarily align
with cpus available to vpp

Change-Id: Ife8c2a2351c08c5c6c4fdf7c729eeff2697bc39a
Signed-off-by: hsandid <halsandi@cisco.com>
2024-03-29 16:29:44 +00:00
3eb6cbec50 vppinfra: os agnostic api for getting CPU information
Avoid direct sysfs reads when possible...

Type: improvement
Change-Id: I2b84cd18f3da47925d068951f24b79b5b6e20bb1
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-03-18 16:01:35 +00:00
c3148b1be8 misc: remove GNU Indent directives
Type: refactor
Change-Id: I5235bf3e9aff58af6ba2c14e8c6529c4fc9ec86c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-03-12 19:29:56 +00:00
3026ffa091 vlib: fix initial stats time for the process nodes
Uninitialized stats time for new dynamic processes (like unix
cli) leads to invalid and too high clocks diff, breaking the
stats collecting.

  $ sudo vppctl sh runtime | grep unix
  unix-cli-new-session  any wait      0  0  30   2.72e3  0.00
  unix-cli-process-0      active     14  0  27  3.73e15  0.00
  unix-epoll-input       polling   6464  0   0   3.56e6  0.00
  unix-epoll-input       polling  68360  0   0   1.31e3  0.00

Type: fix
Signed-off-by: Vladislav Grishenko <themiron@yandex-team.ru>
Change-Id: Ic70678b67a3fb3f78ed4a1a03077ae243ed5d7cd
2024-03-05 18:49:49 +00:00
4b6614030f vppinfra: add os_get_online_cpu_core() and os_get_online_cpu_node()
Type: improvement
Change-Id: I6f99f09c7724ce656a4f41a1d5f9c88d74c00faf
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-03-04 08:53:08 +00:00
5ed27ef8b6 build: Discover libepoll-shim on FreeBSD
FreeBSD doesn't offer epoll, but an implementation which uses kqueue is
available as an external library. On FreeBSD in subsystems which require
epoll have cmake look for libepoll-shim.

Type: improvement
Change-Id: Iafd5406a9e2ebaa53fd94034489ffbbf87a7d040
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-02-23 10:07:49 +00:00
72cb9f7e44 vlib: Use platform specific headers for sched.h
FreeBSD only defines posix scheduling policies, remove the others from
the for each look to allow build.

Type: improvement
Change-Id: Ifdb9414417e8b6ffdf216fd001708b347c496b97
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-02-23 10:02:34 +00:00
532a1ce98a build: Add support to add_vpp_library for platform specific sources
Add support for adding Linux and FreeBSD specific sources to
add_vpp_library and support for installing Linux specific headers. Don't
add support for FreeBSD specific headers until we have some to install.

Type:	improvement
Change-Id: I38549cf4d71999b71b3298e529323956e54ddc36
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-02-23 10:01:47 +00:00
aaa2a89a3d vlib: Only init vfio on Linux
Type: improvement
Change-Id: Id1c72928324b85bc67860e600722bf8baeff45b2
Signed-off-by: Tom Jones <thj@freebsd.org>
2024-02-18 16:02:27 +00:00
04fd51c03c buffers: bring back cache occupancy improvement
The improvement was removed in 40129,
causing 5-40% regressions in AVF tests.

There is a memory-speed trade-off,
this change prefers speed over memory efficiency.

Ideally, the choice should be configurable,
but that is not easy to achieve, considering
how early is vlib_buffer_main_init called.

Type: fix
Fixes: 038dad7ef2

Change-Id: I4746f3634abe6d233c9d092a372de05b3d1ae4b6
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2024-02-12 16:20:31 +00:00
e060b0a079 vlib: flush rpcs on worker sync
Type: fix

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I89624254649089e497d119eb6cee2cecf3dc11f3
2024-02-02 17:46:44 +00:00
4cadd3b15b vlib api: move wrkr rpc flushing to vlib
Move vlib worker rpc flushing to main worker from memclnt api into vlib.
RPCs are no longer delivered via binary api queues

Type: refactor

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I5b8761a57d1f4a1f2220e860cfb2e6b980f46f6d
2024-02-02 14:44:26 +00:00
038dad7ef2 buffers: add compile-time option to set buffer alignment
Type: improvement
Change-Id: I88c4c45bed0bdd8686e17e4f77a7d32a08c995aa
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-01-20 18:28:09 +00:00
1cfac4e6e4 vlib: remove unused code
Type: improvement
Change-Id: I2c54cb3eb41ed35f114eca75fb6e2be3eb3f4ca1
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-01-16 17:40:56 +00:00
e1489260ca vlib: lowercase vmbus device names
Use lowercased hex in uuid in sysfs/vmbus names

Type: fix

Change-Id: Ic4b7b995441723dc1b29ff8a75346cc8ba7f85f1
Signed-off-by: Alexander Skorichenko <askorichenko@netgate.com>
2023-12-22 15:46:51 +00:00
832342e3a4 vlib: add error checks to thread pinning
Type: fix

Added error checks around pthread_setaffinity_np
calls to stop vpp launch if pinning fails.

Change-Id: Iec391c485d1832b6c2ff20fbf789608f6bcf7b57
Signed-off-by: hsandid <halsandi@cisco.com>
2023-12-22 15:45:07 +00:00
82a68e91d0 buffers: add vlib_buffer_unalloc_to_pool()
Change-Id: Icd616989a525ef0be6c73a2ad91ae4871cf3921d
Type: improvement
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-12-21 20:23:41 +00:00
bc67e9fd20 bpf_trace_filter: allow use whithout classifier
Change-Id: I7ac5693ca547fe7249e7b6297bade70a6052b169
Type: improvement
Signed-off-by: Mohammed Hawari <mohammed@hawari.fr>
2023-12-14 16:29:53 +00:00
2d725c6128 ena: Amazon Elastic Network Adapter (ENA) native driver
Type: feature
Change-Id: Icd9de05f2cbac0e5a6dfb1f1414f21dc4b893104
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-11-13 19:05:17 +00:00
76d14b797c misc: silence -Wmaybe-uninitialized warnings
GCC 12 complains about such errors while the code itself looks good.

Type: fix
Signed-off-by: Jieqiang Wang <jieqiang.wang@arm.com>
Change-Id: I021719fdbf7d9bd93a12eac76aeac8cbca13a810
2023-11-07 10:58:50 +00:00
7444fd2a0d pci: remove unnecessary ioctl() call and improve logging in vfio_set_irqs
Type: improvement
Change-Id: Ic8e2785bf375882defe5a1d299948d522cdd4895
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-11-06 10:15:12 +00:00
60529a8ef8 pci: fix VFIO set IRQ code
Type: fix
Fixes: 599a16b
Change-Id: I954e037ab944028798f9aa2a93f45322c8c7b4bb
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-11-06 00:09:23 +00:00
7f75e80f08 vppinfra: refactor interrupt code
Type: improvement
Change-Id: Ie6987736faf7d8a641762e276775da8ee0c03ea4
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-11-03 22:56:29 +00:00
38c619115b dev: new device driver infra
Type: feature
Change-Id: I20c56e0d3103624407f18365c2bc1273dea5c199
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-11-02 13:41:32 +00:00
55a1f288ed buffers: buffer name is vector, not c string
Type: fix
Fixes: 8973b07
Change-Id: Iba6e29bab35be0148b469965c3af8391b5ab9140
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-25 11:51:30 +00:00