83 Commits

Author SHA1 Message Date
a8766ce2d3 vlib: fix gcc 13.2.0 build failure on ubuntu 24.04
Type: fix

Change-Id: I007d73d28633677395d4eb81534dbe2307b1a96f
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2024-09-11 14:34:34 +00:00
b4808de247 buffers: support disabling allocation per numa domain
Currently, the per-numa buffer allocation doesn't support skipping the
allocation altogether for a particular NUMA domain - if it's set to
zero, it instead uses the default allocation.

Therefore, this changes the behavior so that the default is only used
if no per-numa allocation is specified, and an allocation of zero
is honored by not allocating a buffer at all.

Type: improvement
Change-Id: Ie9133b179a88eb884aa3da921e65292daea0df2d
Signed-off-by: Lukas Stockner <lstockner@genesiscloud.com>
2024-08-29 08:40:34 +00:00
63ff7a6325 buffers: support per-numa-domain buffer size
Currently, buffers-per-numa is used for all NUMA domains, and there's
no way to allocate different amounts for each domain.

Therefore, this adds a per-NUMA-domain buffer config section, which
currently has a single option to set the number of buffers.

If it's not specified or set to zero, the code falls back to
buffers-per-numa as before.

Type: improvement
Change-Id: If35d7a9eff6f8d1d78063ea7873dbf50780d0ec3
Signed-off-by: Lukas Stockner <lstockner@genesiscloud.com>
2024-06-05 12:28:27 +00:00
3eb6cbec50 vppinfra: os agnostic api for getting CPU information
Avoid direct sysfs reads when possible...

Type: improvement
Change-Id: I2b84cd18f3da47925d068951f24b79b5b6e20bb1
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-03-18 16:01:35 +00:00
c3148b1be8 misc: remove GNU Indent directives
Type: refactor
Change-Id: I5235bf3e9aff58af6ba2c14e8c6529c4fc9ec86c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-03-12 19:29:56 +00:00
4b6614030f vppinfra: add os_get_online_cpu_core() and os_get_online_cpu_node()
Type: improvement
Change-Id: I6f99f09c7724ce656a4f41a1d5f9c88d74c00faf
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-03-04 08:53:08 +00:00
04fd51c03c buffers: bring back cache occupancy improvement
The improvement was removed in 40129,
causing 5-40% regressions in AVF tests.

There is a memory-speed trade-off,
this change prefers speed over memory efficiency.

Ideally, the choice should be configurable,
but that is not easy to achieve, considering
how early is vlib_buffer_main_init called.

Type: fix
Fixes: 038dad7ef2

Change-Id: I4746f3634abe6d233c9d092a372de05b3d1ae4b6
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2024-02-12 16:20:31 +00:00
038dad7ef2 buffers: add compile-time option to set buffer alignment
Type: improvement
Change-Id: I88c4c45bed0bdd8686e17e4f77a7d32a08c995aa
Signed-off-by: Damjan Marion <damarion@cisco.com>
2024-01-20 18:28:09 +00:00
55a1f288ed buffers: buffer name is vector, not c string
Type: fix
Fixes: 8973b07
Change-Id: Iba6e29bab35be0148b469965c3af8391b5ab9140
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-25 11:51:30 +00:00
bf236630f5 buffers: introduce vlib_buffer_template_t
Type: improvement
Change-Id: Ie86a5edf2ada21355543e9a0382052b16ff86927
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-17 17:44:41 +00:00
65dc34bb0b buffers: buffer allocation improvements
- pass buffer pool name trough va
- make buffers naturaly aligned
- fix calculation of total number of buffers

Type: improvement
Change-Id: I6aebf249ebd67823b4632ac08905bfa3aa7d1ee5
Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-09 09:58:37 +00:00
321bd106a6 vlib: add VLIB_NUM_WORKERS_CHANGE_FN() handler
Allows features to update their data structures after change in number
of worker threads.

Type: improvement
Change-Id: Icd4d197e28608f5bbb1edd13eb624cd98e33cafe
Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-06-01 11:34:03 +00:00
8973b07eec stats: refactor
Type: refactor
Change-Id: Ifd533a095d979dc55bfbe5fac7e0b7510a4d900c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-03-09 13:51:31 +00:00
30d46e7455 vlib: fix the total len not including first buffer formating
Type: fix

total len not including first buffer is in vlib_buffer_t second cacheline.
It is not reset after the buffer has been consumed. It leads to printing
garbage in packet trace for subsequent use of this buffer. This patch
fixes the issue to only print when VLIB_BUFFER_NEXT_PRESENT flag is set.

Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
Change-Id: Ied72308bdb907a5e1ca16d181f2add062807e968
2022-01-11 12:47:52 +00:00
e09a2337b8 bufmon: add buffer monitoring plugin
This plugin allow to keep track of buffer usage in VPP graph nodes. The
main use is to detect buffer leakages.

Type: feature

Change-Id: Iadcf4ab98207fab6e2fa375060879bc2a25b711e
Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-08-27 10:05:31 +00:00
70b1cbdf2b vlib: fix buffer pool alignment size
Alignment size should be CLIB_CACHE_LINE_BYTES(64)
instead of CLIB_LOG2_CACHE_LINE_BYTES(6)

Type: fix

Signed-off-by: Tianyu Li <tianyu.li@arm.com>
Change-Id: If2d5ae324093be64454377866297f5e76ccddc93
2021-07-02 14:28:24 +00:00
fd8deb48c7 vlib: split vlib_main_t into global and per-thread
Type: refactor
Change-Id: I8b273bc3bf16aa360f031f1b2692f766e5fc4613
Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26 16:33:42 +01:00
6ffb7c6189 vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()
Type: improvement
Change-Id: If3da7d4338470912f37ff1794620418d928fb77f
Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26 16:33:21 +01:00
f0ca1e8d92 misc: refactor clib_bitmap_foreach macro
Type: refactor
Change-Id: I077110e1a422722e20aa546a6f3224c06ab0cde5
Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-14 12:14:21 +00:00
61559029da buffers: add page-size config
Type: feature

Add a `buffers {page-size}` parameter to specify page size
for buffers. This also fixes an issue with the parsing in
unformat_log2_page_size.

Change-Id: I7d7b1fa0bb7febaa7509cf2c625882f07eeafaad
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2020-11-24 15:56:08 +01:00
0be1b764a3 buffers: improve cache occupancy
Adjust buffer allocation so it always have odd number of cache lines.
That should result in better distribution of cachelines among cache sets.

Type: improvement
Change-Id: I0d39d4cf01cff36ad6f70a700730823a96448c22
Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-04 12:45:46 +00:00
c16fe46890 buffers: fix non-default sized buffers initialization
Type: fix

Change-Id: I4a93e1d9936414c514cb237a22624986b3ef5b3d
Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-06-15 14:52:04 +00:00
c74b43c807 buffers: configurable buffer fault injector
When configured at compile time via the cmake
VPP_BUFFER_FAULT_INJECTOR option, the buffer allocator will appear to
fail a certain fraction of the time.

By default, the allocator succeeds 80% of the time. Detailed command
line configuration options are available, but only when the image has
been compiled with cmake option described above:

    vlib { buffer-alloc-success-rate [0.0 ... 1.0]
           buffer-alloc-success-seed <nnnn> }

Modify vlib_buffer_pool_create(...) so 0 is always an invalid buffer
index.

Debug images: add checks for bad buffer index enqueues, and also
verify that f->n_vectors doesn't accidentally map one or more
instances of the frame poison pattern 0xfefefefe.

Type: improvement

Signed-off-by: Dave Barach <dave@barachs.net>
Change-Id: Iab939858014463d1e664682805013d334d6fcbe5
2020-04-13 15:54:31 +00:00
4354317bf3 vlib: only dump 1st buffer in chain by default
Several nodes include buffers in their traces, but only the 1st. When
formatting the trace we must not try to iterate through all chained
buffers.
Default to display only the 1st buffer.

Type: fix

Change-Id: Ib3c668bbf4ab70ae68eba2ac402c7b7329825b70
Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-10-31 22:46:04 +00:00
b6e8b1a7c8 vlib: don't use vector for keeping buffer indices in the pool
Type: refactor

Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-10-28 14:07:27 +00:00
785daf4f84 vlib: fix vlib_buffer_main_init_numa_node memory leak.
Type: fix

Signed-off-by: Guanghua Zhang <ghzhang@fiberhome.com>
Change-Id: I8252ed2555f5af6db2f12dc7c30e41cc1ec7dde0
2019-08-19 13:41:23 +00:00
a638c18375 vlib: packet tracer support for pkt thread handoffs
Type: feature

Change-Id: Ia3d9a47679202c2a47cd3746b50e86c6b8627ef6
Signed-off-by: Dave Barach <dave@barachs.net>
2019-06-24 17:13:24 +00:00
8feeaff56f Typos. A bunch of typos I've been collecting.
Change-Id: I53ab8d17914e6563110354e4052109ac02bf8f3b
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-28 06:36:04 +00:00
951551081b vlib: coverity woes
Remove logically dead code to keep coverity from complaining

Change-Id: If27d6684d19ab3c8886732a67922c86e5f0b3554
Signed-off-by: Steven Luong <sluong@cisco.com>
2019-03-06 11:35:00 +00:00
b592d1b641 buffers: support cases where numa node memory is not available
Change-Id: I550ef893e41d86310d4e5af16d6100b8e557b68a
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-03-04 12:22:56 +00:00
b806d0e1d7 stats: fix crash on early init
Change-Id: I911b5070f544c3556ad1ca584d42689c00cc38f3
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-03-04 11:17:15 +00:00
849cbe225e stats: buffer gauge callbacks use index instead of name.
Change-Id: I500b8e72d7e021b9564dad0ffc087cb0e1ceb96f
Signed-off-by: Ole Troan <ot@cisco.com>
2019-02-26 20:17:30 +00:00
d2bbdef694 stats: add buffer gauges
Change-Id: I7f7a459f25d64ea5fa36e30d7dccc667bc19c5a9
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-22 20:44:21 +00:00
8934a04596 buffers: fix typo
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-09 23:29:26 +01:00
5de3fec531 buffers: make buffer data size configurable from startup config
Example:

buffers {
	default data-size 1536
}

Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06 18:00:17 +00:00
672ab69018 buffers: improve buffer init performance
Change-Id: Ib59a3c32af754a898ade17c42e60a88f48b797ff
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06 17:58:32 +00:00
e82c73a96e assert on double-free in debug build
The assert allows easier debugging in gdb by aborting instead of exiting
cleanly. Also a core is generated so a stack trace can be obtained.

Change-Id: I2d05488c4522f4d4570fdfe0283130eb4c853d2a
Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-01-31 16:27:37 +00:00
24dcbe4520 buffers: reinitialize metadata, add additional validation
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need
  reinitialize it

- additional checks added to ensure ref_count is never < 1

Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31 12:33:16 +01:00
910d3694e8 buffers: major cleanup and improvements
This patch introduces following changes:
- deprecated free lists which are not used and not compatible
  with external buffer managers (i.e. DPDK)
- introduces native support for per-numa buffer pools
- significantly improves performance of buffer alloc and free

Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30 16:19:22 +00:00
3235382f23 buffers: remove VLIB_BUFFER_DEFAULT_FREE_LIST macro and fl->n_data_bytes
Change-Id: I0ba5175be077c40556f2a3ce629c5bbcd71e0a81
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20 21:01:03 +00:00
b67196d823 buffers: remove vlib_buffer_delete_free_list
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20 19:28:27 +00:00
6434430fd5 buffers: don't init metadata, as it is already initialized
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20 19:24:46 +00:00
d50e347763 buffers: keep buffer_main in vlib_main_t
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20 19:23:53 +00:00
36eb7c2142 buffers: remove free-list information from buffer metadata
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19 00:17:51 +00:00
ab9b7ec210 buffers: remove freelist argument from vlib_buffer_add_data
Change-Id: Idb224b807373128b78dc46f177b2d2e0ed288349
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18 23:18:05 +00:00
3a63fc5470 Handle buffer alloc failure in vlib_buffer_add_data
It's not OK to crash due to a transient buffer allocation failure.

Return 1 if the requested operation failed, otherwise 0.

Buffer index parameter change to a value-result, so the caller can
differentiate between partial and complete allocation failure: callers
which request an initial allocation (inbound bi = ~0) need to check
the (out) value to decide whether or not to call vlib_buffer_free(...).

Change-Id: I03029d7f2714c17dca4630dfd95a1eb578b68384
Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-07 16:47:09 +00:00
671e60e656 buffers: remove unused code
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6
Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-01 14:57:54 +00:00
ce55bcde5d vlib:init free list buffers vec
fixes a problem seen when dhcp proxy uses vlib_buffer_chain_linearize
function which tries to vlib_buffer_alloc zero buffers - which succeeds
if the buffers vector is initialized but otherwise crashes when trying to
update the _vec_len in vlib_buffer_alloc_from_free_list
solved by initializing the free_list buffers vec

Change-Id: I1186d7aac05e54864d72f3f144e1bef8064f8efa
Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-11-25 15:30:13 +00:00
178cf493d0 Remove c-11 memcpy checks from perf-critical code
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1
Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-14 15:54:01 +00:00
b688fb1297 vlib:remove unused argument
Change-Id: I88c3d3e516401bb1c84991515cd701c156ae19dd
Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-11-13 18:03:43 +00:00