Commit Graph

445 Commits

Author SHA1 Message Date
Dave Wallace
604c627dc3 vapi: fix coverity warnings
- CID-322713, CID-322712, CID-314881, CID-314880,
  CID-314878 COPY_INSTEAD_OF_MOVE

Type: fix

Change-Id: Idad6806fcd4e1a89a750dcc0584b5b1f0432bf3c
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-09-16 07:50:17 +00:00
Sylvain Cadilhac
db8c285081 vapi: fix verification for reply message
Type: fix
Byte swapping should be done before verifying message otherwise the message length will be wrongly computed.

Change-Id: I90b2f60bd33e5362e0edd5ee425f6a6f07886f1e
Signed-off-by: Sylvain Cadilhac <sylvain.cadilhac@freepro.com>
2023-09-06 08:23:55 +00:00
Klement Sekera
03e1d559f9 vapi: improve vl_api_string_t handling
Define vl_api_string_t to correspond with vlibapi/api_types.h
Fix allocation and size calculation routine generation.

Type: improvement
Change-Id: I6b0a3eb3459d75d326e67bfb76dac8125e480afa
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
2023-08-18 06:09:41 +00:00
Stanislav Zaikin
56777b9409 vapi: support services
Add missing support for

service { rpc X_get returns X_get_reply stream X_details; }

Type: improvement

Change-Id: I27555f61a2974e414cb6554f32c550b8ee5eb037
Signed-off-by: Stanislav Zaikin <stanislav.zaikin@46labs.com>
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
2023-08-18 06:09:10 +00:00
Dave Wallace
7b8b465269 tests docs: update python3 venv packages
- Package update performed by
  1. updating pip, pip-tools, setuptools
  2. 'make test-refresh-deps' on ubuntu 22.04
  3. fixing 'make test' and 'make docs' issues
     on ubuntu 22.04
  4. 'make test-refresh-deps' on ubuntu 20.04
- Add dependency for 'make test-refresh-deps'
  to insure python venv is set up.
- Update of python formatter, black,
  caused reformating of 41 python code
  files.

Type: make

Change-Id: I7cafdf4b5189065ac57cb6b254937f6e0897a924
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-08-16 19:53:12 +00:00
Duncan Eastoe
cf214cc600 stats: fix memory leak in stat_segment_dump_r()
The built stat_segment_data_t is leaked if stat_segment_access_end()
returns false.

Type: fix
Signed-off-by: Duncan Eastoe <duncan@graphiant.com>
Change-Id: I70adabbe7947d3e8a798cdfb3eaa14c683dce9da
2023-05-19 21:17:37 +00:00
Ole Troan
22ca0d03db stats: check if stats vector entry is empty
When a stats entry is removed it is marked empty.
The stats client did not check for that and returned an empty string.
This resulted in blank lines in vpp_get_stats. Fix by returning null instead
and checking value.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I08a39ba3ef4421bf275747a6300f97fe36791b50
2023-04-17 14:04:06 +00:00
Ole Troan
86698fca30 papi: vla list of fixed strings
Handle a variable length array of fixed strings.
Like:
fixed_string = VPPType("fixed_string", [["string", "data", 32]])
s = VPPType("string_vla", [["u32", "length"], ["fixed_string", "services", 0, "length"]])

Previously instead of packing and unpacking as strings, exception packed as u8 instead
of list.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I501a8a4755828042e1539fd5a54eacec21c5e364
Signed-off-by: Ole Troan <ot@cisco.com>
2023-03-21 13:53:09 +00:00
Damjan Marion
1cada9ad78 build: make Python3 mandatory
Type: refactor
Change-Id: Iac27ac4d11745b68c57a0394ced51942db8f0431
Signed-off-by: Damjan Marion <dmarion@0xa5.net>
2023-03-07 02:22:47 +00:00
Ole Troan
04853c67e4 stats: expose symlink to stats client
For e.g. prometheus export it makes more sense to use the same metric name,
and expose the various symlinks as labels.

The VPP symlink metric:
/interfaces/local0/rx_unicast

that points to
/if/rx_unicast

Becomes in Prometheus:
interfaces_rx_unicast_bytes{index="0",label="local0"} 0

Type: improvement
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Ide0ab4fda4b3eb7ba7ddfc44680121c53f5267f6
2023-03-02 08:00:50 +00:00
Matthew Smith
57f177d0b7 vapi: add vapi_stop_rx_thread()
Type: improvement

Allow vapi to signal to an application's RX thread that it should wake
up and exit.

Before disconnecting from VPP's API, libvlibmemoryclient inserts an
rx_thread_exit message into the client's own input queue to cause its
RX thread to wake up from its blocking dequeue and exit cleanly. Add a
function to vapi's API which will allow libvapi client applications
which have an RX thread waiting for incoming messages using vapi_wait()
to do the same thing.

The existing libvlibmemoryclient code which does this was moved to a
separate function and made available for vapi_stop_rx_thread() to call.

Also fixed some inconsistencies in indentation of function prototypes in
vapi.h to make checkstyle.sh happy.

Signed-off-by: Matthew Smith <mgsmith@netgate.com>
Change-Id: I7bbb73470807123cc63ef313cfb91d1fd31b34e5
2023-01-04 17:00:15 +00:00
Mauro Sardara
8c626b41ea vapi: use the correct my_context_id when disconnecting API clients
While before the my_client_index variable was stored as global variable
in api_main_t, after commit 2ca88ff978
the my_client_index becomes part of vapi_ctx_t.

Each API client (internal/external) connected to VPP stores its
client index in vapi_ctx_t.

The issue is in the client disconnection. The vapi_disconnect is
untouched in patch 2ca88ff978,
so it keeps the behavior of using the my_client_index stored
in api_main_t.

Ticket: VPP-2069
Type: fix
Fixes: 2ca88ff978

Signed-off-by: Mauro Sardara <msardara@cisco.com>
Change-Id: Idf8c1d1056cbd631cc3057cf7acc486216fa8303
2022-12-20 12:08:06 +00:00
Matthew Smith
4b9935cd54 vapi: implement vapi_wait() for reads
Type: improvement

The function vapi_wait() is intended to allow a caller to block while
waiting until the API queue can be read/written. It was a stub that
returned VAPI_ENOTSUP. Add code which implements the wait on being able
to read an incoming message.

Had to touch a few other things in vapi.h to make checkstyle.sh happy
after changing the prototype of vapi_wait().

Signed-off-by: Matthew Smith <mgsmith@netgate.com>
Change-Id: Ida80c1a1d34fe297ab23268087be65ea53ad7040
2022-12-14 14:07:11 +00:00
Pim van Pelt
1588222e72 papi: fix VPP_API_DIR
Docstring in VPP Python API says that find_api_dir() will search for
environment variable VPP_API_DIR first and foremost, except it doesn't.
Prepend VPP_API_DIR if it exists, and allow dirs to be omitted in case
it will be the empty list []

Type: fix
Signed-off-by: pim@ipng.nl
Change-Id: Ic892e4bb7d8ff50f18e64ddfd2a61343883f07ea
2022-12-12 00:00:49 +00:00
Ole Troan
bcdde1a0a1 papi: export packed message structures
Use the Python API binding to generate a set of API messages
in binary format, that can later be replayed independently
of the Python API.

Type: improvement
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Iaab6ca31fd2809193e461ab53f7cc7332a231eb5
Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07 10:33:37 +00:00
Ole Troan
37157dad51 tests: multiple apidir locations
To support testing of external plugins, add support to the test framework and PAPI
for specifying a list of locations to look for api.json files.

Type: improvement
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I128a306e3c091dc8ef994801b1470b82d2f4595d
Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07 10:33:20 +00:00
Ole Troan
f22824bbef papi: fix async support for socket transport
Async use of the API is much faster than blocking calls.
Seemed like it only worked over shared memory transport.
This patches re-enables support for async calls over socket
transport.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I05f3b362035ce0a1c16788ba9003a35601ddb04e
Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07 10:32:53 +00:00
Andrew Yourtchenko
9067f3332e stats: return empty vector rather than NULL if stat_segment_dump_r() is run on an empty vector from ls
The return value in this function is initialized with 0, so if a vector of length 0 is passed
to stat_segment_dump_r, then this return value is never populated, resulting in inability
to distinguish between a successful dump of an empty vector and an error.

Solution: call vec_alloc(). As a side effect might get some trivial speed-up.

Type: fix
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Change-Id: I33fefd801df457152e9ec257742305182e91f339
2022-12-05 09:50:18 +00:00
Andrew Yourtchenko
3f885f97f0 stats: fix the memory leak in stat_client.c
The issue can be reproduced by running "vpp_get_stats tightpoll"

The root cause is that the control flow discards the "result" struct
being prepared, along with pointer its allocated name.
This results in a memory leak.

Type: fix
Change-Id: Ibf884e92314f19b983a0159fc1257b3fa0110443
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2022-11-29 13:48:45 +00:00
Matthew Smith
d1c75063da vapi: write enumflag types to vapi headers
Type: fix
Fixes: a51f9b3747

Some IPsec message type definitions were not being written to
ipsec.api.vapi.h. These include ipsec_sad_entry_add_del_v3 and
ipsec_sad_entry_add.

The cause appears to be that tunnel_flags, which is defined in
tunnel_types.api is a special case of enum called an enumflag. These do
not appear to have been handled in the code that generates the vapi
header files.

This patch adds processing of enumflag objects for vapi.

Change-Id: Ie506c4fcb5a07fe97a330ba11c252d1df98adfd9
Signed-off-by: Matthew Smith <mgsmith@netgate.com>
2022-11-28 21:34:00 +00:00
Damjan Marion
dd298e804a misc: fix issues reported by clang-15
Type: improvement

Change-Id: I3fbbda0378b72843ecd39a7e8592dedc9757793a
Signed-off-by: Damjan Marion <dmarion@me.com>
2022-10-12 16:04:13 +02:00
Viktor Velichkin
7c3096dac7 papi: vpp_serializer.py - replace slow bytes() with fast bytearray()
https://docs.python.org/3/library/stdtypes.html

"if concatenating bytes objects, you can similarly use bytes.join() or io.BytesIO, or you can do in-place concatenation with a bytearray object. bytearray objects are mutable and have an efficient overallocation mechanism"

Type: improvement

Signed-off-by: Viktor Velichkin <avisom@yandex.ru>
Change-Id: Id20d337f909cce83fcd9e08e8049bb0bf5970fbc
2022-06-01 14:36:40 +00:00
Klement Sekera
d9b0c6fbf7 tests: replace pycodestyle with black
Drop pycodestyle for code style checking in favor of black. Black is
much faster, stable PEP8 compliant code style checker offering also
automatic formatting. It aims to be very stable and produce smallest
diffs. It's used by many small and big projects.

Running checkstyle with black takes a few seconds with a terse output.
Thus, test-checkstyle-diff is no longer necessary.

Expand scope of checkstyle to all python files in the repo, replacing
test-checkstyle with checkstyle-python.

Also, fixstyle-python is now available for automatic style formatting.

Note: python virtualenv has been consolidated in test/Makefile,
test/requirements*.txt which will eventually be moved to a central
location.  This is required to simply the automated generation of
docker executor images in the CI.

Type: improvement
Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-05-10 18:52:08 +00:00
Ole Troan
2ca88ff978 vapi: support api clients within vpp process
Add vapi_connect_from_vpp() and vapi_disconnect_from_vpp()
calls to allow API clients from within VPP process.

Add a new memclnt_create version that gives the user a
knob to enable or disable dead client scans (keepalive).

Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Id0b7bb89308db3a3aed2d3fcbedf4e1282dcd03f
Signed-off-by: Ole Troan <ot@cisco.com>
2022-05-02 16:26:24 +02:00
Renato Botelho do Couto
ed1f9adad7 vapi: Fix build when directory contains @
During build some header guards are created based on full path where
build is happening.  If one directory contains @ character build breaks
because compiler believes it's a macro declaration.

Jenkins adds `@${EXECUTOR_NUMBER}` suffix to workspace directory when
it uses more than one executor for that job, breaking the build.

Replace any @ character on guard name by _ to get it fixed.

Type: fix
Change-Id: Id0f4cfc33fda95e168541aa4e353a0d08aa3b664
Signed-off-by: Renato Botelho do Couto <renato@netgate.com>
2022-04-27 17:54:50 -03:00
Damjan Marion
66c858385f stats: convert error counters to normal counters
Change-Id: I9794da718805b40cc922e4f3cf316255398029a9
Type: improvement
Signed-off-by: Damjan Marion <damarion@cisco.com>
Signed-off-by: Ole Troan <ot@cisco.com>
2022-03-31 15:51:26 +00:00
Damjan Marion
8973b07eec stats: refactor
Type: refactor
Change-Id: Ifd533a095d979dc55bfbe5fac7e0b7510a4d900c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-03-09 13:51:31 +00:00
Dave Wallace
6081dda84b vapi: Missing include file in vlib.api.vapi.h
- A call to vapi_msg_control_ping_hton() is generated in
  a static inline function in vlib.api.vapi.h, which is
  defined/generated in memclnt.api.vapi.h without
  including memclnt.api.vapi.h in vlib.api.vapi.h.
  This breaks the compilation of plugins which include
  only vlib.api.vapi.h (e.g. hicn_plugin from the HICN
  project).

Type: fix
Fixes: a1400cecb

Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I5574f4ed496183ea93265f493bf3624254a865a2
2022-02-24 23:19:17 +00:00
Steven Luong
0a214804f8 stats: vpp_get_stats crashes in stat_segment_data_free
STAT_DIR_TYPE_EMPTY is not handled. This can happen when the
interface is deleted.

Type: fix

Signed-off-by: Steven Luong <sluong@cisco.com>
Change-Id: Ic1f5a1a0e7005059628d1dde31118d692c1967d8
2022-02-01 15:19:06 +00:00
Klement Sekera
5f0106a76f api: vapi: honor non-blocking setting
Pass correct conditional based on how vapi is configured wrt blocking.

Type: fix
Fixes: 3fca567ff4
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: I47adca19f104d7a758cb2940e93c9fd8c7cc9bfa
2022-01-25 14:35:02 +00:00
Klement Sekera
a25ce96ba3 vapi: verify message size when received
Verifying message size including VLA size allows to dismiss some
coverity warnings in generated code.

Type: improvement
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: I824658881254b3e7a9bfca228a266cfee448cc2e
2021-11-16 10:30:50 +00:00
Klement Sekera
eb416708b1 tests: change code to avoid gcc-11 warning
For some reason gcc-11 doesn't like memcpy in a particular location,
change memcpy to assignment.

Type: refactor
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: Iddf4b788cdca8e55221b121e74cf50b1e6bdf2d5
2021-11-02 22:27:03 +00:00
Filip Tehlar
f0e67d78ae interface: add api test file
Type: improvement

Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
Change-Id: Ib07029204ecf12bf2adb5a39afa54bc98fb81f34
2021-10-18 07:03:12 +00:00
Klement Sekera
a0f55c9ce9 tests: fix coverity warnings
Fix coverity warnings by initialising variables and removing dead code.

Type: fix
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: I0254b81020cdc3d7075df003309065438526918f
2021-10-14 07:12:43 +00:00
Nathan Skrzypczak
d4a70647e6 docs: convert vpp doc md->rst
Type: improvement

Change-Id: If453321785b04f9c16e8cea36fb1910efaeb2c59
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-10-13 15:32:33 +00:00
Florin Coras
2dfeef522c vapi: fix vapi test coverity warning
Type: fix

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I5ecb73009c6ebb00b5d9e14bd09b4b3e80ab5601
2021-10-13 14:31:21 +00:00
Filip Tehlar
36217e3ca8 api: API trace improvements
Type: improvement

 * add support for JSON format in API trace
 * add ability to replay JSON API trace in both VPP and VAT2
 * use CRC for backward compatibility check during JSON API replay
 * fix API trace CLI (and remove duplicits)
 * remove custom dump
 * remove vppapitrace.py
 * update docs accordingly

Change-Id: I5294f68bebe6cbe738630f457f3a87720e06486b
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-28 16:06:19 +00:00
Ole Troan
3459ece6da misc: vpe.api messages dynamically allocated
This is the last in the series of moving API messages from vpp/api/vpe.api to vlibmemory/memclnt.api.
This patch makes the remaining vpe.api messages dynamic, to help VAT2 binary-api command.
Moves the VAT test code to a separate file and removes the now unnused API meta files.

Type: improvement
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I01dd78eaff1d3715dff17d2643bf0f7f0089935b
Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-28 15:17:37 +00:00
Nick Brown
d67ad46903 build: complete python3 support, no hardcode path
find_package(Python3) will not set variables that are later used, so set
those needed. Perhaps the python2 support, which is EOL, could be
dropped?

Use DESTDIR, instead of hardcoding the path. This allows system
packaging, or local installs, to work properly.

Type: make
Signed-off-by: Nick Brown <nickbroon@gmail.com>
Change-Id: I045516c61473c612ab70858cd9b58c4e2838b347
2021-09-28 11:11:55 +00:00
Dmitry Valter
41d7d0569b tests: don't install vapi_c{,pp}_test
Don't install vapi_c{,pp}_test. It confuses dpkg-shlibdeps to think we
need libsubunit as shared lib dependency

Type: fix
Fixes: a2d6d352c6
Signed-off-by: d-valter@yandex-team.ru
Change-Id: Ifb702a61be32b37e79b48780cc61cb0838e87153
2021-09-27 14:38:47 +00:00
Florin Coras
a1400cecb8 misc: api move continued
Move control ping and change dependencies from vpe.api_types to
memclnt.api_types

Type: refactor

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I9f8bc442e28738c48d64d1f6794082c8c4f5725b
2021-09-27 08:01:44 +00:00
Nick Brown
e3cf4d0cf3 build: use GNUInstallDirs install destinations
So as to be compliant with distribution layouts, as recommend by:
https://cmake.org/cmake/help/latest/command/install.html#installing-files

Type: make
Change-Id: Ic46ace4f26aab1aa4902cbd013c40a92c480680d
Signed-off-by: Nick Brown <nickbroon@gmail.com>
2021-09-17 20:25:49 +00:00
Ole Troan
43eb6083c5 stats: check epoch in python vpp_stats ls
The VPP Stats Python binding was missing checking epoch.
Resulting it would not pick up changes in the directory.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I3226d71c77be2e80e24b5cd48fc9820833f7d30e
Signed-off-by: Ole Troan <ot@cisco.com>
2021-08-31 15:26:11 +00:00
Ole Troan
041372b79b papi: remove shared memory transport
This patch removes the papi transport shared memory plugin.
It also removes any dependency on CFFI.

Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Ia81701c0dc506871e511495d837e41420e1fdf72
Signed-off-by: Ole Troan <ot@cisco.com>
2021-07-12 10:41:10 +00:00
Arthur de Kerhor
905c15319c stats: reverts part of a fix on Python client
Puts return statements back inside code blocks under the lock

Type: fix

Change-Id: I76d426f336200035026b92bcb0ffe2b472a3142d
Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com>
2021-06-25 23:26:40 +00:00
Arthur de Kerhor
c9ae8cfacc stats: fix race conditions in vpp-api stats client
Type: fix

Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: Ie5c197f6ec0d41d5e405b22662701d83ad94d29e
2021-06-25 01:04:16 +00:00
Damjan Marion
7cf80af582 build: kindly ask python not to polute src/
Type: make
Change-Id: I1ec82e0f3c19dee79f942996339240c73d4e380c
Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-05-27 20:53:50 +00:00
Damjan Marion
a2d6d352c6 tests: move vapi C/C++ test files into src/
Type: improvement
Change-Id: Iaad004661db1e9076e6c27227878f1612fa7289f
Signed-off-by: Damjan Marion <damarion@cisco.com>
Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-05-26 06:32:43 +00:00
Ole Troan
d640ae52f6 stats: catch stat segment overrun in retry loop
Raise IOError so that the blocking code retries not only on
optimistic locking failures but also on data segment illegal reads.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I6bb250e239486b60192004271c1690e790513318
2021-05-21 21:07:38 +00:00
Dave Wallace
eddd8e3588 tests: move test source to vpp/test
- Generate copyright year and version
  instead of using hard-coded data

Type: refactor

Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
2021-05-13 09:33:06 +00:00