175 Commits

Author SHA1 Message Date
9a3f387c30 vppapigen: endian generation for vla for non u32
The endian function for VLAs assumed length field to be of type u32.
That failed for APIs using different integer width.

Type: fix
Change-Id: I6ecaabb4563c8bafeb100a4c6c1eee9a08e6cabf
Signed-off-by: Ole Troan <otroan@employees.org>
2024-09-12 12:56:56 +00:00
9aa833b144 vppapigen: ensure address types are nul terminated
A string generated from format() may not be nul terminated.

Type: fix
Change-Id: I88452e446c3504d70758e9009c65be5466034d92
Signed-off-by: Ole Troan <otroan@employees.org>
2024-08-07 11:46:51 +00:00
ed61b20ad7 build: vppapigen dependency handling
Add dependency generation to the vppapigen compiler, so that
when an API file depends on another, that's registered as a dependency
with the build system.

Add a build dependency on vppapigen submodules so that all api files
are regenerated if the compiler itself changes.

Type: improvement
Change-Id: I392853754129778ef15532d1b04813786b943b44
Signed-off-by: Ole Troan <otroan@employees.org>
2024-06-19 13:18:36 +02:00
1a319aadc6 api: add to_net parameter to endian messages
The VPP API auto-generated endian conversion functions are intended to
be symmetrical. They are used both by the API client and the API server.
Called on send to convert from host endian to network endian and on
receive to convert back.

For variable length arrays, we have to iterate over the array and call
a more specific handler for the array type. Unfortunately the length of
the array is part of the api definition, and if it's endian swapped
prior to the for loop, unexpected behaviour will ensue.

There was an earlier fix, for some specific messages, but unfortunately
that only fixed the problem from the VPP (server) side.

This adds a new parameters to the endian handler, so the boundary
argument to the loop can be treated differently depending on if this
message is to the network or from the network.

Type: fix
Change-Id: I43011aed384e3b847579a1dd2c390867ae17a9ad
Signed-off-by: Ole Troan <otroan@employees.org>
2024-05-07 09:59:31 +00:00
4013851e42 api: upgrade cjson and fix realloc
Upgrade cJSON library to patchlevel 17.
Replace internal realloc added earlier 36217e3ca.

Type: fix
Fixes: 36217e3ca
Change-Id: I7d8a80dc4241e9f952895d24adca8fa2d873e746
Signed-off-by: Ole Troan <otroan@employees.org>
2024-05-07 09:59:00 +00:00
d7f0daecce g2: fix gpointer cast complaints
Type: fix

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: Iadd6a4dd928002e764fc52343c14403620279aa7
2024-04-11 01:07:59 +00:00
f521df7200 build: use GNUInstallDirs where possible
Some installation destinations were already converted to the
GNUInstallDirs standard in e3cf4d0cf. This patch converts the share and
include directories (typically /usr/share and /usr/include respectively)
as well.

Type: improvement
Change-Id: Id481b16604f3f52203166a5c0647f43d6eb05239
Signed-off-by: Wim de With <wf@dewith.io>
2024-04-10 13:59:41 +00:00
ac0babd412 api: provide api definition over api
This patch allows a client to bootstrap itself by downloading the
JSON API definitions over the API itself.

This patch enables it for Python (probably need a dynamic language).
Call VPPApiClient with the new bootstrapapi=True parameter.

Example (Python):

from vpp_papi import VPPApiClient
vpp = VPPApiClient(bootstrapapi=True)
rv = vpp.connect("foobar")
assert rv == 0
print(f'SHOW VERSION: {vpp.api.show_version()}')
vpp.disconnect()

Type: feature
Change-Id: Id903fdccc82b2e22aa1994331d2c150253f2ccae
Signed-off-by: Ole Troan <otroan@employees.org>
2024-01-29 22:57:31 +00:00
7e43005d4d perftool: fix delsvec use of _vec_len
Type: fix

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: Iab80e009d8717f759edb8133beaa55ed05e57bd5
2023-08-26 13:14:46 -07:00
7b8b465269 tests docs: update python3 venv packages
- Package update performed by
  1. updating pip, pip-tools, setuptools
  2. 'make test-refresh-deps' on ubuntu 22.04
  3. fixing 'make test' and 'make docs' issues
     on ubuntu 22.04
  4. 'make test-refresh-deps' on ubuntu 20.04
- Add dependency for 'make test-refresh-deps'
  to insure python venv is set up.
- Update of python formatter, black,
  caused reformating of 41 python code
  files.

Type: make

Change-Id: I7cafdf4b5189065ac57cb6b254937f6e0897a924
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-08-16 19:53:12 +00:00
f72a32a1e4 docs: add api change process from wiki
- Move the VPP API Change Process documentation
  from the wiki page into the in-tree VPP docs

Type: docs

Change-Id: I42f661618b8632230bebe3aa8fbad455b9a05d01
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-07-17 14:55:06 +00:00
39c40fa349 vppapigen: fix crash with autoendian arrays
Type: fix
Ticket: VPP-2078

Change-Id: I418269632bdfc823c5f0ba7652957277276d294d
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-06-13 13:25:18 +00:00
683bdb6743 vppapigen: support counters only .api
In some cases an .api file may contain only counter definitions.
If so do not generate the setup_msg functions.

Type: improvement
Change-Id: Idf89a7a5ab135428e9577726bc356acfd7c30113
Signed-off-by: Ole Troan <otroan@employees.org>
2023-05-11 22:04:46 +02:00
16cc15b232 build: check for presence of python ply
Type: improvement
Change-Id: I4f190607bfce404fbe68ec968e6923509ea9519b
Signed-off-by: Damjan Marion <dmarion@me.com>
2023-03-07 02:29:56 +00:00
a4f994f31e vppapigen: fix incorrect comments in json
Type: fix

Signed-off-by: Ondrej Fabry <ofabry@cisco.com>
Change-Id: I241cefbbce98cf6fef83f36bd87ae2c1f4b067f0
2023-02-03 13:29:35 +01:00
139b2da5c5 vppapigen: enable codegen for stream message types
Enable codegen for C type from 'rpc A returns B stream C' notation

Type: improvement

Change-Id: I05cfce71c385d414d7b177a080009628bc8c8fad
Signed-off-by: Stanislav Zaikin <zstaseg@gmail.com>
2023-01-23 21:01:06 +01:00
5d2346801b vppapigen: include comments in json
Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Ibd796adea734b64d9209c5e18c5b9800cbaf62c6
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-01-18 01:32:24 +00:00
af5744c359 misc: use right include for fctnl.h and poll.h
Musl is stricter than glibc and has a warning that including fctnl.h and
poll.h should be prefered rather than their sys/ counterparts, which
breaks -Wall setups.

Type: fix
Signed-off-by: Guillaume Solignac <gsoligna@cisco.com>
Change-Id: Id101e999371951b0927cc8c4109f8f1536de1bc2
2023-01-12 13:18:36 +00:00
a2eb507055 api: deprecate vl_msg_api_set_handlers
Type: refactor

Change-Id: I7b7ca9ec62cb70243c5b7e87968eab1338d67ec8
Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-09-29 15:36:00 +00:00
fe45f8f5af api: replace print functions wth format
Type: improvement
Change-Id: I7f7050c19453a69a7fb6c5e62f8f57db847d9144
Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-09-26 08:14:29 +00:00
1b299fa467 vppapigen: make json in parallel
Type: improvement

This patches makes the make json-api-files
run in parallel in the same python runtime.

Default number of workers is 8, and run time
goes from ~20s to ~2s on average.

Change-Id: Id8cff013889db2671f6b6b4af9a019460c656f81
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2022-08-05 08:33:10 +00:00
78d24f3e5e vppapigen: fix make go-api for go1.18
This patch updates the go-api-files logic for supporting go1.18.
Notable changes are that `go get ...` changed to `go install`
and that we need to bump the govpp binapigen version to integrate
a go1.18 fix.

This patch also simplifies the cli execution syntax

Type: fix

Change-Id: I1d8aac65490fe3ea4c1965a4775b6bf8d5c05d26
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2022-05-30 13:15:57 +00:00
9a8d3d7107 api: start enum values from 0
C enum values by default start from 0.
All unassigned names should get value as value of previous name plus one.
The problem was that default value was 0 and adding one for the first
value would make generated api files start from 1 instead of 0.

Type: fix

Change-Id: I772d6411435648de3ec514f57025ef1acae87338
Signed-off-by: Filip Varga <fivarga@cisco.com>
2022-05-24 12:36:29 +02:00
d9b0c6fbf7 tests: replace pycodestyle with black
Drop pycodestyle for code style checking in favor of black. Black is
much faster, stable PEP8 compliant code style checker offering also
automatic formatting. It aims to be very stable and produce smallest
diffs. It's used by many small and big projects.

Running checkstyle with black takes a few seconds with a terse output.
Thus, test-checkstyle-diff is no longer necessary.

Expand scope of checkstyle to all python files in the repo, replacing
test-checkstyle with checkstyle-python.

Also, fixstyle-python is now available for automatic style formatting.

Note: python virtualenv has been consolidated in test/Makefile,
test/requirements*.txt which will eventually be moved to a central
location.  This is required to simply the automated generation of
docker executor images in the CI.

Type: improvement
Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8
Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-05-10 18:52:08 +00:00
8bea589cfe vppinfra: make _vec_len() read-only
Use of _vec_len() to set vector length breaks address sanitizer.
Users should use vec_set_len(), vec_inc_len(), vec_dec_len () instead.

Type: improvement
Change-Id: I441ae948771eb21c23a61f3ff9163bdad74a2cb8
Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-04-04 23:17:13 +00:00
d591b82cc8 misc: Improve go bindings gen
Type: improvement

Change-Id: Id705dab895602a60b053296b560ca3db5b0cd344
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2022-03-17 13:05:48 +00:00
9b7e8acf79 api: verify message size on receipt
When a message is received, verify that it's sufficiently large to
accomodate any VLAs within message. To do that, we need a way to
calculate message size including any VLAs. This patch adds such
funcionality to vppapigen and necessary C code to use those to validate
message size on receipt. Drop messages which are malformed.

Type: improvement
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: I2903aa21dee84be6822b064795ba314de46c18f4
2021-12-14 09:15:48 +00:00
0eb83f484c api: improve REPLY_MACRO safety
Improve vppapigen to generate per-message #define indicating whether
said message is dynamically sized (due to VLA or string) or not. Use
these #defines in REPLY_MACROs to prevent improper usage. Fix existing
improper REPLY_MACRO* usage.

Type: improvement
Change-Id: Ia77aaf9f6cf3ed68ea21075a4cc8deda78a68651
Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-12-08 09:39:21 +00:00
4551715aaa vppapigen: coverity 219549, dead code in generated file
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I64804092917f1d8dc520549bacae81672a95cd1e
2021-11-25 15:25:34 +00:00
d4a70647e6 docs: convert vpp doc md->rst
Type: improvement

Change-Id: If453321785b04f9c16e8cea36fb1910efaeb2c59
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-10-13 15:32:33 +00:00
36217e3ca8 api: API trace improvements
Type: improvement

 * add support for JSON format in API trace
 * add ability to replay JSON API trace in both VPP and VAT2
 * use CRC for backward compatibility check during JSON API replay
 * fix API trace CLI (and remove duplicits)
 * remove custom dump
 * remove vppapitrace.py
 * update docs accordingly

Change-Id: I5294f68bebe6cbe738630f457f3a87720e06486b
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-28 16:06:19 +00:00
a1400cecb8 misc: api move continued
Move control ping and change dependencies from vpe.api_types to
memclnt.api_types

Type: refactor

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I9f8bc442e28738c48d64d1f6794082c8c4f5725b
2021-09-27 08:01:44 +00:00
248210c6ca misc: move part of vpe apis to vlibmemory
VPE apis are actually vlib apis. This moves those that are not tightly
coupled with vapi to vlib_api

Type: refactor

Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I456a64ce49a0cdeff4a0931c6ea513cb639f683e
Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-27 07:04:51 +00:00
91144fbf54 vat2: add shared memory argument
Add prefix argument to specifiy shared memory segment.
Add long arguments.
Add argument to dump apis.
Add help.
Add template argument

E.g:
vat2 --template sw_interface_add_del_address
{
	"_msgname":	"sw_interface_add_del_address",
	"sw_if_index":	0,
	"is_add":	false,
	"del_all":	false,
	"prefix":	"0.0.0.0"
}

Usage: vat2 [OPTION] <message-name> <JSON object>
Send API message to VPP and print reply

-d, --debug       Print additional information
-p, --prefix      Specify shared memory prefix to connect to a given VPP instance
-f, --file        File containing a JSON object with the arguments for the message to send
--dump-apis       List all APIs available from connected VPP instance

Type: improvement
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I2d32483a727bc16990c9a30dfa9bc1fa7b1fa85a
Signed-off-by: Ole Troan <ot@cisco.com>
2021-08-18 08:05:16 +00:00
976a0ed6ac api: enable trace / replay flag on messages
For an unknown reason the trace/replay flags where missed
when moving API message registration code from manually
cut and pasted to aut-generated.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Ib7625a57d3a263aac154682007459648953b1803
2021-07-22 08:59:24 +00:00
b7e4d4487c misc: remove vnet_all_api_h and vnet_msg_enum
These file are no longer needed

Type: improvement

Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
Change-Id: I34f8e0b7e17d9e8c06dcd6c5ffe51aa273cdec07
2021-07-13 19:27:22 +00:00
2c4acdd8d4 vlib: rename vl_counter_t to vlib_error_desc_t
Type: improvement
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I368b0410db2d633d3c52199c840e24d21952c1b4
2021-05-06 14:19:55 +00:00
3ff1598846 misc: auto-generate go bindings
Type: feature

Added target 'make go-api-files' creating compatible go bindings
using JSON API definition and GoVPP binary API generator.

Signed-off-by: Vladimir Lavor <vlavor@cisco.com>
Change-Id: I5bae113b85eaf5ebda8e292d34c9826075ef19b5
2021-04-27 09:24:48 +00:00
da1b76aa89 vppapigen: remove flag day code
The flag day for the crc algorithm change has long passed and
should not be carried into the next release.

Type: refactor

Change-Id: I1b1027139a778907926e48a8a553b908a79db08f
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2021-04-26 07:17:29 +00:00
ef9c23503c vppapigen: expose the values of per-message "options" in the api.json files
Also fix the vapi parser's assumption about what
the container with CRC is supposed to look like..

Change-Id: I3a23ef6c1502232742c03d227eb3654fb757709c
Type: improvement
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-03-05 21:12:56 +00:00
ab9f57355f api: crchcecker ignore version < 1.0.0 and outside of src directory
- For check patchset ignore files outside of src directory
- For check patchset ignore files that have version < 1.0.0
- fix Pylint warnings
- Modify vppapigen_crc to include version in JSON output

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I171cf6397e129e2438b2a494c5656236a7810f7b
2021-03-05 10:52:51 +00:00
edc73fddba vat2: add sanity checking - coverity errors
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I3cd56690fe52402d4cfa9ea67f1de53d8d919dee
2021-02-17 15:51:22 +00:00
384c72f630 vppapigen: resource leakage in fromjson array - coverity
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I43283c59fd121dcb2486b26151108c90b027748b
2021-02-17 13:46:54 +01:00
93c4b1bb3f vppapigen: more _fromjson autogeneration coverity fixes
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I9a7bb617a3fa87d6ef49c75277e53425310cdcf9
Signed-off-by: Ole Troan <ot@cisco.com>
2021-02-17 08:08:08 +00:00
b00c49ca67 vppapigen: py2 cleanup - remove subclassing of object
Type: refactor

Change-Id: I7136cb8ba101ea3917dacc31ceb3a76a31328301
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2021-02-15 10:00:38 +00:00
cf0102b3ba vppapigen: coverity issues in autogenerated code pass 3.
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I5ee2e8aba3ee7281bbca11825dece79983e52f06
2021-02-12 14:51:41 +00:00
fb0afab7f5 vppapigen: fix fromjson coverity errors in generation
Fix memory leak coverity errors where free was not called
on error conditions. Or called twice.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I21cffa8b01e4f72f10501f202f6a762ae300a941
Signed-off-by: Ole Troan <ot@cisco.com>
2021-02-11 15:30:11 +00:00
c9c9143898 nat: fix coverity errors
Including a general missing free in fromjson autogenerated code.

Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I9ab2b0193135e2fb3d62d51b3c114df56969e341
Signed-off-by: Ole Troan <ot@cisco.com>
2021-02-09 09:56:56 +00:00
18327be5d4 nat: 1:1 policy NAT
A NAT sub-plugin doing statically configured match/rewrite on IP4 input or output.
It's stateless (no connection tracking).
Currently it supports rewriting of SA, DA and TCP/UDP ports.
It should be simple to add new rewrites if required.

API:
pnat_binding_add, pnat_binding_del, pnat_bindings_get, pnat_interfaces_get

CLI:
set pnat translation interface <name> match <5-tuple> rewrite <5-tuple> {in|out} [del]
show pnat translations
show pnat interfaces

Trying a new C based unit testing scheme. Where the graph node is tested
in isolation. See pnat/pnat_test.c.
Also added new cmake targets to generate coverage directly.
E.g.:
make test_pnat-ccov-report

File '/vpp/sdnat/src/plugins/nat/pnat/pnat.c':
Name                               Regions    Miss   Cover     Lines    Miss   Cover
------------------------------------------------------------------------------------
pnat_interface_by_sw_if_index           39       8  79.49%        13       0 100.00%
pnat_instructions_from_mask              9       0 100.00%        13       0 100.00%
pnat_binding_add                        64       8  87.50%        31       2  93.55%
pnat_flow_lookup                         4       4   0.00%        10      10   0.00%
pnat_binding_attach                    104      75  27.88%        33       6  81.82%
pnat_binding_detach                     30       5  83.33%        23       2  91.30%
pnat_binding_del                        97      33  65.98%        17       3  82.35%
pnat.c:pnat_calc_key_from_5tuple         9       1  88.89%        14       1  92.86%
pnat.c:pnat_interface_check_mask        10       2  80.00%        11       2  81.82%
pnat.c:pnat_enable                       5       0 100.00%        11       0 100.00%
pnat.c:pnat_enable_interface           107      26  75.70%        60      15  75.00%
pnat.c:pnat_disable_interface           91      30  67.03%        32       7  78.12%
pnat.c:pnat_disable                      7       2  71.43%        13       7  46.15%
------------------------------------------------------------------------------------
TOTAL                                  576     194  66.32%       281      55  80.43%

File '/vpp/sdnat/src/plugins/nat/pnat/pnat_node.h':
Name                               Regions    Miss   Cover     Lines    Miss   Cover
------------------------------------------------------------------------------------
pnat_test.c:pnat_node_inline            67      11  83.58%       115       1  99.13%
pnat_test.c:pnat_calc_key                9       2  77.78%        14       2  85.71%
pnat_test.c:pnat_rewrite_ip4            55      11  80.00%        60      12  80.00%
pnat_test.c:format_pnat_trace            1       1   0.00%        12      12   0.00%
pnat_node.c:pnat_node_inline            63      63   0.00%       115     115   0.00%
pnat_node.c:pnat_calc_key                9       9   0.00%        14      14   0.00%
pnat_node.c:pnat_rewrite_ip4            55      55   0.00%        60      60   0.00%
pnat_node.c:format_pnat_trace            5       5   0.00%        12      12   0.00%
------------------------------------------------------------------------------------
TOTAL                                  264     157  40.53%       402     228  43.28%

Type: feature
Change-Id: I9c897f833603054a8303e7369ebff6512517c9e0
Signed-off-by: Ole Troan <ot@cisco.com>
2021-02-05 13:27:48 +00:00
9302cfea9d vppapigen: Support an 'autoendian' keyword for message definitions in
.api files

Type: feature

Make the auto-endian nature explicit, rather than hidden in the x_api.c
file.

Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: Ibe647117ceeaf6f99a38a96576a5a41a3cbb1615
2021-02-03 15:09:50 +00:00