This commit allow use od cooperative multitasking with multiple
descheduling reasons (i.e. event wait and suspend) inside the same
process node. In previus code remote node will wake up process node
by sending event evein if process node is waiting in
vlib_process_ssupend().
This change also allowed new vlib_process_yield() API which deschedules
current process and it puts it into the end of queue.
Change-Id: I846e5a99b4ea1809eb80895f6ffe0ef0b2fd21ae
Type: improvement
Signed-off-by: Damjan Marion <damarion@cisco.com>
The VPP API auto-generated endian conversion functions are intended to
be symmetrical. They are used both by the API client and the API server.
Called on send to convert from host endian to network endian and on
receive to convert back.
For variable length arrays, we have to iterate over the array and call
a more specific handler for the array type. Unfortunately the length of
the array is part of the api definition, and if it's endian swapped
prior to the for loop, unexpected behaviour will ensue.
There was an earlier fix, for some specific messages, but unfortunately
that only fixed the problem from the VPP (server) side.
This adds a new parameters to the endian handler, so the boundary
argument to the loop can be treated differently depending on if this
message is to the network or from the network.
Type: fix
Change-Id: I43011aed384e3b847579a1dd2c390867ae17a9ad
Signed-off-by: Ole Troan <otroan@employees.org>
This patch allows a client to bootstrap itself by downloading the
JSON API definitions over the API itself.
This patch enables it for Python (probably need a dynamic language).
Call VPPApiClient with the new bootstrapapi=True parameter.
Example (Python):
from vpp_papi import VPPApiClient
vpp = VPPApiClient(bootstrapapi=True)
rv = vpp.connect("foobar")
assert rv == 0
print(f'SHOW VERSION: {vpp.api.show_version()}')
vpp.disconnect()
Type: feature
Change-Id: Id903fdccc82b2e22aa1994331d2c150253f2ccae
Signed-off-by: Ole Troan <otroan@employees.org>
clang-16 complains about signed one-bit bitfield value
changes from 1 to -1. Use unsigned type instead.
Type: fix
Signed-off-by: Tianyu Li <tianyu.li@arm.com>
Change-Id: Iab006c036d4068af2fe3caaefc871a95b26cc578
- remove non-inclusive language in message and improve
clarity of the error message
Type: style
Change-Id: I3f4895d6a502c2583a8b6b3c325a3f30ced03f84
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
single struct to hold all api handler, flags, etc.
Provide functions to toggle flags instead of writing directly to
internal data.
Type: refactor
Change-Id: I4730d7290e57489de8eda34a72211527e015b721
Signed-off-by: Damjan Marion <damarion@cisco.com>
Add vapi_connect_from_vpp() and vapi_disconnect_from_vpp()
calls to allow API clients from within VPP process.
Add a new memclnt_create version that gives the user a
knob to enable or disable dead client scans (keepalive).
Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Id0b7bb89308db3a3aed2d3fcbedf4e1282dcd03f
Signed-off-by: Ole Troan <ot@cisco.com>
Type: improvement
Short message lengths are correctly handled by the code, asserting makes unit tests that verify this behaviour (e.g. test_ip_punt_api_validation) fail/crash with a debug image.
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: Ib3a377e5d49fde0eee252b92f0e58a8a0d0d83ec
Use of _vec_len() to set vector length breaks address sanitizer.
Users should use vec_set_len(), vec_inc_len(), vec_dec_len () instead.
Type: improvement
Change-Id: I441ae948771eb21c23a61f3ff9163bdad74a2cb8
Signed-off-by: Damjan Marion <damarion@cisco.com>
- move memory and server specific vl_msg_api_handler_with_vm_node()
to memory server code only
- keep api_global_main static
Apart from being cleaner, this also helps avoiding symbols conflict
when both client and server libs are loaded in the same process, as is
done by the prom plugin.
Those symbols conflict confuse ASan and can be nasty to debug.
Type: improvement
Change-Id: Iaf58596cc753ad8d3fedd8d65c4bf480ac129c2c
Signed-off-by: Benoît Ganne <bganne@cisco.com>
When a message is received, verify that it's sufficiently large to
accomodate any VLAs within message. To do that, we need a way to
calculate message size including any VLAs. This patch adds such
funcionality to vppapigen and necessary C code to use those to validate
message size on receipt. Drop messages which are malformed.
Type: improvement
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: I2903aa21dee84be6822b064795ba314de46c18f4
Improve vppapigen to generate per-message #define indicating whether
said message is dynamically sized (due to VLA or string) or not. Use
these #defines in REPLY_MACROs to prevent improper usage. Fix existing
improper REPLY_MACRO* usage.
Type: improvement
Change-Id: Ia77aaf9f6cf3ed68ea21075a4cc8deda78a68651
Signed-off-by: Klement Sekera <ksekera@cisco.com>
While looking at a coverity warning it was discovered that it's caused
by return value modified by pd msg handlers. As these are legacy, it's
a good time to remove them and also fix the warning this way.
Type: fix
Signed-off-by: Klement Sekera <ksekera@cisco.com>
Change-Id: Ic72ab8b2b7a2e55188d1c31cfd18a74b7cf82c43
Type: improvement
* add support for JSON format in API trace
* add ability to replay JSON API trace in both VPP and VAT2
* use CRC for backward compatibility check during JSON API replay
* fix API trace CLI (and remove duplicits)
* remove custom dump
* remove vppapitrace.py
* update docs accordingly
Change-Id: I5294f68bebe6cbe738630f457f3a87720e06486b
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
Signed-off-by: Ole Troan <ot@cisco.com>
VPE apis are actually vlib apis. This moves those that are not tightly
coupled with vapi to vlib_api
Type: refactor
Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: I456a64ce49a0cdeff4a0931c6ea513cb639f683e
Signed-off-by: Ole Troan <ot@cisco.com>
This issue happens if:
- the API client connects via Unix socket
- the client issues the *_dump API call and immediately disconnects
What happens after is that the API handler keeps sending the *_details
messages, however at some point the write fails, and the socket is
deleted.
The attempt of a use of the registration pointer results in interpreting
the socket as a shared memory socket. This results in a crash, because
the data in this structure then does not make sense, like the below:
|
|Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.
|__GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:67
|67 ../nptl/pthread_mutex_lock.c: No such file or directory.
|(gdb) bt
|#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:67
|#1 0x00007ffff500f957 in svm_queue_lock (q=0x0) at /home/ubuntu/vpp/src/svm/queue.c:101
|#2 svm_queue_add (q=0x0, elem=0x7fffa76c2de0 "\210\365\006\060\001", nowait=0) at /home/ubuntu/vpp/src/svm/queue.c:274
|#3 0x00007ffff6e131e3 in vl_api_send_msg (rp=<optimized out>, elem=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/api.h:43
|#4 send_sw_interface_details (am=<optimized out>, rp=<optimized out>, swif=0x7fffb957a0bc, interface_name=<optimized out>, context=<optimized out>)
| at /home/ubuntu/vpp/src/vnet/interface_api.c:353
|#5 0x00007ffff6e0edeb in vl_api_sw_interface_dump_t_handler (mp=<optimized out>) at /home/ubuntu/vpp/src/vnet/interface_api.c:412
|#6 0x00007ffff7daeb48 in msg_handler_internal (am=<optimized out>, the_msg=0x7fffb839a5e0, trace_it=<optimized out>, do_it=1, free_it=0)
| at /home/ubuntu/vpp/src/vlibapi/api_shared.c:501
|#7 vl_msg_api_socket_handler (the_msg=0x7fffb839a5e0) at /home/ubuntu/vpp/src/vlibapi/api_shared.c:790
|#8 0x00007ffff7d7c608 in vl_socket_process_api_msg (rp=<optimized out>, input_v=0x7fffa76c2de0 "\210\365\006\060\001") at /home/ubuntu/vpp/src/vlibmemory/socket_api.c:212
|#9 0x00007ffff7d89ff1 in vl_api_clnt_process (vm=<optimized out>, node=<optimized out>, f=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/vlib_api.c:405
|#10 0x00007ffff53bf9a7 in vlib_process_bootstrap (_a=<optimized out>) at /home/ubuntu/vpp/src/vlib/main.c:1490
|#11 0x00007ffff4da0b2c in clib_calljmp () from /home/ayourtch/vpp/build-root/install-vpp-native/vpp/lib/libvppinfra.so.21.06
|#12 0x00007fffa99a4d90 in ?? ()
|#13 0x00007ffff53b6cb2 in vlib_process_startup (vm=0x7ffff56a9880 <vlib_global_main>, p=0x7fffb5d41380, f=0x0) at /home/ubuntu/vpp/src/vlib/main.c:1515
|#14 dispatch_process (vm=0x7ffff56a9880 <vlib_global_main>, p=0x7fffb5d41380, f=0x0, last_time_stamp=<optimized out>) at /home/ubuntu/vpp/src/vlib/main.c:1571
|#15 0x0000000000000000 in ?? ()
|(gdb) frame 3
|#3 0x00007ffff6e131e3 in vl_api_send_msg (rp=<optimized out>, elem=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/api.h:43
|43 vl_msg_api_send_shmem (rp->vl_input_queue, (u8 *) & elem);
|(gdb) l
|38 {
|39 vl_socket_api_send (rp, elem);
|40 }
|41 else
|42 {
|43 vl_msg_api_send_shmem (rp->vl_input_queue, (u8 *) & elem);
|44 }
|45 }
|46
|47 always_inline int
|(gdb)
|
The approach in this change is to avoid the closing operations "here and
now", but instead mark the the registration as a zombie and place
a forced RPC towards a callback that does the actual cleanup work.
Forced RPC is handled via the API processing loop with barrier sync,
so we are guaranteed not to have any API processing in-process.
Type: fix
Change-Id: I1972d42da620bdb4fd773c83262863c2781d9005
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Sending 128 bytes of zeroes via API shared memory interface (or putting any other message ID that results in
null message name - e.g. 2,4,5) results in this crash:
Program received signal SIGSEGV, Segmentation fault.
va_format (s=0x7fffb5643a60 "memclnt_create", fmt=0x0, va=0x7fffa739ccf0) at /home/ubuntu/vpp/src/vppinfra/format.c:395
395 c = *f;
(gdb) bt
0 va_format (s=0x7fffb5643a60 "memclnt_create", fmt=0x0, va=0x7fffa739ccf0) at /home/ubuntu/vpp/src/vppinfra/format.c:395
1 0x00007ffff50e7259 in elog_string (em=0x7ffff5f2fbc8 <vlib_global_main+968>, fmt=0x0) at /home/ubuntu/vpp/src/vppinfra/elog.c:592
2 0x00007ffff7bc4cec in vl_msg_api_handler_with_vm_node (am=0x7ffff7dd1c90 <api_global_main>, vlib_rp=0x130023000, the_msg=0x130086d50, vm=0x7ffff5f2f800 <vlib_global_main>,
node=0x7fffb5edd4c0, is_private=0 '\000') at /home/ubuntu/vpp/src/vlibapi/api_shared.c:585
3 0x00007ffff7ba6c86 in void_mem_api_handle_msg_i (am=0x7ffff7dd1c90 <api_global_main>, vlib_rp=0x130023000, vm=<optimized out>, node=<optimized out>, is_private=0 '\000')
at /home/ubuntu/vpp/src/vlibmemory/memory_api.c:696
4 vl_mem_api_handle_msg_main (vm=0x7ffff5f2f800 <vlib_global_main>, node=0x7fffb5edd4c0) at /home/ubuntu/vpp/src/vlibmemory/memory_api.c:707
5 0x00007ffff7bb573e in vl_api_clnt_process (vm=0x7ffff5f2f800 <vlib_global_main>, node=0x7fffb5edd4c0, f=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/vlib_api.c:338
6 0x00007ffff5cb3bb7 in vlib_process_bootstrap (_a=<optimized out>) at /home/ubuntu/vpp/src/vlib/main.c:1477
7 0x00007ffff514088c in clib_calljmp () from /home/ubuntu/vpp/build-root/install-vpp-native/vpp/lib/libvppinfra.so.21.01
8 0x00007fffa95b0550 in ?? ()
9 0x00007ffff5ca93e2 in vlib_process_startup (vm=0x7ffff5f2f800 <vlib_global_main>, p=0x7fffb5edd4c0, f=0x0) at /home/ubuntu/vpp/src/vlib/main.c:1502
10 dispatch_process (vm=0x7ffff5f2f800 <vlib_global_main>, p=0x7fffb5edd4c0, f=0x0, last_time_stamp=<optimized out>) at /home/ubuntu/vpp/src/vlib/main.c:1558
11 0x0000000000000000 in ?? ()
(gdb)
I am not sure if this place is better to fix it or the 4-5 places in vl_msg_api_handler_with_vm_node that it gets called from,
but submitting this one since it is the shortest
Type: fix
Change-Id: I659d2bea7405d8763181336f35ef468682f64cf2
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Type: feature
please see FEATURE.yaml for details.
Signed-off-by: Neale Ranns <nranns@cisco.com>
Signed-off-by: Matthew Smith <mgsmith@netgate.com>
Signed-off-by: Jon Loeliger <jdl@netgate.com>
Signed-off-by: Pim van Pelt <pim@ipng.nl>
Change-Id: I04a45c15c0838906aa787e06660fa29f39f755fa
Type: feature
Add API message for an API client to subscribe/unsubscribe to receive
an event when a VRRP VR changes state. Add code to build and send the
events.
Change-Id: Ie92cadd4850d4352c1aaa79c4b0a7daa0f3b04e7
Signed-off-by: Matthew Smith <mgsmith@netgate.com>
Also spiffed up the vpp_api_test plugin loader so it executes
VLIB_INIT_FUNCTIONs and VLIB_API_INIT_FUNCTIONs.
Type: feature
Change-Id: Id9a4f455d73738c41bcfea220df2112bb9679681
Signed-off-by: Jon Loeliger <jdl@netgate.com>
Signed-off-by: Ole Troan <ot@cisco.com>
Signed-off-by: Dave Barach <dave@barachs.net>
Endian handlers was not registered for reply messages.
Causing endian-neutral handlers to crash.
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Id14173300ee1ab2601e92d58c70d2fa260814b69
Callbacks for monitoring and performance measurement:
- Add new callback list type, with context
- Add callbacks for API, CLI, and barrier sync
- Modify node dispatch callback to pass plugin-specific context
- Modify perfmon plugin to keep PMC samples local to the plugin
- Include process nodes in dispatch callback
- Pass dispatch function return value to callback
Type: refactor
Signed-off-by: Tom Seidenberg <tseidenb@cisco.com>
Change-Id: I28b06c58490611e08d76ff5b01b2347ba2109b22
For src/tools/g2, src/vlibapi, and src/vlibmemory
Type: docs
Signed-off-by: Dave Barach <dave@barachs.net>
Change-Id: I800c3e3a1ecb011d5e692d78a015e78342706786
If the id is invalid we cannot check whether we must free the message or
not, free it anyway.
Type: fix
Change-Id: Ie4426f601390d1e5e14c739f670e8c1e6e3aaf1e
Signed-off-by: Benoît Ganne <bganne@cisco.com>
Add a new boolean to signal that the API infrastructure should performan any required
endian conversions for the API handler.
am->is_autoendian[mm->msg_id_base + VL_API_MAP_ADD_DOMAIN] = 1;
Similarly add new REPLY_ macros that perform endian conversion.
These changes do not change the on-the-wire encoding of the API messages, and therefore the API CRC is not changed.
Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I7588f8ccb38b2d1e8d85ea17be99bac43f756267
Signed-off-by: Ole Troan <ot@cisco.com>
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal
reply / request pair. As expressed in the below service statement.
Example:
service {
rpc map_domains_gets returns map_domains_get_reply
stream map_domain_details;
};
define map_domains_get
{
u32 client_index;
u32 context;
u32 cursor;
};
define map_domains_get_reply
{
u32 context;
i32 retval;
u32 cursor;
};
To avoid blocking the main thread for too long, the replies are now sent in client message queue size
chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read.
The API handler must also include a "cursor" that is used to the next call to the get function.
API handler example:
REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains,
({
send_domain_details (cursor, rp, mp->context);
}));
The macro starts from cursor and iterates through the pool
until vl_api_process_may_suspend() returns true or the iteration
reaches the end of the list.
Client Example:
cursor = 0
d = []
while True:
rv, details = map_domains_get(cursor=cursor)
d += details
if rv.retval == 0 or rv.retval != -165:
break
cursor = rv.cursor
or the convenience iterator:
for x in vpp.details_iter(vpp.api.map_domains_get):
pass
or
list(details_iter(map_domains_get))
Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e
Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Add a hook to src/vlibapi/api_shared.c to fuzz (screw up) binary API
messages, e.g. by xoring random data into them before processing. We
specifically exempt client connection messages, and inband debug CLI
messages. We step over msg_id, client index, client context, and
sw_if_index. Otherwise, "make test" vectors fail too rapidly to learn
anything.
The goal is to reduce the number of crashes caused to zero. We're
fairly close with this patch.
Add vl_msg_api_max_length(void *mp), which returns the maximum
plausible length for a binary API message.
Use it to hardern vl_api_from_api_to_new_vec(...) which takes an
additional argument - message pointer - so it can verify that
astr->length is sane. If it's not sane, return a u8 *vector of the
form "insane astr->length nnnn\0".
Verify array lengths in vl_api_dhcp6_send_client_message_t_handler(...)
and vl_api_dhcp6_pd_send_client_message_t_handler(...).
Add a fairly effective binary API fuzz hook to the unittest plugin,
and modify the "make test" framework.py to pass "api-fuzz { on|off }"
to enable API fuzzing: "make API_FUZZ=on TEST=xxx test-debug" or similar
Type: improvement
Signed-off-by: Dave Barach <dave@barachs.net>
Change-Id: I0157267652a163c01553d5267620f719cc6c3bde
vat_socket_connect(...) needs to set the api main client index. The M2
helper macro needed to allocate a buffer of size sizeof(*mp) + n, not
sizeof(*mp).
Type: fix
Signed-off-by: Dave Barach <dave@barachs.net>
Change-Id: I7abf35d2ba045d82765bd27f88899287fd351602
Add macros REPLY_MACRO2_ZERO, REPLY_MACRO3_ZERO that zeros the buffer
before the values are set. This way if say, a u8[64] field is
only filled with 10 octets, the previous buffers trailing
contents are not sent.
Type: feature
Change-Id: Ia69cc9aa477f463336483f7556de32476a6f2d51
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
- Remove vl_api_from_api_string
to prevent use of not nul-terminated strings.
- Rename vl_api_from_api_to_vec -> vl_api_from_api_to_new_vec
to imply a new vector is created. NOT nul terminated.
- Add vl_api_from_api_to_new_c_string. Returns
nul terminated string in a new vector.
- Add vl_api_c_string_to_api_string. Convert nul terminated
string to vl_api_string_t
- Add vl_api_vec_to_api_string. Convert NON nul terminated
vector to vl_api_string_t
Type: fix
Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com>
Change-Id: Iadd59b612c0d960a34ad0dd07a9d17f56435c6ea
Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com>
Type: feature
- ip-neighbour: generic neighbour handling; APIs, DBs, event handling,
aging
- arp: ARP protocol implementation
- ip6-nd; IPv6 neighbor discovery implementation; separate ND,
MLD, RA
- ip6-link; manage link-local addresses
- l2-arp-term; events separated from IP neighbours, since they are not
the same.
vnet retains just enough education to perform ND/ARP packet
construction.
arp and ip6-nd to be moved to plugins soon.
Change-Id: I88dedd0006b299344f4c7024a0aa5baa6b9a8bbe
Signed-off-by: Neale Ranns <nranns@cisco.com>
