ligang/vpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: document "make test-help"

Browse Source 
Type: docs

Signed-off-by: Dave Barach <dave@barachs.net>
Change-Id: I67a582b7bf16c3fe307941973e983a0fe28cb221
This commit is contained in:
Dave Barach
2021-04-20 07:52:38 -04:00
committed by Dave Wallace
parent 5420885978
commit 9292c32b45
3 changed files with 208 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/gettingstarted/developers/index.rst
View File

@ -20,6 +20,7 @@ The Developers section covers the following areas:
building building
running_vpp running_vpp
testing_vpp
gdb_examples gdb_examples
add_plugin add_plugin
add_plugin_goapi add_plugin_goapi

141
docs/gettingstarted/developers/testing_vpp.md Normal file
View File

@ -0,0 +1,141 @@
Testing VPP
===========
As of this writing, the vpp source tree includes over 1,000 unit test
vectors. Best practices prior to pushing patches for code review: make
sure that all of the "make test" test vectors pass.
We attempt to maintain the top-level "make test-help" command so that it
accurately describes all of the "make test" options.
Examples
--------
Basic test run, all test vectors, single-vpp instance, optimized image:
```
$ make test
```
10-way parallel basic test run:
```
$ make TEST_JOBS=10 test
```
Run a specific test suite (mpls, in this case):
```
$ make TEST=test_mpls test
```
Run a specifc test suite, debug image, pause prior to running the test
suite; attach to the vpp image in gdb:
```
$ make TEST=xxx DEBUG=gdb test-debug
```
Detailed Documentation
----------------------
Current "make test-help" output:
```
$ make test-help
test - build and run (basic) functional tests
test-debug - build and run (basic) functional tests (debug build)
test-all - build and run functional and extended tests
test-all-debug - build and run functional and extended tests (debug build)
retest - run functional tests
retest-debug - run functional tests (debug build)
retest-all - run functional and extended tests
retest-all-debug - run functional and extended tests (debug build)
test-cov - generate code coverage report for test framework
test-gcov - build and run functional tests (gcov build)
test-wipe - wipe (temporary) files generated by unit tests
test-wipe-cov - wipe code coverage report for test framework
test-wipe-doc - wipe documentation for test framework
test-wipe-papi - rebuild vpp_papi sources
test-wipe-all - wipe (temporary) files generated by unit tests, docs, and coverage
test-shell - enter shell with test environment
test-shell-debug - enter shell with test environment (debug build)
test-checkstyle - check PEP8 compliance for test framework
test-refresh-deps - refresh the Python dependencies for the tests
Arguments controlling test runs:
V=[0|1|2] - set test verbosity level
0=ERROR, 1=INFO, 2=DEBUG
TEST_JOBS=[<n>|auto] - use at most <n> parallel python processes for test execution, if auto, set to number of available cpus (default: 1)
MAX_VPP_CPUS=[<n>|auto]- use at most <n> cpus for running vpp main and worker threads, if auto, set to number of available cpus (default: auto)
CACHE_OUTPUT=[0|1] - cache VPP stdout/stderr and log as one block after test finishes (default: 1)
FAILFAST=[0|1] - fail fast if 1, complete all tests if 0
TIMEOUT=<timeout> - fail test suite if any single test takes longer than <timeout> (in seconds) to finish (default: 600)
RETRIES=<n> - retry failed tests <n> times
DEBUG=<type> - set VPP debugging kind
DEBUG=core - detect coredump and load it in gdb on crash
DEBUG=gdb - allow easy debugging by printing VPP PID
and waiting for user input before running
and tearing down a testcase
DEBUG=gdbserver - run gdb inside a gdb server, otherwise
same as above
DEBUG=attach - attach test case to already running vpp in gdb (see test-start-vpp-in-gdb)
STEP=[yes|no] - ease debugging by stepping through a testcase
SANITY=[yes|no] - perform sanity import of vpp-api/sanity vpp run before running tests (default: yes)
EXTENDED_TESTS=[1|y] - used by '[re]test-all' & '[re]test-all-debug' to run extended tests
TEST=<filter> - filter the set of tests:
by file-name - only run tests from specified file, e.g. TEST=test_bfd selects all tests from test_bfd.py
by file-suffix - same as file-name, but 'test_' is omitted e.g. TEST=bfd selects all tests from test_bfd.py
by wildcard - wildcard filter is <file>.<class>.<test function>, each can be replaced by '*'
e.g. TEST='test_bfd.*.*' is equivalent to above example of filter by file-name
TEST='bfd.*.*' is equivalent to above example of filter by file-suffix
TEST='bfd.BFDAPITestCase.*' selects all tests from test_bfd.py which are part of BFDAPITestCase class
TEST='bfd.BFDAPITestCase.test_add_bfd' selects a single test named test_add_bfd from test_bfd.py/BFDAPITestCase
TEST='*.*.test_add_bfd' selects all test functions named test_add_bfd from all files/classes
VARIANT=<variant> - specify which march node variant to unit test
e.g. VARIANT=skx test the skx march variants
e.g. VARIANT=icl test the icl march variants
COREDUMP_SIZE=<size> - pass <size> as unix { coredump-size <size> } argument to vpp
e.g. COREDUMP_SIZE=4g
COREDUMP_SIZE=unlimited
COREDUMP_COMPRESS=1 - compress core files if not debugging them
EXTERN_TESTS=<path> - path to out-of-tree test_<name>.py files containing test cases
EXTERN_PLUGINS=<path> - path to out-of-tree plugins to be loaded by vpp under test
EXTERN_COV_DIR=<path> - path to out-of-tree prefix, where source, object and .gcda files can be found for coverage report
PROFILE=1 - enable profiling of test framework via cProfile module
PROFILE_SORT_BY=opt - sort profiling report by opt - consult cProfile documentation for possible values (default: cumtime)
PROFILE_OUTPUT=file - output profiling info to file - use absolute path (default: stdout)
TEST_DEBUG=1 - turn on debugging of the test framework itself (expert)
SKIP_AARCH64=1 - skip tests that are failing on the ARM platorm in FD.io CI
RND_SEED=seed - Seed RND with given seed
Starting VPP in GDB for use with DEBUG=attach:
test-start-vpp-in-gdb - start VPP in gdb (release)
test-start-vpp-debug-in-gdb - start VPP in gdb (debug)
Arguments controlling VPP in GDB runs:
VPP_IN_GDB_TMP_DIR - specify directory to run VPP IN (default: /tmp/unittest-attach-gdb)
VPP_IN_GDB_NO_RMDIR=0 - don't remove existing tmp dir but fail instead
VPP_IN_GDB_CMDLINE=1 - add 'interactive' to VPP arguments to run with command line
Creating test documentation
test-doc - generate documentation for test framework
test-wipe-doc - wipe documentation for test framework
Creating test code coverage report
test-cov - generate code coverage report for test framework
test-wipe-cov - wipe code coverage report for test framework
Verifying code-style
test-checkstyle - check PEP8 compliance
```

132
docs/usecases/container_test.md
View File

@ -64,9 +64,9 @@ Create three network segments
Aka, linux bridges. Aka, linux bridges.
``` ```
# lxc network create dora # lxc network create respond
# lxc network create internet # lxc network create internet
# lxc network create swan # lxc network create initiate
``` ```
We'll explain the test topology in a bit. Stay tuned. We'll explain the test topology in a bit. Stay tuned.
@ -95,12 +95,12 @@ containers, run vpp without installing it, etc.
eth2: eth2:
name: eth2 name: eth2
nictype: bridged nictype: bridged
parent: dora parent: respond
type: nic type: nic
eth3: eth3:
name: eth3 name: eth3
nictype: bridged nictype: bridged
parent: swan parent: initiate
type: nic type: nic
root: root:
path: / path: /
@ -142,10 +142,10 @@ and host configuration:
- none - none
``` ```
Repeat the process with the "dora" and "swan" networks, using these Repeat the process with the "respond" and "initiate" networks, using these
configurations: configurations:
### dora network configuration ### respond network configuration
``` ```
config: config:
@ -155,7 +155,7 @@ configurations:
ipv6.address: none ipv6.address: none
ipv6.nat: "false" ipv6.nat: "false"
description: "" description: ""
name: dora name: respond
type: bridge type: bridge
used_by: used_by:
managed: true managed: true
@ -163,7 +163,7 @@ configurations:
locations: locations:
- none - none
``` ```
### swan network configuration ### initiate network configuration
``` ```
config: config:
@ -173,7 +173,7 @@ configurations:
ipv6.address: none ipv6.address: none
ipv6.nat: "false" ipv6.nat: "false"
description: "" description: ""
name: swan name: initiate
type: bridge type: bridge
used_by: used_by:
managed: true managed: true
@ -191,14 +191,14 @@ build vpp, ssh into the container, edit source code, run gdb, etc.
Make sure that e.g. public key auth ssh works. Make sure that e.g. public key auth ssh works.
``` ```
# lxd launch ubuntu:18.04 dora # lxd launch ubuntu:18.04 respond
<spew> <spew>
# lxc exec dora bash # lxc exec respond bash
dora# cd /scratch/my-vpp-workspace respond# cd /scratch/my-vpp-workspace
dora# apt-get install make ssh respond# apt-get install make ssh
dora# make install-dep respond# make install-dep
dora# exit respond# exit
# lxc stop dora # lxc stop respond
``` ```
Mark the container image privileged. If you forget this step, you'll Mark the container image privileged. If you forget this step, you'll
@ -206,7 +206,7 @@ trip over a netlink error (-11) aka EAGAIN when you try to roll in the
vpp configurations. vpp configurations.
``` ```
# lxc config set dora security.privileged "true" # lxc config set respond security.privileged "true"
``` ```
Duplicate the "master" container image Duplicate the "master" container image
@ -216,10 +216,10 @@ To avoid having to configure N containers, be sure that the master
container image is fully set up before you help it have children: container image is fully set up before you help it have children:
``` ```
# lxc copy dora dorahost # lxc copy respond respondhost
# lxc copy dora swan # lxc copy respond initiate
# lxc copy dora swanhost # lxc copy respond initiatehost
# lxc copy dora dhcpserver # optional, to test ipv6 prefix delegation # lxc copy respond dhcpserver # optional, to test ipv6 prefix delegation
``` ```
Install handy script Install handy script
@ -250,7 +250,7 @@ Here's the script:
#!/bin/bash #!/bin/bash
set -u set -u
export containers="dora dorahost swan swanhost dhcpserver" export containers="respond respondhost initiate initiatehost dhcpserver"
if [ x$1 = "x" ] ; then if [ x$1 = "x" ] ; then
echo missing command echo missing command
@ -292,16 +292,16 @@ Finally, we're ready to describe a test topology. First, a picture:
v | v v | v
eth0 | eth0 eth0 | eth0
+------+ eth1 eth1 +------+ +------+ eth1 eth1 +------+
| dora | 10.26.88.100 <= internet bridge => 10.26.88.101 | swan | | respond | 10.26.88.100 <= internet bridge => 10.26.88.101 | initiate |
+------+ +------+ +------+ +------+
eth2 / bvi0 10.166.14.2 | 10.219.188.2 eth3 / bvi0 eth2 / bvi0 10.166.14.2 | 10.219.188.2 eth3 / bvi0
| | | | | |
| ("dora" bridge) | ("swan" bridge) | | ("respond" bridge) | ("initiate" bridge) |
| | | | | |
v | v v | v
eth2 10.166.14.3 | eth3 10.219.188.3 eth2 10.166.14.3 | eth3 10.219.188.3
+----------+ | +----------+ +----------+ | +----------+
| dorahost | | | dorahost | | respondhost | | | respondhost |
+----------+ | +----------+ +----------+ | +----------+
eth0 (management lan) <========+========> eth0 (management lan) eth0 (management lan) <========+========> eth0 (management lan)
``` ```
@ -309,14 +309,14 @@ Finally, we're ready to describe a test topology. First, a picture:
### Test topology discussion ### Test topology discussion
This topology is suitable for testing almost any tunnel encap/decap This topology is suitable for testing almost any tunnel encap/decap
scenario. The two containers "dorahost" and "swanhost" are end-stations scenario. The two containers "respondhost" and "initiatehost" are end-stations
connected to two vpp instances running on "dora" and "swan". connected to two vpp instances running on "respond" and "initiate".
We leverage the Linux end-station network stacks to generate traffic We leverage the Linux end-station network stacks to generate traffic
of all sorts. of all sorts.
The so-called "internet" bridge models the public internet. The "dora" and The so-called "internet" bridge models the public internet. The "respond" and
"swan" bridges connect vpp instances to local hosts "initiate" bridges connect vpp instances to local hosts
End station configs End station configs
------------------- -------------------
@ -325,14 +325,14 @@ The end-station Linux configurations set up the eth2 and eth3 ip
addresses shown above, and add tunnel routes to the opposite addresses shown above, and add tunnel routes to the opposite
end-station networks. end-station networks.
### dorahost configuration ### respondhost configuration
``` ```
ifconfig eth2 10.166.14.3/24 up ifconfig eth2 10.166.14.3/24 up
route add -net 10.219.188.0/24 gw 10.166.14.2 route add -net 10.219.188.0/24 gw 10.166.14.2
``` ```
### swanhost configuration ### initiatehost configuration
``` ```
sudo ifconfig eth3 10.219.188.3/24 up sudo ifconfig eth3 10.219.188.3/24 up
@ -343,9 +343,9 @@ VPP configs
----------- -----------
Split nat44 / ikev2 + ipsec tunneling, with ipv6 prefix delegation in Split nat44 / ikev2 + ipsec tunneling, with ipv6 prefix delegation in
the "dora" config. the "respond" config.
### dora configuration ### respond configuration
``` ```
set term pag off set term pag off
@ -356,10 +356,10 @@ the "dora" config.
set int ip6 table host-eth1 0 set int ip6 table host-eth1 0
set int state host-eth1 up set int state host-eth1 up
comment { default route via swan } comment { default route via initiate }
ip route add 0.0.0.0/0 via 10.26.68.101 ip route add 0.0.0.0/0 via 10.26.68.101
comment { "dora-private-net" } comment { "respond-private-net" }
create host-interface name eth2 create host-interface name eth2
bvi create instance 0 bvi create instance 0
set int l2 bridge bvi0 1 bvi set int l2 bridge bvi0 1 bvi
@ -376,16 +376,16 @@ the "dora" config.
comment { nat44 untranslated subnet 10.219.188.0/24 } comment { nat44 untranslated subnet 10.219.188.0/24 }
comment { responder profile } comment { responder profile }
ikev2 profile add swan ikev2 profile add initiate
ikev2 profile set swan udp-encap ikev2 profile set initiate udp-encap
ikev2 profile set swan auth rsa-sig cert-file /scratch/setups/doracert.pem ikev2 profile set initiate auth rsa-sig cert-file /scratch/setups/respondcert.pem
set ikev2 local key /scratch/setups/swankey.pem set ikev2 local key /scratch/setups/initiatekey.pem
ikev2 profile set swan id local fqdn swan.barachs.net ikev2 profile set initiate id local fqdn initiator.my.net
ikev2 profile set swan id remote fqdn broiler2.barachs.net ikev2 profile set initiate id remote fqdn responder.my.net
ikev2 profile set swan traffic-selector remote ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0 ikev2 profile set initiate traffic-selector remote ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0
ikev2 profile set swan traffic-selector local ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0 ikev2 profile set initiate traffic-selector local ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0
create ipip tunnel src 10.26.68.100 dst 10.26.68.101 create ipip tunnel src 10.26.68.100 dst 10.26.68.101
ikev2 profile set swan tunnel ipip0 ikev2 profile set initiate tunnel ipip0
comment { ipv6 prefix delegation } comment { ipv6 prefix delegation }
ip6 nd address autoconfig host-eth1 default-route ip6 nd address autoconfig host-eth1 default-route
@ -400,21 +400,21 @@ the "dora" config.
ip route add 10.219.188.0/24 via ipip0 ip route add 10.219.188.0/24 via ipip0
``` ```
### swan configuration ### initiate configuration
``` ```
set term pag off set term pag off
comment { "internet" } comment { "internet" }
create host-interface name eth1 create host-interface name eth1
comment { set dhcp client intfc host-eth1 hostname swan } comment { set dhcp client intfc host-eth1 hostname initiate }
set int ip address host-eth1 10.26.68.101/24 set int ip address host-eth1 10.26.68.101/24
set int state host-eth1 up set int state host-eth1 up
comment { default route via "internet gateway" } comment { default route via "internet gateway" }
comment { ip route add 0.0.0.0/0 via 10.26.68.1 } comment { ip route add 0.0.0.0/0 via 10.26.68.1 }
comment { "swan-private-net" } comment { "initiate-private-net" }
create host-interface name eth3 create host-interface name eth3
bvi create instance 0 bvi create instance 0
set int l2 bridge bvi0 1 bvi set int l2 bridge bvi0 1 bvi
@ -430,24 +430,24 @@ the "dora" config.
comment { nat44 untranslated subnet 10.166.14.0/24 } comment { nat44 untranslated subnet 10.166.14.0/24 }
comment { initiator profile } comment { initiator profile }
ikev2 profile add dora ikev2 profile add respond
ikev2 profile set dora udp-encap ikev2 profile set respond udp-encap
ikev2 profile set dora auth rsa-sig cert-file /scratch/setups/swancert.pem ikev2 profile set respond auth rsa-sig cert-file /scratch/setups/initiatecert.pem
set ikev2 local key /scratch/setups/dorakey.pem set ikev2 local key /scratch/setups/respondkey.pem
ikev2 profile set dora id local fqdn broiler2.barachs.net ikev2 profile set respond id local fqdn responder.my.net
ikev2 profile set dora id remote fqdn swan.barachs.net ikev2 profile set respond id remote fqdn initiator.my.net
ikev2 profile set dora traffic-selector remote ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0 ikev2 profile set respond traffic-selector remote ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0
ikev2 profile set dora traffic-selector local ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0 ikev2 profile set respond traffic-selector local ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0
ikev2 profile set dora responder host-eth1 10.26.68.100 ikev2 profile set respond responder host-eth1 10.26.68.100
ikev2 profile set dora ike-crypto-alg aes-cbc 256 ike-integ-alg sha1-96 ike-dh modp-2048 ikev2 profile set respond ike-crypto-alg aes-cbc 256 ike-integ-alg sha1-96 ike-dh modp-2048
ikev2 profile set dora esp-crypto-alg aes-cbc 256 esp-integ-alg sha1-96 esp-dh ecp-256 ikev2 profile set respond esp-crypto-alg aes-cbc 256 esp-integ-alg sha1-96 esp-dh ecp-256
ikev2 profile set dora sa-lifetime 3600 10 5 0 ikev2 profile set respond sa-lifetime 3600 10 5 0
create ipip tunnel src 10.26.68.101 dst 10.26.68.100 create ipip tunnel src 10.26.68.101 dst 10.26.68.100
ikev2 profile set dora tunnel ipip0 ikev2 profile set respond tunnel ipip0
ikev2 initiate sa-init dora ikev2 initiate sa-init respond
set int mtu packet 1390 ipip0 set int mtu packet 1390 ipip0
set int unnum ipip0 use host-eth1 set int unnum ipip0 use host-eth1
@ -463,13 +463,13 @@ mentioned. These certificates are used in the ikev2 key exchange.
Here's how to generate the certificates: Here's how to generate the certificates:
``` ```
openssl req -x509 -nodes -newkey rsa:4096 -keyout dorakey.pem -out doracert.pem -days 3560 openssl req -x509 -nodes -newkey rsa:4096 -keyout respondkey.pem -out respondcert.pem -days 3560
openssl x509 -text -noout -in doracert.pem openssl x509 -text -noout -in respondcert.pem
openssl req -x509 -nodes -newkey rsa:4096 -keyout swankey.pem -out swancert.pem -days 3560 openssl req -x509 -nodes -newkey rsa:4096 -keyout initiatekey.pem -out initiatecert.pem -days 3560
openssl x509 -text -noout -in swancert.pem openssl x509 -text -noout -in initiatecert.pem
``` ```
Make sure that the "dora" and "swan" configurations point to the certificates. Make sure that the "respond" and "initiate" configurations point to the certificates.
DHCPv6 server setup DHCPv6 server setup
------------------- -------------------