ligang/vpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: document "make test-help"

Browse Source 
Type: docs

Signed-off-by: Dave Barach <dave@barachs.net>
Change-Id: I67a582b7bf16c3fe307941973e983a0fe28cb221
This commit is contained in:
Dave Barach
2021-04-20 07:52:38 -04:00
committed by Dave Wallace
parent 5420885978
commit 9292c32b45
3 changed files with 208 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/gettingstarted/developers/index.rst
View File

@ -20,6 +20,7 @@ The Developers section covers the following areas:
building
running_vpp
testing_vpp
gdb_examples
add_plugin
add_plugin_goapi

141
docs/gettingstarted/developers/testing_vpp.md Normal file
View File

@ -0,0 +1,141 @@
Testing VPP
===========
As of this writing, the vpp source tree includes over 1,000 unit test
vectors. Best practices prior to pushing patches for code review: make
sure that all of the "make test" test vectors pass.
We attempt to maintain the top-level "make test-help" command so that it
accurately describes all of the "make test" options.
Examples
--------
Basic test run, all test vectors, single-vpp instance, optimized image:
```
$ make test
```
10-way parallel basic test run:
```
$ make TEST_JOBS=10 test
```
Run a specific test suite (mpls, in this case):
```
$ make TEST=test_mpls test
```
Run a specifc test suite, debug image, pause prior to running the test
suite; attach to the vpp image in gdb:
```
$ make TEST=xxx DEBUG=gdb test-debug
```
Detailed Documentation
----------------------
Current "make test-help" output:
```
$ make test-help
test - build and run (basic) functional tests
test-debug - build and run (basic) functional tests (debug build)
test-all - build and run functional and extended tests
test-all-debug - build and run functional and extended tests (debug build)
retest - run functional tests
retest-debug - run functional tests (debug build)
retest-all - run functional and extended tests
retest-all-debug - run functional and extended tests (debug build)
test-cov - generate code coverage report for test framework
test-gcov - build and run functional tests (gcov build)
test-wipe - wipe (temporary) files generated by unit tests
test-wipe-cov - wipe code coverage report for test framework
test-wipe-doc - wipe documentation for test framework
test-wipe-papi - rebuild vpp_papi sources
test-wipe-all - wipe (temporary) files generated by unit tests, docs, and coverage
test-shell - enter shell with test environment
test-shell-debug - enter shell with test environment (debug build)
test-checkstyle - check PEP8 compliance for test framework
test-refresh-deps - refresh the Python dependencies for the tests
Arguments controlling test runs:
V=[0|1|2] - set test verbosity level
0=ERROR, 1=INFO, 2=DEBUG
TEST_JOBS=[<n>|auto] - use at most <n> parallel python processes for test execution, if auto, set to number of available cpus (default: 1)
MAX_VPP_CPUS=[<n>|auto]- use at most <n> cpus for running vpp main and worker threads, if auto, set to number of available cpus (default: auto)
CACHE_OUTPUT=[0|1] - cache VPP stdout/stderr and log as one block after test finishes (default: 1)
FAILFAST=[0|1] - fail fast if 1, complete all tests if 0
TIMEOUT=<timeout> - fail test suite if any single test takes longer than <timeout> (in seconds) to finish (default: 600)
RETRIES=<n> - retry failed tests <n> times
DEBUG=<type> - set VPP debugging kind
DEBUG=core - detect coredump and load it in gdb on crash
DEBUG=gdb - allow easy debugging by printing VPP PID
and waiting for user input before running
and tearing down a testcase
DEBUG=gdbserver - run gdb inside a gdb server, otherwise
same as above
DEBUG=attach - attach test case to already running vpp in gdb (see test-start-vpp-in-gdb)
STEP=[yes|no] - ease debugging by stepping through a testcase
SANITY=[yes|no] - perform sanity import of vpp-api/sanity vpp run before running tests (default: yes)
EXTENDED_TESTS=[1|y] - used by '[re]test-all' & '[re]test-all-debug' to run extended tests
TEST=<filter> - filter the set of tests:
by file-name - only run tests from specified file, e.g. TEST=test_bfd selects all tests from test_bfd.py
by file-suffix - same as file-name, but 'test_' is omitted e.g. TEST=bfd selects all tests from test_bfd.py
by wildcard - wildcard filter is <file>.<class>.<test function>, each can be replaced by '*'
e.g. TEST='test_bfd.*.*' is equivalent to above example of filter by file-name
TEST='bfd.*.*' is equivalent to above example of filter by file-suffix
TEST='bfd.BFDAPITestCase.*' selects all tests from test_bfd.py which are part of BFDAPITestCase class
TEST='bfd.BFDAPITestCase.test_add_bfd' selects a single test named test_add_bfd from test_bfd.py/BFDAPITestCase
TEST='*.*.test_add_bfd' selects all test functions named test_add_bfd from all files/classes
VARIANT=<variant> - specify which march node variant to unit test
e.g. VARIANT=skx test the skx march variants
e.g. VARIANT=icl test the icl march variants
COREDUMP_SIZE=<size> - pass <size> as unix { coredump-size <size> } argument to vpp
e.g. COREDUMP_SIZE=4g
COREDUMP_SIZE=unlimited
COREDUMP_COMPRESS=1 - compress core files if not debugging them
EXTERN_TESTS=<path> - path to out-of-tree test_<name>.py files containing test cases
EXTERN_PLUGINS=<path> - path to out-of-tree plugins to be loaded by vpp under test
EXTERN_COV_DIR=<path> - path to out-of-tree prefix, where source, object and .gcda files can be found for coverage report
PROFILE=1 - enable profiling of test framework via cProfile module
PROFILE_SORT_BY=opt - sort profiling report by opt - consult cProfile documentation for possible values (default: cumtime)
PROFILE_OUTPUT=file - output profiling info to file - use absolute path (default: stdout)
TEST_DEBUG=1 - turn on debugging of the test framework itself (expert)
SKIP_AARCH64=1 - skip tests that are failing on the ARM platorm in FD.io CI
RND_SEED=seed - Seed RND with given seed
Starting VPP in GDB for use with DEBUG=attach:
test-start-vpp-in-gdb - start VPP in gdb (release)
test-start-vpp-debug-in-gdb - start VPP in gdb (debug)
Arguments controlling VPP in GDB runs:
VPP_IN_GDB_TMP_DIR - specify directory to run VPP IN (default: /tmp/unittest-attach-gdb)
VPP_IN_GDB_NO_RMDIR=0 - don't remove existing tmp dir but fail instead
VPP_IN_GDB_CMDLINE=1 - add 'interactive' to VPP arguments to run with command line
Creating test documentation
test-doc - generate documentation for test framework
test-wipe-doc - wipe documentation for test framework
Creating test code coverage report
test-cov - generate code coverage report for test framework
test-wipe-cov - wipe code coverage report for test framework
Verifying code-style
test-checkstyle - check PEP8 compliance
```

132
docs/usecases/container_test.md
View File

@ -64,9 +64,9 @@ Create three network segments
Aka, linux bridges.
```
# lxc network create dora
# lxc network create respond
# lxc network create internet
# lxc network create swan
# lxc network create initiate
```
We'll explain the test topology in a bit. Stay tuned.
@ -95,12 +95,12 @@ containers, run vpp without installing it, etc.
eth2:
name: eth2
nictype: bridged
parent: dora
parent: respond
type: nic
eth3:
name: eth3
nictype: bridged
parent: swan
parent: initiate
type: nic
root:
path: /
@ -142,10 +142,10 @@ and host configuration:
- none
```
Repeat the process with the "dora" and "swan" networks, using these
Repeat the process with the "respond" and "initiate" networks, using these
configurations:
### dora network configuration
### respond network configuration
```
config:
@ -155,7 +155,7 @@ configurations:
ipv6.address: none
ipv6.nat: "false"
description: ""
name: dora
name: respond
type: bridge
used_by:
managed: true
@ -163,7 +163,7 @@ configurations:
locations:
- none
```
### swan network configuration
### initiate network configuration
```
config:
@ -173,7 +173,7 @@ configurations:
ipv6.address: none
ipv6.nat: "false"
description: ""
name: swan
name: initiate
type: bridge
used_by:
managed: true
@ -191,14 +191,14 @@ build vpp, ssh into the container, edit source code, run gdb, etc.
Make sure that e.g. public key auth ssh works.
```
# lxd launch ubuntu:18.04 dora
# lxd launch ubuntu:18.04 respond
<spew>
# lxc exec dora bash
dora# cd /scratch/my-vpp-workspace
dora# apt-get install make ssh
dora# make install-dep
dora# exit
# lxc stop dora
# lxc exec respond bash
respond# cd /scratch/my-vpp-workspace
respond# apt-get install make ssh
respond# make install-dep
respond# exit
# lxc stop respond
```
Mark the container image privileged. If you forget this step, you'll
@ -206,7 +206,7 @@ trip over a netlink error (-11) aka EAGAIN when you try to roll in the
vpp configurations.
```
# lxc config set dora security.privileged "true"
# lxc config set respond security.privileged "true"
```
Duplicate the "master" container image
@ -216,10 +216,10 @@ To avoid having to configure N containers, be sure that the master
container image is fully set up before you help it have children:
```
# lxc copy dora dorahost
# lxc copy dora swan
# lxc copy dora swanhost
# lxc copy dora dhcpserver # optional, to test ipv6 prefix delegation
# lxc copy respond respondhost
# lxc copy respond initiate
# lxc copy respond initiatehost
# lxc copy respond dhcpserver # optional, to test ipv6 prefix delegation
```
Install handy script
@ -250,7 +250,7 @@ Here's the script:
#!/bin/bash
set -u
export containers="dora dorahost swan swanhost dhcpserver"
export containers="respond respondhost initiate initiatehost dhcpserver"
if [ x$1 = "x" ] ; then
echo missing command
@ -292,16 +292,16 @@ Finally, we're ready to describe a test topology. First, a picture:
v | v
eth0 | eth0
+------+ eth1 eth1 +------+
| dora | 10.26.88.100 <= internet bridge => 10.26.88.101 | swan |
| respond | 10.26.88.100 <= internet bridge => 10.26.88.101 | initiate |
+------+ +------+
eth2 / bvi0 10.166.14.2 | 10.219.188.2 eth3 / bvi0
| | |
| ("dora" bridge) | ("swan" bridge) |
| ("respond" bridge) | ("initiate" bridge) |
| | |
v | v
eth2 10.166.14.3 | eth3 10.219.188.3
+----------+ | +----------+
| dorahost | | | dorahost |
| respondhost | | | respondhost |
+----------+ | +----------+
eth0 (management lan) <========+========> eth0 (management lan)
```
@ -309,14 +309,14 @@ Finally, we're ready to describe a test topology. First, a picture:
### Test topology discussion
This topology is suitable for testing almost any tunnel encap/decap
scenario. The two containers "dorahost" and "swanhost" are end-stations
connected to two vpp instances running on "dora" and "swan".
scenario. The two containers "respondhost" and "initiatehost" are end-stations
connected to two vpp instances running on "respond" and "initiate".
We leverage the Linux end-station network stacks to generate traffic
of all sorts.
The so-called "internet" bridge models the public internet. The "dora" and
"swan" bridges connect vpp instances to local hosts
The so-called "internet" bridge models the public internet. The "respond" and
"initiate" bridges connect vpp instances to local hosts
End station configs
-------------------
@ -325,14 +325,14 @@ The end-station Linux configurations set up the eth2 and eth3 ip
addresses shown above, and add tunnel routes to the opposite
end-station networks.
### dorahost configuration
### respondhost configuration
```
ifconfig eth2 10.166.14.3/24 up
route add -net 10.219.188.0/24 gw 10.166.14.2
```
### swanhost configuration
### initiatehost configuration
```
sudo ifconfig eth3 10.219.188.3/24 up
@ -343,9 +343,9 @@ VPP configs
-----------
Split nat44 / ikev2 + ipsec tunneling, with ipv6 prefix delegation in
the "dora" config.
the "respond" config.
### dora configuration
### respond configuration
```
set term pag off
@ -356,10 +356,10 @@ the "dora" config.
set int ip6 table host-eth1 0
set int state host-eth1 up
comment { default route via swan }
comment { default route via initiate }
ip route add 0.0.0.0/0 via 10.26.68.101
comment { "dora-private-net" }
comment { "respond-private-net" }
create host-interface name eth2
bvi create instance 0
set int l2 bridge bvi0 1 bvi
@ -376,16 +376,16 @@ the "dora" config.
comment { nat44 untranslated subnet 10.219.188.0/24 }
comment { responder profile }
ikev2 profile add swan
ikev2 profile set swan udp-encap
ikev2 profile set swan auth rsa-sig cert-file /scratch/setups/doracert.pem
set ikev2 local key /scratch/setups/swankey.pem
ikev2 profile set swan id local fqdn swan.barachs.net
ikev2 profile set swan id remote fqdn broiler2.barachs.net
ikev2 profile set swan traffic-selector remote ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0
ikev2 profile set swan traffic-selector local ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0
ikev2 profile add initiate
ikev2 profile set initiate udp-encap
ikev2 profile set initiate auth rsa-sig cert-file /scratch/setups/respondcert.pem
set ikev2 local key /scratch/setups/initiatekey.pem
ikev2 profile set initiate id local fqdn initiator.my.net
ikev2 profile set initiate id remote fqdn responder.my.net
ikev2 profile set initiate traffic-selector remote ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0
ikev2 profile set initiate traffic-selector local ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0
create ipip tunnel src 10.26.68.100 dst 10.26.68.101
ikev2 profile set swan tunnel ipip0
ikev2 profile set initiate tunnel ipip0
comment { ipv6 prefix delegation }
ip6 nd address autoconfig host-eth1 default-route
@ -400,21 +400,21 @@ the "dora" config.
ip route add 10.219.188.0/24 via ipip0
```
### swan configuration
### initiate configuration
```
set term pag off
comment { "internet" }
create host-interface name eth1
comment { set dhcp client intfc host-eth1 hostname swan }
comment { set dhcp client intfc host-eth1 hostname initiate }
set int ip address host-eth1 10.26.68.101/24
set int state host-eth1 up
comment { default route via "internet gateway" }
comment { ip route add 0.0.0.0/0 via 10.26.68.1 }
comment { "swan-private-net" }
comment { "initiate-private-net" }
create host-interface name eth3
bvi create instance 0
set int l2 bridge bvi0 1 bvi
@ -430,24 +430,24 @@ the "dora" config.
comment { nat44 untranslated subnet 10.166.14.0/24 }
comment { initiator profile }
ikev2 profile add dora
ikev2 profile set dora udp-encap
ikev2 profile set dora auth rsa-sig cert-file /scratch/setups/swancert.pem
set ikev2 local key /scratch/setups/dorakey.pem
ikev2 profile set dora id local fqdn broiler2.barachs.net
ikev2 profile set dora id remote fqdn swan.barachs.net
ikev2 profile add respond
ikev2 profile set respond udp-encap
ikev2 profile set respond auth rsa-sig cert-file /scratch/setups/initiatecert.pem
set ikev2 local key /scratch/setups/respondkey.pem
ikev2 profile set respond id local fqdn responder.my.net
ikev2 profile set respond id remote fqdn initiator.my.net
ikev2 profile set dora traffic-selector remote ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0
ikev2 profile set dora traffic-selector local ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0
ikev2 profile set respond traffic-selector remote ip-range 10.166.14.0 - 10.166.14.255 port-range 0 - 65535 protocol 0
ikev2 profile set respond traffic-selector local ip-range 10.219.188.0 - 10.219.188.255 port-range 0 - 65535 protocol 0
ikev2 profile set dora responder host-eth1 10.26.68.100
ikev2 profile set dora ike-crypto-alg aes-cbc 256 ike-integ-alg sha1-96 ike-dh modp-2048
ikev2 profile set dora esp-crypto-alg aes-cbc 256 esp-integ-alg sha1-96 esp-dh ecp-256
ikev2 profile set dora sa-lifetime 3600 10 5 0
ikev2 profile set respond responder host-eth1 10.26.68.100
ikev2 profile set respond ike-crypto-alg aes-cbc 256 ike-integ-alg sha1-96 ike-dh modp-2048
ikev2 profile set respond esp-crypto-alg aes-cbc 256 esp-integ-alg sha1-96 esp-dh ecp-256
ikev2 profile set respond sa-lifetime 3600 10 5 0
create ipip tunnel src 10.26.68.101 dst 10.26.68.100
ikev2 profile set dora tunnel ipip0
ikev2 initiate sa-init dora
ikev2 profile set respond tunnel ipip0
ikev2 initiate sa-init respond
set int mtu packet 1390 ipip0
set int unnum ipip0 use host-eth1
@ -463,13 +463,13 @@ mentioned. These certificates are used in the ikev2 key exchange.
Here's how to generate the certificates:
```
openssl req -x509 -nodes -newkey rsa:4096 -keyout dorakey.pem -out doracert.pem -days 3560
openssl x509 -text -noout -in doracert.pem
openssl req -x509 -nodes -newkey rsa:4096 -keyout swankey.pem -out swancert.pem -days 3560
openssl x509 -text -noout -in swancert.pem
openssl req -x509 -nodes -newkey rsa:4096 -keyout respondkey.pem -out respondcert.pem -days 3560
openssl x509 -text -noout -in respondcert.pem
openssl req -x509 -nodes -newkey rsa:4096 -keyout initiatekey.pem -out initiatecert.pem -days 3560
openssl x509 -text -noout -in initiatecert.pem
```
Make sure that the "dora" and "swan" configurations point to the certificates.
Make sure that the "respond" and "initiate" configurations point to the certificates.
DHCPv6 server setup
-------------------