9ad39c026c
This patch refactors the VPP sphinx docs in order to make it easier to consume for external readers as well as VPP developers. It also makes sphinx the single source of documentation, which simplifies maintenance and operation. Most important updates are: - reformat the existing documentation as rst - split RELEASE.md and move it into separate rst files - remove section 'events' - remove section 'archive' - remove section 'related projects' - remove section 'feature by release' - remove section 'Various links' - make (Configuration reference, CLI docs, developer docs) top level items in the list - move 'Use Cases' as part of 'About VPP' - move 'Troubleshooting' as part of 'Getting Started' - move test framework docs into 'Developer Documentation' - add a 'Contributing' section for gerrit, docs and other contributer related infos - deprecate doxygen and test-docs targets - redirect the "make doxygen" target to "make docs" Type: refactor Change-Id: I552a5645d5b7964d547f99b1336e2ac24e7c209f Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com> Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
238 lines
11 KiB
ReStructuredText
238 lines
11 KiB
ReStructuredText
.. _iperf3:
|
||
|
||
Introduction
|
||
============
|
||
|
||
This tutorial shows how to use VPP use iperf3 and TRex to get some basic performance
|
||
numbers from a few basic configurations. Four examples are shown. In the first two
|
||
examples, the **iperf3** tool is used to generate traffic, and in the last two examples
|
||
the Cisco's `TRex Realistic Traffic Generator <http://trex-tgn.cisco.com/>`_ is used. For
|
||
comparison purposes, the first example shows packet forwarding using ordinary kernel
|
||
IP forwarding, and the second example shows packet forwarding using VPP.
|
||
|
||
Three Intel Xeon processor platform systems are used to connect to the VPP host to pass traffic
|
||
using **iperf3** and Cisco’s `TRex <http://trex-tgn.cisco.com/>`_.
|
||
|
||
Intel 40 Gigabit Ethernet (GbE) network interface cards (NICs) are used to connect the hosts.
|
||
|
||
|
||
Using Kernel Packet Forwarding with Iperf3
|
||
===========================================
|
||
|
||
In this test, 40 GbE Intel Ethernet Network Adapters are used to connect the three
|
||
systems. Figure 1 illustrates this configuration.
|
||
|
||
.. figure:: /_images/iperf3fig1.png
|
||
|
||
Figure 1: VPP runs on a host that connects to two other systems via 40 GbE NICs.
|
||
|
||
For comparison purposes, in the first example, we configure kernel forwarding in
|
||
*csp2s22c03* and use the **iperf3** tool to measure network bandwidth between
|
||
*csp2s22c03* and *net2s22c05*.
|
||
|
||
In the second example, we start the VPP engine in *csp2s22c03* instead of using
|
||
kernel forwarding. On *csp2s22c03*, we configure the system to have the addresses
|
||
10.10.1.1/24 and 10.10.2.1/24 on the two 40-GbE NICs. To find all network interfaces
|
||
available on the system, use the lshw Linux command to list all network interfaces
|
||
and the corresponding slots *[0000:xx:yy.z]*.
|
||
|
||
In this example, the 40-GbE interfaces are *ens802f0* and *ens802f1*.
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c03$ sudo lshw -class network -businfo
|
||
Bus info Device Class Description
|
||
========================================================
|
||
pci@0000:03:00.0 enp3s0f0 network Ethernet Controller 10-Gig
|
||
pci@0000:03:00.1 enp3s0f1 network Ethernet Controller 10-Gig
|
||
pci@0000:82:00.0 ens802f0 network Ethernet Controller XL710
|
||
pci@0000:82:00.1 ens802f1 network Ethernet Controller XL710
|
||
pci@0000:82:00.0 ens802f0d1 network Ethernet interface
|
||
pci@0000:82:00.1 ens802f1d1 network Ethernet interface
|
||
|
||
|
||
Configure the system *csp2s22c03* to have 10.10.1.1 and 10.10.2.1 on the two 40-GbE NICs
|
||
*ens802f0* and *ens802f1*, respectively.
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c03$ sudo ip addr add 10.10.1.1/24 dev ens802f0
|
||
csp2s22c03$ sudo ip link set dev ens802f0 up
|
||
csp2s22c03$ sudo ip addr add 10.10.2.1/24 dev ens802f1
|
||
csp2s22c03$ sudo ip link set dev ens802f1 up
|
||
|
||
List the route table:
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c03$ route
|
||
Kernel IP routing table
|
||
Destination Gateway Genmask Flags Metric Ref Use Iface
|
||
default jf111-ldr1a-530 0.0.0.0 UG 0 0 0 enp3s0f1
|
||
default 192.168.0.50 0.0.0.0 UG 100 0 0 enp3s0f0
|
||
10.10.1.0 * 255.255.255.0 U 0 0 0 ens802f0
|
||
10.10.2.0 * 255.255.255.0 U 0 0 0 ens802f1
|
||
10.23.3.0 * 255.255.255.0 U 0 0 0 enp3s0f1
|
||
link-local * 255.255.0.0 U 1000 0 0 enp3s0f1
|
||
192.168.0.0 * 255.255.255.0 U 100 0 0 enp3s0f0
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c03$ ip route
|
||
default via 10.23.3.1 dev enp3s0f1
|
||
default via 192.168.0.50 dev enp3s0f0 proto static metric 100
|
||
10.10.1.0/24 dev ens802f0 proto kernel scope link src 10.10.1.1
|
||
10.10.2.0/24 dev ens802f1 proto kernel scope link src 10.10.2.1
|
||
10.23.3.0/24 dev enp3s0f1 proto kernel scope link src 10.23.3.67
|
||
169.254.0.0/16 dev enp3s0f1 scope link metric 1000
|
||
192.168.0.0/24 dev enp3s0f0 proto kernel scope link src 192.168.0.142 metric 100
|
||
|
||
On *csp2s22c04*, we configure the system to have the address 10.10.1.2 and use
|
||
the interface *ens802* to route IP packets 10.10.2.0/24. Use the lshw Linux
|
||
command to list all network interfaces and the corresponding slots *[0000:xx:yy.z]*.
|
||
|
||
For example, the interface *ens802d1* *(ens802)* is connected to slot *[82:00.0]*:
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c04$ sudo lshw -class network -businfo
|
||
Bus info Device Class Description
|
||
=====================================================
|
||
pci@0000:03:00.0 enp3s0f0 network Ethernet Controller 10-Gigabit X540-AT2
|
||
pci@0000:03:00.1 enp3s0f1 network Ethernet Controller 10-Gigabit X540-AT2
|
||
pci@0000:82:00.0 ens802d1 network Ethernet Controller XL710 for 40GbE QSFP+
|
||
pci@0000:82:00.0 ens802 network Ethernet interface
|
||
|
||
For kernel forwarding, set 10.10.1.2 to the interface *ens802*, and add a static
|
||
route for IP packet 10.10.2.0/24:
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c04$ sudo ip addr add 10.10.1.2/24 dev ens802
|
||
csp2s22c04$ sudo ip link set dev ens802 up
|
||
csp2s22c04$ sudo ip route add 10.10.2.0/24 via 10.10.1.1
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c04$ ifconfig
|
||
enp3s0f0 Link encap:Ethernet HWaddr a4:bf:01:00:92:73
|
||
inet addr:10.23.3.62 Bcast:10.23.3.255 Mask:255.255.255.0
|
||
inet6 addr: fe80::a6bf:1ff:fe00:9273/64 Scope:Link
|
||
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
|
||
RX packets:3411 errors:0 dropped:0 overruns:0 frame:0
|
||
TX packets:1179 errors:0 dropped:0 overruns:0 carrier:0
|
||
collisions:0 txqueuelen:1000
|
||
RX bytes:262230 (262.2 KB) TX bytes:139975 (139.9 KB)
|
||
|
||
ens802 Link encap:Ethernet HWaddr 68:05:ca:2e:76:e0
|
||
inet addr:10.10.1.2 Bcast:0.0.0.0 Mask:255.255.255.0
|
||
inet6 addr: fe80::6a05:caff:fe2e:76e0/64 Scope:Link
|
||
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
|
||
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
|
||
TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
|
||
collisions:0 txqueuelen:1000
|
||
RX bytes:0 (0.0 B) TX bytes:5480 (5.4 KB)
|
||
|
||
lo Link encap:Local Loopback
|
||
inet addr:127.0.0.1 Mask:255.0.0.0
|
||
inet6 addr: ::1/128 Scope:Host
|
||
UP LOOPBACK RUNNING MTU:65536 Metric:1
|
||
RX packets:31320 errors:0 dropped:0 overruns:0 frame:0
|
||
TX packets:31320 errors:0 dropped:0 overruns:0 carrier:0
|
||
collisions:0 txqueuelen:1
|
||
RX bytes:40301788 (40.3 MB) TX bytes:40301788 (40.3 MB)
|
||
|
||
After setting the route, we can ping from *csp2s22c03* to *csp2s22c04*, and vice versa:
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c03$ ping 10.10.1.2 -c 3
|
||
PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
|
||
64 bytes from 10.10.1.2: icmp_seq=1 ttl=64 time=0.122 ms
|
||
64 bytes from 10.10.1.2: icmp_seq=2 ttl=64 time=0.109 ms
|
||
64 bytes from 10.10.1.2: icmp_seq=3 ttl=64 time=0.120 ms
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c04$ ping 10.10.1.1 -c 3
|
||
PING 10.10.1.1 (10.10.1.1) 56(84) bytes of data.
|
||
64 bytes from 10.10.1.1: icmp_seq=1 ttl=64 time=0.158 ms
|
||
64 bytes from 10.10.1.1: icmp_seq=2 ttl=64 time=0.096 ms
|
||
64 bytes from 10.10.1.1: icmp_seq=3 ttl=64 time=0.102 ms
|
||
|
||
Similarly, on *net2s22c05*, we configure the system to have the address *10.10.2.2*
|
||
and use the interface *ens803f0* to route IP packets *10.10.1.0/24*. Use the lshw
|
||
Linux command to list all network interfaces and the corresponding slots
|
||
*[0000:xx:yy.z]*. For example, the interface *ens803f0* is connected to slot *[87:00.0]*:
|
||
|
||
.. code-block:: console
|
||
|
||
NET2S22C05$ sudo lshw -class network -businfo
|
||
Bus info Device Class Description
|
||
========================================================
|
||
pci@0000:03:00.0 enp3s0f0 network Ethernet Controller 10-Gigabit X540-AT2
|
||
pci@0000:03:00.1 enp3s0f1 network Ethernet Controller 10-Gigabit X540-AT2
|
||
pci@0000:81:00.0 ens787f0 network 82599 10 Gigabit TN Network Connection
|
||
pci@0000:81:00.1 ens787f1 network 82599 10 Gigabit TN Network Connection
|
||
pci@0000:87:00.0 ens803f0 network Ethernet Controller XL710 for 40GbE QSFP+
|
||
pci@0000:87:00.1 ens803f1 network Ethernet Controller XL710 for 40GbE QSFP+
|
||
|
||
For kernel forwarding, set 10.10.2.2 to the interface ens803f0, and add a static
|
||
route for IP packet 10.10.1.0/24:
|
||
|
||
.. code-block:: console
|
||
|
||
NET2S22C05$ sudo ip addr add 10.10.2.2/24 dev ens803f0
|
||
NET2S22C05$ sudo ip link set dev ens803f0 up
|
||
NET2S22C05$ sudo ip route add 10.10.1.0/24 via 10.10.2.1
|
||
|
||
After setting the route, you can ping from *csp2s22c03* to *net2s22c05*, and vice
|
||
versa. However, in order to ping between *net2s22c05* and *csp2s22c04*, kernel IP
|
||
forwarding in *csp2s22c03* has to be enabled:
|
||
|
||
.. code-block:: console
|
||
|
||
csp2s22c03$ sysctl net.ipv4.ip_forward
|
||
net.ipv4.ip_forward = 0
|
||
csp2s22c03$ echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward
|
||
csp2s22c03$ sysctl net.ipv4.ip_forward
|
||
net.ipv4.ip_forward = 1
|
||
|
||
If successful, verify that now you can ping between *net2s22c05* and *csp2s22c04*:
|
||
|
||
.. code-block:: console
|
||
|
||
NET2S22C05$ ping 10.10.1.2 -c 3
|
||
PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
|
||
64 bytes from 10.10.1.2: icmp_seq=1 ttl=63 time=0.239 ms
|
||
64 bytes from 10.10.1.2: icmp_seq=2 ttl=63 time=0.224 ms
|
||
64 bytes from 10.10.1.2: icmp_seq=3 ttl=63 time=0.230 ms
|
||
|
||
We use the **iperf3** utility to measure network bandwidth between hosts. In this
|
||
test, we download the **iperf3** utility tool on both *net2s22c05* and *csp2s22c04*.
|
||
On *csp2s22c04*, we start the **iperf3** server with “iperf3 –s”, and then on *net2s22c05*,
|
||
we start the **iperf3** client to connect to the server:
|
||
|
||
.. code-block:: console
|
||
|
||
NET2S22C05$ iperf3 -c 10.10.1.2
|
||
Connecting to host 10.10.1.2, port 5201
|
||
[ 4] local 10.10.2.2 port 54074 connected to 10.10.1.2 port 5201
|
||
[ ID] Interval Transfer Bandwidth Retr Cwnd
|
||
[ 4] 0.00-1.00 sec 936 MBytes 7.85 Gbits/sec 2120 447 KBytes
|
||
[ 4] 1.00-2.00 sec 952 MBytes 7.99 Gbits/sec 1491 611 KBytes
|
||
[ 4] 2.00-3.00 sec 949 MBytes 7.96 Gbits/sec 2309 604 KBytes
|
||
[ 4] 3.00-4.00 sec 965 MBytes 8.10 Gbits/sec 1786 571 KBytes
|
||
[ 4] 4.00-5.00 sec 945 MBytes 7.93 Gbits/sec 1984 424 KBytes
|
||
[ 4] 5.00-6.00 sec 946 MBytes 7.94 Gbits/sec 1764 611 KBytes
|
||
[ 4] 6.00-7.00 sec 979 MBytes 8.21 Gbits/sec 1499 655 KBytes
|
||
[ 4] 7.00-8.00 sec 980 MBytes 8.22 Gbits/sec 1182 867 KBytes
|
||
[ 4] 8.00-9.00 sec 1008 MBytes 8.45 Gbits/sec 945 625 KBytes
|
||
[ 4] 9.00-10.00 sec 1015 MBytes 8.51 Gbits/sec 1394 611 KBytes
|
||
- - - - - - - - - - - - - - - - - - - - - - - - -
|
||
[ ID] Interval Transfer Bandwidth Retr
|
||
[ 4] 0.00-10.00 sec 9.45 GBytes 8.12 Gbits/sec 16474 sender
|
||
[ 4] 0.00-10.00 sec 9.44 GBytes 8.11 Gbits/sec receiver
|
||
|
||
iperf Done.
|