192. VF L3 forwarding kernel PF test in EM mode

This document provides benchmark test for NIC VFs which are created from kernel PFs or. These tests use l3fwd as a simple forwarder between NIC vfs. The goal of this test plan is to have a tested benchmark between NIC vfs. The Layer-3 Forwarding performance results are produced using l3fwd application. For detail test plan, please refer to l3fwd_test_plan.rst.

192.1. Prerequisites

192.1.1. Topology

It requires at least 1 port connected traffic generator::
Port0 — TG0
2 ports::
Port0 — TG0 Port1 — TG1
4 ports::
Port0 — TG0 Port1 — TG1 Port2 — TG2 Port3 — TG3

192.1.2. Hardware

This suite focus on l3fwd application, so any standard Ethernet Network Adapter is qualified.

192.1.3. Software

dpdk: git clone http://dpdk.org/git/dpdk trex: git clone http://trex-tgn.cisco.com/trex/release/v2.93.tar.gz

192.2. Test Case

The test case check the throughput result with em mode and ipv4/ipv6, in the case, we will send the bi-direction flows with line rate, then we can check the passthrough rate.

The l3fwd application has a number of command line options, here list the key options will be tested:

./dpdk-l3fwd [EAL options] -- -p PORTMASK
                         [--force-max-simd-bitwidth=max-simd-bitwidth]
                         --config(port,queue,lcore)[,(port,queue,lcore)]
                         [--rx-queue-size NPKTS]
                         [--tx-queue-size NPKTS]
                         [--parse-ptype]
                         [-L]|[-E]
                         ...
Note:
    --force-max-simd-bitwidth: This setting is used to determine the vector path for component selection.
                               And the default is avx2.
    --rx-queue-size: Rx queue size in decimal and default is 1024.
    --tx-queue-size: Tx queue size in decimal and default is 1024.
    --parse-ptype: Optional, set to use software to analyze packet type.
                   Without this option, hardware will check the packet type.
    [-L]|[-E]: L3fwd uses the parameters "-L" and "-E" to correspond to LPM and EM modes respectively.
               And the default is LPM mode.

192.2.1. Common Steps

  1. Bind PF ports to kernel driver, i40e or ice, then create 1 VF from each PF, take E810 for example:

    <dpdk_dir>#echo 1 > /sys/bus/pci/devices/0000\:17\:00.0/sriov_numvfs
    

    Set vf mac address:

    <dpdk_dir>#ip link set ens5f0 vf 0 mac 00:12:34:56:78:01
    

    Bind all the created VFs to vfio-pci:

    <dpdk_dir>#./usertools/dpdk-devbind.py -s
    0000:17:00.0 'Device 1592' if=ens5f0 drv=ice unused=vfio-pci
    0000:17:01.0 'Device 1592' if=ens5f0v0 drv=iavf unused=vfio-pci
    <dpdk_dir>#./usertools/dpdk-devbind.py -b vfio-pci <pci device id>
    <dpdk_dir>#./usertools/dpdk-devbind.py -b vfio-pci 0000:17:01.0
    
  2. Configure traffic generator to send traffic

    Routing table for IPv4 packets
    • In EM mode, the EM table used for packet routing is:
    Entry
    #
    IPv4 destination address IPv4 source address Port destination Port source L4 protocol
    0 198.18.0.0 Random 10 1 UDP
    1 198.18.1.0 Random 10 1 UDP
    2 198.18.2.0 Random 11 1 UDP
    3 198.18.3.0 Random 11 1 UDP
    Routing table for IPv6 packets
    • In EM mode, the EM table used for packet routing is:
    Entry IPv6 destination address IPv6 source address Port destination Port source L4 protocol
    0
    2001:0200:0000:0000:0000:
    0000:0000:0000
    Random 10 1 UDP
    1
    2001:0200:0000:0001:0000:
    0000:0000:0000
    Random 10 1 UDP
    2
    2001:0200:0000:0002:0000:
    0000:0000:0000
    Random 11 1 UDP
    3
    2001:0200:0000:0003:0000:
    0000:0000:0000
    Random 11 1 UDP
    The flows need to be configured and started by the traffic generator:
    • IPv4 packets
    Flow Traffic Gen. Port IPv4 Dst. Address IPv4 Src. Address
    1 TG0 198.18.1.1 Any Ip
    2 TG1 198.18.0.1 Any Ip
    3 TG2 198.18.3.1 Any Ip
    4 TG3 198.18.2.1 Any Ip

    Set the packet length : 64 bytes-1518 bytes The IPV4 Src Address increase with the num 1024.

    • IPv6 packets
    Flow Traffic Gen. Port IPV6 Dst. Address IPv6 Src. Address
    1 TG0 2001:0200:0000:0000:0000:0000:0000:0000 Any Ip
    2 TG1 2001:0200:0000:0001:0000:0000:0000:0000 Any Ip
    3 TG2 2001:0200:0000:0002:0000:0000:0000:0000 Any Ip
    4 TG3 2001:0200:0000:0003:0000:0000:0000:0000 Any Ip

    Set the packet length : 66 bytes-1518 bytes The IPV6 Src Address increase with the num 1024.

  3. Test result table

    Fwd_core Frame Size Throughput Rate
    1C/1T 64 xxxxx Mpps xxx %
    1C/1T ... xxxxx Mpps xxx %
    2C/2T 64 xxxxx Mpps xxx %
    2C/2T ... xxxxx Mpps xxx %
    4C/4T 64 xxxxx Mpps xxx %
    4C/4T ... xxxxx Mpps xxx %
    8C/8T 64 xxxxx Mpps xxx %
    8C/8T ... xxxxx Mpps xxx %

192.2.2. Test Case: VF L3 Forwarding in EM Mode with IPv4/IPv6 Packets

  1. Bind VF ports to dpdk driver as common step 1.

  2. Launch l3fwd:

    <build_dir>/examples/dpdk-l3fwd -l 5-8 -n 8 --force-max-simd-bitwidth=512 \
    -- -p 0x1 --config "(0,0,5),(0,1,6),(0,2,7),(0,3,8)" --rx-queue-size 1024 \
    --tx-queue-size 1024 -E --parse-ptype
    
  3. Configure traffic generator to send traffic as common step 2.

  4. Record Test results as common step 3.