61. CVL IAVF Flexible Descriptor

61.1. Description

To carry more metadata to descriptor and fill them to mbuf to save CPU cost on packet parsing. CVL flexible descriptor capability can be used. iavf driver could negotiate descriptor format with PF driver by virtchnl. It is implemented in DPDK20.11, and requires ice base driver >= 1.3.0.

61.2. Prerequisites

  1. NIC requires
    • Intel E810 series ethernet cards: columbiaville_25g, columbiaville_100g, etc.
  2. Toplogy
    • 1 Intel E810 interface
    • 1 network interface for sending test packet, which could be connect to the E810 interface
    • Directly connect the 2 interfaces
  3. DDP Package Preparation
    • copy correct ice.pkg into /lib/firmware/updates/intel/ice/ddp/.
    • unbind, then bind device back to ice.ko to update ddp package in the NIC.

Warning

Need unbind all ports in NIC, then bind device back to ice.ko

Note

To test all test cases, comms/wireless package is expected, as os package only support MPLS cases.

  1. DPDK Application Preparation

The default DPDK don’t support dump flexible descriptor fields, so need to patch dpdk and re-compile it.

  1. Patch testpmd for dumping flexible fields from RXD:

    diff --git a/app/test-pmd/meson.build b/app/test-pmd/meson.build
    index 7e9c7bdd6..b75b90a9c 100644
    --- a/app/test-pmd/meson.build
    +++ b/app/test-pmd/meson.build
    @@ -49,6 +49,9 @@ endif
    if dpdk_conf.has('RTE_NET_I40E')
    deps += 'net_i40e'
    endif
    +if dpdk_conf.has('RTE_NET_ICE')
    +       deps += ['net_ice', 'net_iavf']
    +endif
    if dpdk_conf.has('RTE_NET_IXGBE')
    deps += 'net_ixgbe'
    endif
    
    diff --git a/app/test-pmd/util.c b/app/test-pmd/util.c
    index a9e431a8b..3447a9b43 100644
    --- a/app/test-pmd/util.c
    +++ b/app/test-pmd/util.c
    @@ -12,6 +12,7 @@
    #include <rte_vxlan.h>
    #include <rte_ethdev.h>
    #include <rte_flow.h>
    +#include <rte_pmd_iavf.h>
    
    #include "testpmd.h"
    
    @@ -151,6 +152,7 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue, struct rte_mbuf *pkts[],
    eth_type, (unsigned int) mb->pkt_len,
    (int)mb->nb_segs);
    ol_flags = mb->ol_flags;
    +                rte_pmd_ifd_dump_proto_xtr_metadata(mb);
    if (ol_flags & PKT_RX_RSS_HASH) {
    MKDUMPSTR(print_buf, buf_size, cur_len,
    " - RSS hash=0x%x",
    
  2. Compile DPDK and testpmd:

    CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib x86_64-native-linuxapp-gcc
    ninja -C x86_64-native-linuxapp-gcc -j 70
    
  1. Generate 1 VF on each PF and set mac address for each VF:

    echo 1 > /sys/bus/pci/devices/0000:af:00.0/sriov_numvfs
    ip link set ens802f0 vf 0 mac 00:11:22:33:44:55
    
  2. Bind the vf interface to vfio-pci driver:

    ./usertools/dpdk-devbind.py -b vfio-pci af:01.0
    

61.3. VLAN cases

  1. Launch testpmd by:

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=vlan -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2
    testpmd>set verbose 1
    testpmd>set fwd io
    testpmd>start
    
  2. check RXDID value correct:

    expected: RXDID[17]
    

Note

Please change the core setting (-l option) and port’s PCI (-w option) by your DUT environment

61.3.1. Test Case: Check single VLAN fields in RXD (802.1Q)

Send a packet with VLAN tag from test network interface:

p = Ether(src="68:05:ca:a3:1b:28", dst="00:11:22:33:44:55", type=0x9100)/Dot1Q(prio=1,vlan=23)/IP()/UDP()/DNS()
sendp(p, iface='ens192f0', count=1)

Note

  • Change ethernet source address with your test network interface’s address
  • Make sure the ethernet destination address is NOT your real E810 interface’s address

Check the output in testpmd, ctag=1:0:23 is expected, which is consistent with VLAN tag set in test packet:

testpmd> port 0/queue 28: received 1 packets
src=68:05:CA:A3:1B:28 - dst=00:11:22:33:44:55 - type=0x8100 - length=60 - nb_segs=1 - RSS hash=0xf31f649c - RSS queue=0x1c -
Protocol Extraction:[0x0000:0x2017],vlan,stag=0:0:0,ctag=1:0:23  - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP  -
sw ptype: L2_ETHER_VLAN L3_IPV4 L4_UDP  - l2_len=18 - l3_len=20 - l4_len=8 - Receive queue=0x1c
ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_GOOD PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN

61.3.2. Test Case: Check single VLAN fields in RXD (802.1ad)

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55', type=0x88A8)/Dot1Q(prio=1,vlan=23)/IP()/UDP()/DNS()

Expected output in testpmd:

stag=1:0:23

61.3.3. Test Case: Check double VLAN fields in RXD (802.1Q) only 1 VLAN tag

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55', type=0x9100)/Dot1Q(prio=1,vlan=23)/IP()/UDP()/DNS()

Expected output in testpmd:

stag=1:0:23

61.3.4. Test Case: Check double VLAN fields in RXD (802.1Q) 2 VLAN tags

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55', type=0x9100)/Dot1Q(prio=1,vlan=23)/Dot1Q(prio=4,vlan=56)/IP()/UDP()/DNS()

Expected output in testpmd:

stag=1:0:23
ctag=4:0:56

61.3.5. Test Case: Check double VLAN fields in RXD (802.1ad)

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55', type=0x88A8)/Dot1Q(prio=1,vlan=23)/Dot1Q(prio=4,vlan=56)/IP()/UDP()/DNS()

Expected output in testpmd:

stag=1:0:23
ctag=4:0:56

61.4. Check IPv4 fields in RXD

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

Launch testpmd command:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=ipv4 -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

check RXDID value correct:

expected: RXDID[18]

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55')/IP(tos=23,ttl=98)/UDP()/Raw(load='XXXXXXXXXX')

Expected output in testpmd:

ver=4
hdrlen=5
tos=23
ttl=98
proto=17

61.5. Check IPv6 fields in RXD

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

Launch testpmd command:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=ipv6 -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

check RXDID value correct:

expected: RXDID[19]

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55')/IPv6(tc=12,hlim=34,fl=0x98765)/UDP()/Raw(load='XXXXXXXXXX')

Expected output in testpmd:

ver=6
tc=12
flow_hi4=0x9
nexthdr=17
hoplimit=34

61.6. Check IPv6 flow field in RXD

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

Launch testpmd command:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=ipv6_flow -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

check RXDID value correct:

expected: RXDID[20]

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55')/IPv6(tc=12,hlim=34,fl=0x98765)/UDP()/Raw(load='XXXXXXXXXX')

Expected output in testpmd:

ver=6
tc=12
flow=0x98765

61.7. Check TCP fields in IPv4 in RXD

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

Launch testpmd command:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=tcp -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

check RXDID value correct:

expected: RXDID[21]

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55')/IP()/TCP(flags='AS')/Raw(load='XXXXXXXXXX')

Expected output in testpmd:

doff=5
flags=AS

61.8. Check TCP fields in IPv6 in RXD

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

Launch testpmd command:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=tcp -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

check RXDID value correct:

expected: RXDID[21]

Test packet:

p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55')/IPv6()/TCP(flags='S')/Raw(load='XXXXXXXXXX')

Expected output in testpmd:

doff=5
flags=S

61.9. Check IPv4, IPv6, TCP fields in RXD on specific queues

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

Launch testpmd command:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr='[(2):ipv4,(3):ipv6,(4):tcp]' -- -i --rxq=16 --txq=16 --portmask=0x1

check RXDID value correct:

expected: RXDID[16], RXDID[18], RXDID[19], RXDID[21]

Create generic flow on NIC:

flow create 0 ingress pattern eth / ipv4 src is 192.168.0.1 dst is 192.168.0.2 tos is 23 ttl is 98 / end actions queue index 2 / end
flow create 0 ingress pattern eth / ipv6 src is 2001::3 dst is 2001::4 tc is 12 / end actions queue index 3 / end
flow create 0 ingress pattern eth / ipv4 src is 192.168.0.1 dst is 192.168.0.2 / tcp src is 25 dst is 23 / end actions queue index 4 / end

Test packet:

p = Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.1",dst="192.168.0.2",tos=23,ttl=98)/UDP()/Raw(load='XXXXXXXXXX')
p = Ether(src='68:05:ca:a3:1b:28', dst='00:11:22:33:44:55')/IPv6(src='2001::3', dst='2001::4', tc=12,hlim=34,fl=0x98765)/UDP()/Raw(load='XXXXXXXXXX')
p = Ether(dst='00:11:22:33:44:55')/IP(src='192.168.0.1',dst='192.168.0.2')/TCP(flags='AS', dport=23, sport=25)/Raw(load='XXXXXXXXXX')

Expected output in testpmd:

Receive queue=0x2
ver=4
hdrlen=5
tos=23
ttl=98
proto=17

Receive queue=0x3
ver=6
tc=12
flow_hi4=0x9
nexthdr=17
hoplimit=34

Receive queue=0x4
doff=5
flags=AS

61.10. Check testpmd use different parameters start

Test steps are same to VLAN cases, use different “proto_xtr” parameters the launch command of testpmd, check RXDID value.

use error parameter Launch testpmd:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=vxlan -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

testpmd can’t started, check “iavf_lookup_flex_desc_type(): wrong flex_desc type, it should be: vlan|ipv4|ipv6|ipv6_flow|tcp|ovs|ip_offset” in testpmd output.

don’t use parameter launch testpmd:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0 -- -i --rxq=4 --txq=4 --portmask=0x1 --nb-cores=2

testpmd started, check “iavf_configure_queues(): request RXDID[16] in Queue[0]” in testpmd output

61.11. MPLS cases

Test steps are same to VLAN cases, just change the launch command of testpmd, test packet and expected output

MPLS cases use same parameter Launch testpmd:

./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 6-9 -n 4 -w af:01.0,proto_xtr=ip_offset -- -i  --portmask=0x1 --nb-cores=2

check RXDID value correct:

expected: RXDID[25]

scapy prepare:

about scapy:
from scapy.contrib.mpls import MPLS

61.11.1. Test Case: Check ip offset of ip

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=18

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=18

61.11.2. Test Case: check ip offset with vlan

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=22

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=22

61.11.3. Test Case: check offset with 2 vlan tag

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=26

61.11.4. Test Case: check ip offset with multi MPLS

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=18

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=22

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=30

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=34

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=18

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=22

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=30

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=34

61.11.5. Test Case: check ip offset with multi MPLS with vlan tag

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=22

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=30

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=34

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=38

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=22

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=30

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=34

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=38

61.11.6. Test Case: check ip offset with multi MPLS with 2 vlan tag

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=30

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=34

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=38

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IP()

Expected output in testpmd:

Protocol Offset:ip_offset=42

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=26

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=30

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=34

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=38

Test packet:

p = Ether(dst="00:11:22:33:44:55",type=0x88A8)/Dot1Q(type=0x8100)/Dot1Q(type=0x8847)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=0)/MPLS(s=1)/IPv6()

Expected output in testpmd:

Protocol Offset:ip_offset=42