156. VEB Switch and floating VEB Tests

156.1. VEB Switching Introduction

IEEE EVB tutorial: http://www.ieee802.org/802_tutorials/2009-11/evb-tutorial-draft-20091116_v09.pdf

Virtual Ethernet Bridge (VEB) - This is an IEEE EVB term. A VEB is a VLAN Bridge internal to Fortville that bridges the traffic of multiple VSIs over an internal virtual network.

Virtual Ethernet Port Aggregator (VEPA) - This is an IEEE EVB term. A VEPA multiplexes the traffic of one or more VSIs onto a single Fortville Ethernet port. The biggest difference between a VEB and a VEPA is that a VEB can switch packets internally between VSIs, whereas a VEPA cannot.

Virtual Station Interface (VSI) - This is an IEEE EVB term that defines the properties of a virtual machine’s (or a physical machine’s) connection to the network. Each downstream v-port on a Fortville VEB or VEPA defines a VSI. A standards-based definition of VSI properties enables network management tools to perform virtual machine migration and associated network re-configuration in a vendor-neutral manner.

My understanding of VEB is that it’s an in-NIC switch(MAC/VLAN), and it can support VF->VF, PF->VF, VF->PF packet forwarding according to the NIC internal switch. It’s similar as Niantic’s SRIOV switch.

156.2. Prerequisites for VEB testing

  1. Get the pci device id of DUT, for example:

    ./dpdk-devbind.py --st
    0000:05:00.0 'Ethernet Controller X710 for 10GbE SFP+' if=ens785f0 drv=i40e
    unused=
    
  2. Host PF in kernel driver. Create 2 VFs from 1 PF with kernel driver, and set the VF MAC address at PF:

    echo 2 > /sys/bus/pci/devices/0000\:05\:00.0/sriov_numvfs
    ./dpdk-devbind.py --st
    
    0000:05:02.0 'XL710/X710 Virtual Function' unused=
    0000:05:02.1 'XL710/X710 Virtual Function' unused=
    
    ip link set ens785f0 vf 0 mac 00:11:22:33:44:11
    ip link set ens785f0 vf 1 mac 00:11:22:33:44:12
    
  3. Host PF in DPDK driver. Create 2VFs from 1 PF with dpdk driver:

    ./dpdk-devbind.py -b igb_uio 05:00.0
    echo 2 >/sys/bus/pci/devices/0000:05:00.0/max_vfs
    ./dpdk-devbind.py --st
    0000:05:02.0 'XL710/X710 Virtual Function' unused=i40evf,igb_uio
    0000:05:02.1 'XL710/X710 Virtual Function' unused=i40evf,igb_uio
    
  4. Bind the VFs to dpdk driver:

    ./tools/dpdk-devbind.py -b vfio-pci 05:02.0 05:02.1
    
  5. Reserve huge pages memory(before using DPDK):

    echo 4096 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB
    /nr_hugepages
    mkdir /mnt/huge
    mount -t hugetlbfs nodev /mnt/huge
    

156.3. Test Case: VEB Switching Inter VF-VF switch

Summary: Kernel PF, then create 2VFs. VFs running dpdk testpmd, send traffic from VF1 to VF2 with VF2’s MAC as DEST address, check if VF2 can receive the packets. Check Inter VF-VF MAC switch.

Details:

  1. In VF1, run testpmd:

    ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 --socket-mem 1024,1024
    -w 05:02.0 --file-prefix=test1 -- -i --crc-strip --eth-peer=0,00:11:22:33:44:12
    testpmd>set fwd txonly
    testpmd>set promisc all off
    testpmd>start
    

    In VF2, run testpmd:

    ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xa -n 4 --socket-mem 1024,1024
    -w 05:02.1 --file-prefix=test2 -- -i --crc-strip
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    

2. Send packets from VF1 to VF2’s MAC address, check if VF2 can get all the packets. Check the packet content is no corrupted.

156.4. Test Case: VEB Switching Inter VF-VF switch MAC forword

Summary: Kernel PF, then create 2VFs. VFs running dpdk testpmd, send traffic to VF1, and set the packet’s DEST MAC to VF2, check if VF2 can receive the packets. Check Inter VF-VF MAC switch.

Details:

  1. In VF1, run testpmd:

    ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 --socket-mem 1024,1024
    -w 05:02.0 --file-prefix=test1 -- -i --crc-strip --eth-peer=0,00:11:22:33:44:12
    testpmd>set fwd mac
    testpmd>set promisc all off
    testpmd>start
    

    In VF2, run testpmd:

    ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xa -n 4 --socket-mem 1024,1024
    -w 05:02.1 --file-prefix=test2 -- -i --crc-strip
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    

2. Send 100 packets to VF1’s MAC address, check if VF2 can get 100 packets. Check the packet content is no corrupted.

156.5. Test Case: VEB Switching Inter VF-VF MAC/VLAN switch

Summary: Kernel PF, then create 2VFs, assign VF1 with VLAN=1, VF2 with VLAN=2. VFs are running dpdk testpmd, send traffic to VF1 with VLAN=1, then let it forward to VF2, it should not work since they are not in the same VLAN; set VF2 with VLAN=1, then send traffic to VF1 with VLAN=1, and VF2 can receive the packets. Check inter VF MAC/VLAN switch.

Details:

  1. Set the VLAN id of VF1 and VF2:

    ip link set ens785f0 vf 0 vlan 1
    ip link set ens785f0 vf 1 vlan 2
    
  2. In VF1, run testpmd:

    ./testpmd -c 0xf -n 4 --socket-mem 1024,1024 -w 0000:05:02.0
    --file-prefix=test1 -- -i --crc-strip --eth-peer=0,00:11:22:33:44:12
    testpmd>set fwd mac
    testpmd>set promisc all off
    testpmd>start
    

    In VF2, run testpmd:

    ./testpmd -c 0xf0 -n 4 --socket-mem 1024,1024 -w 0000:05:02.1
    --file-prefix=test2 -- -i --crc-strip
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    
  1. Send 100 packets with VF1’s MAC address and VLAN=1, check if VF2 can’t get 100 packets since they are not in the same VLAN.

  2. Change the VLAN id of VF2:

    ip link set ens785f0 vf 1 vlan 1
    
  3. Send 100 packets with VF1’s MAC address and VLAN=1, check if VF2 can get 100 packets since they are in the same VLAN now. Check the packet content is not corrupted:

    sendp([Ether(dst="00:11:22:33:44:11")/Dot1Q(vlan=1)/IP()
    /Raw('x'*40)],iface="ens785f1")
    

156.6. Test Case: VEB Switching Inter PF-VF MAC switch

Summary: DPDK PF, then create 2VFs, PF in the host running dpdk testpmd, VFs running dpdk testpmd, VF1 send traffic to VF2, check if VF2 can receive the packets. send tracfic from PF to VF1, ensure PF->VF1; send traffic from VF1 to PF, ensure VF1->PF can work.

Details:

  1. vf->pf PF, launch testpmd:

    ./testpmd -c 0xf -n 4 --socket-mem 1024,1024 -w 0000:05:00.0 --file-prefix=test1 -- -i
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    

    VF1, run testpmd:

    ./testpmd -c 0xf0 -n 4 --socket-mem 1024,1024 -w 0000:05:02.0 --file-prefix=test2 -- -i --eth-peer=0,pf_mac_addr
    testpmd>set fwd txonly
    testpmd>set promisc all off
    testpmd>start
    

    Check if PF can get the packets, so VF1->PF is working. Check the packet content is not corrupted.

  2. pf->vf PF, launch testpmd:

    ./testpmd -c 0xf -n 4 --socket-mem 1024,1024 -w 0000:05:00.0 --file-prefix=test1 -- -i --eth-peer=0,vf1_mac_addr
    testpmd>set fwd txonly
    testpmd>set promisc all off
    testpmd>start
    

    VF1, run testpmd:

    ./testpmd -c 0xf0 -n 4 --socket-mem 1024,1024 -w 0000:05:02.0 --file-prefix=test2 -- -i
    testpmd>mac_addr add 0 vf1_mac_addr
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    

    Check if VF1 can get the packets, so PF->VF1 is working. Check the packet content is not corrupted.

  3. tester->vf PF, launch testpmd:

    ./testpmd -c 0xf -n 4 --socket-mem 1024,1024 -w 0000:05:00.0 --file-prefix=test1 -- -i
    testpmd>set fwd mac
    testpmd>set promisc all off
    testpmd>start
    

    VF1, run testpmd:

    ./testpmd -c 0xf0 -n 4 --socket-mem 1024,1024 -w 0000:05:02.0 --file-prefix=test2 -- -i
    testpmd>mac_addr add 0 vf1_mac_addr
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    

    Send 100 packets with VF’s MAC address from tester, check if VF1 can get 100 packets, so tester->VF1 is working. Check the packet content is not corrupted.

  4. vf1->vf2 PF, launch testpmd:

    ./testpmd -c 0xf -n 4 --socket-mem 1024,1024 -w 0000:05:00.0 --file-prefix=test1 -- -i
    testpmd>set promisc all off
    

    VF1, run testpmd:

    ./testpmd -c 0xf0 -n 4 --socket-mem 1024,1024 -w 0000:05:02.0 --file-prefix=test2 -- -i --eth-peer=0,vf2_mac_addr
    testpmd>set fwd txonly
    testpmd>set promisc all off
    testpmd>start
    

    VF2, run testpmd:

    ./testpmd -c 0xf00 -n 4 --socket-mem 1024,1024 -w 0000:05:02.1 --file-prefix=test3 -- -i
    testpmd>mac_addr add 0 vf2_mac_addr
    testpmd>set fwd rxonly
    testpmd>set promisc all off
    testpmd>start
    

    Check if VF2 can get the packets, so VF1->VF2 is working. Check the packet content is not corrupted.

156.7. Test Case: VEB Switching Inter-VM PF-VF/VF-VF MAC switch Performance

Performance testing, repeat Testcase1 (VF-VF) and Testcase4 (PF-VF) to check the performance at different sizes(64B–1518B and jumbo frame–3000B) with 100% rate sending traffic