.. SPDX-License-Identifier: BSD-3-Clause
   Copyright(c) 2011-2019 Intel Corporation

=======
QoS API
=======

Description
===========
The generic API for the Quality of Service (QoS) Traffic Management of Ethernet
devices, which includes the following main features: hierarchical scheduling,
traffic shaping, congestion management, packet marking.
The TM setting commands are as following:

1. Add the port traffic management private shaper profile::

    testpmd> add port tm node shaper profile (port_id) (shaper_profile_id) \
    (cmit_tb_rate) (cmit_tb_size) (peak_tb_rate) (peak_tb_size) \
    (packet_length_adjust)

2. Add nonleaf node to port traffic management hiearchy::

    testpmd> add port tm nonleaf node (port_id) (node_id) (parent_node_id) \
    (priority) (weight) (level_id) (shaper_profile_id) \
    (n_sp_priorities) (stats_mask) (n_shared_shapers) \
    [(shared_shaper_0) (shared_shaper_1) ...] \

3. Add leaf node to port traffic management hiearchy::

    testpmd> add port tm leaf node (port_id) (node_id) (parent_node_id) \
    (priority) (weight) (level_id) (shaper_profile_id) \
    (cman_mode) (wred_profile_id) (stats_mask) (n_shared_shapers) \
    [(shared_shaper_id) (shared_shaper_id) ...] \

4. Commit the traffic management hierarchy on the port::

    testpmd> port tm hierarchy commit (port_id) (clean_on_fail)

Prerequisites
=============
For i40e, need enable rss
For ixgbe, need disable rss.
The DUT must have two 10G Ethernet ports connected to two ports on tester::

    dut_port_0 <---> tester_port_0
    dut_port_1 <---> tester_port_1

Assume two DUT 10G Ethernet ports' pci device id is as the following::

    dut_port_0 : "0000:05:00.0"
    dut_port_1 : "0000:05:00.1"

Bind two ports to dpdk driver::

    ./usertools/dpdk-devbind.py -b igb_uio 05:00.0 05:00.1

Test Cases for I40e:
====================

Test Case: dcb 4 tc queue mapping
=================================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 23-27 -n 4 --master-lcore=23 -- -i --nb-cores=4 --rxq=4 --txq=4 --rss-ip
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 4 pfc off
    testpmd> port config 1 dcb vt off 4 pfc off
    testpmd> port start all
    testpmd> set fwd rxonly
    testpmd> set verbose 1
    testpmd> start

2. Send packets with set vlan user priority 0, 1, 2, 3, verify::

    pkt1 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=0)/IP()/Raw('x'*20)
    pkt2 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=1)/IP()/Raw('x'*20)
    pkt3 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=2)/IP()/Raw('x'*20)
    pkt4 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=3)/IP()/Raw('x'*20)

   packet with vlan user priority 0 should be received by queue 0
   packet with vlan user priority 1 should be received by queue 1
   packet with vlan user priority 2 should be received by queue 2
   packet with vlan user priority 3 should be received by queue 3

Test Case: dcb 8 tc queue mapping
=================================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 23-31 -n 4 --master-lcore=23 -- -i --nb-cores=8 --rxq=8 --txq=8 --rss-ip
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 8 pfc off
    testpmd> port config 1 dcb vt off 8 pfc off
    testpmd> port start all
    testpmd> set fwd rxonly
    testpmd> set verbose 1
    testpmd> start

2. Send packet with set vlan user priority 0, 1, 2, 3, 4, 5, 6, 7, verify::

    pkt1 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=0)/IP()/Raw('x'*20)
    pkt2 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=1)/IP()/Raw('x'*20)
    pkt3 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=2)/IP()/Raw('x'*20)
    pkt4 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=3)/IP()/Raw('x'*20)
    pkt5 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=4)/IP()/Raw('x'*20)
    pkt6 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=5)/IP()/Raw('x'*20)
    pkt7 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=6)/IP()/Raw('x'*20)
    pkt8 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=7)/IP()/Raw('x'*20)

   packet with vlan user priority 0 should be received by queue 0
   packet with vlan user priority 1 should be received by queue 1
   packet with vlan user priority 2 should be received by queue 2
   packet with vlan user priority 3 should be received by queue 3
   packet with vlan user priority 4 should be received by queue 4
   packet with vlan user priority 5 should be received by queue 5
   packet with vlan user priority 6 should be received by queue 6
   packet with vlan user priority 7 should be received by queue 7

Test Case: shaping 1 port 4 tc
==============================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 23-27 -n 4 --master-lcore=23 -- -i --nb-cores=4 --rxq=4 --txq=4 --rss-ip
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 4 pfc off
    testpmd> port config 1 dcb vt off 4 pfc off

2. Add root non leaf node::

    testpmd> add port tm nonleaf node 1 1000000 -1 0 1 0 -1 1 0 0

3. Add private shaper 1, 2, 3, 4 for tc node 0-4::

    testpmd> add port tm node shaper profile 1 1 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 2 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 3 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 4 0 0 25000000 0 0

4. Add tc node::

    testpmd> add port tm nonleaf node 1 900000 1000000 0 1 1 1 1 0 0
    testpmd> add port tm nonleaf node 1 900001 1000000 0 1 1 2 1 0 0
    testpmd> add port tm nonleaf node 1 900002 1000000 0 1 1 3 1 0 0
    testpmd> add port tm nonleaf node 1 900003 1000000 0 1 1 4 1 0 0

5. Add queue leaf node::

    testpmd> add port tm leaf node 1 0 900000 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 1 900001 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 2 900002 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 3 900003 0 1 2 -1 0 0xffffffff 0 0

6. Commit the traffic management hierarchy on the port::

    testpmd> port tm hierarchy commit 1 no
    testpmd> port start all
    testpmd> start

7. Send four flows with user priority 0, 1, 2, 3,
   verify shaping each flow to 200Mbps(25MBps).

Test Case:  shaping 1 port 8 tc
===============================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 23-31 -n 4 --master-lcore=23 -- -i --nb-cores=8 --rxq=8 --txq=8 --rss-ip
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 8 pfc off
    testpmd> port config 1 dcb vt off 8 pfc off

2. Add root non leaf node::

    testpmd> add port tm nonleaf node 1 1000000 -1 0 1 0 -1 1 0 0

3. Add private shaper 1, 2, 3, 4, 5, 6, 7, 8, for tc node 0-7::

    testpmd> add port tm node shaper profile 1 1 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 2 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 3 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 4 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 5 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 6 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 7 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 8 0 0 25000000 0 0

4. Add tc node::

    testpmd> add port tm nonleaf node 1 900000 1000000 0 1 1 1 1 0 0
    testpmd> add port tm nonleaf node 1 900001 1000000 0 1 1 2 1 0 0
    testpmd> add port tm nonleaf node 1 900002 1000000 0 1 1 3 1 0 0
    testpmd> add port tm nonleaf node 1 900003 1000000 0 1 1 4 1 0 0
    testpmd> add port tm nonleaf node 1 900004 1000000 0 1 1 5 1 0 0
    testpmd> add port tm nonleaf node 1 900005 1000000 0 1 1 6 1 0 0
    testpmd> add port tm nonleaf node 1 900006 1000000 0 1 1 7 1 0 0
    testpmd> add port tm nonleaf node 1 900007 1000000 0 1 1 8 1 0 0

5. Add queue leaf node::

    testpmd> add port tm leaf node 1 0 900000 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 1 900001 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 2 900002 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 3 900003 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 4 900004 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 5 900005 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 6 900006 0 1 2 -1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 7 900007 0 1 2 -1 0 0xffffffff 0 0

6. Commit the traffic management hierarchy on the port::

    testpmd> port tm hierarchy commit 1 no
    testpmd> port start all
    testpmd> start

7. Send four flows with user priority 0, 1, 2, 3, 4, 5, 6, 7,
   verify shaping each flow to 200Mbps(25MBps).

Test Case: shaping for port
===========================
1. Start testpmd::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 23-27 -n 4 --master-lcore=23 -- -i --nb-cores=4 --rxq=4 --txq=4 --rss-ip
    testpmd> port stop 1

1. Add private shaper 0::

    testpmd> add port tm node shaper profile 1 0 0 0 25000000 0 0

2. Add port (root nonleaf) node::

    testpmd> add port tm nonleaf node 1 1000000 -1 0 1 0 0 1 0 0

3. Commit the traffic management hierarchy on the port::

    testpmd> port tm hierarchy commit 1 no
    testpmd> port start 1
    testpmd> start

4. Start transmitting,
   verify shaping the traffic to 200Mbps(25MBps).

Test Cases for ixgbe:
=====================

Test Case: dcb 4 tc queue mapping
=================================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 3-7 -n 4 --master-lcore=3 -- -i --nb-cores=4 --rxq=4 --txq=4 --disable-rss
    testpmd> vlan set filter off 0
    testpmd> vlan set filter off 1
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 4 pfc off
    testpmd> port config 1 dcb vt off 4 pfc off
    testpmd> port start all
    testpmd> set fwd rxonly
    testpmd> set verbose 1
    testpmd> start

2. Send packets with set vlan user priority 0, 1, 2, 3, verify::

    pkt1 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=0)/IP()/Raw('x'*20)
    pkt2 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=1)/IP()/Raw('x'*20)
    pkt3 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=2)/IP()/Raw('x'*20)
    pkt4 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=3)/IP()/Raw('x'*20)

   packet with vlan user priority 0 should be received by queue 0
   packet with vlan user priority 1 should be received by queue 32
   packet with vlan user priority 2 should be received by queue 64
   packet with vlan user priority 3 should be received by queue 96

Test Case: dcb 8 tc queue mapping
=================================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 3-11 -n 4 --master-lcore=3 -- -i --nb-cores=8 --rxq=8 --txq=8 --disable-rss
    testpmd> vlan set filter off 0
    testpmd> vlan set filter off 1
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 8 pfc off
    testpmd> port config 1 dcb vt off 8 pfc off
    testpmd> port start all
    testpmd> set fwd rxonly
    testpmd> set verbose 1
    testpmd> start

2. Send packet with set vlan user priority 0, 1, 2, 3, 4, 5, 6, 7, verify::

    pkt1 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=0)/IP()/Raw('x'*20)
    pkt2 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=1)/IP()/Raw('x'*20)
    pkt3 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=2)/IP()/Raw('x'*20)
    pkt4 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=3)/IP()/Raw('x'*20)
    pkt5 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=4)/IP()/Raw('x'*20)
    pkt6 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=5)/IP()/Raw('x'*20)
    pkt7 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=6)/IP()/Raw('x'*20)
    pkt8 = Ether(dst="00:00:00:00:01:00", src="00:02:00:00:00:01")/Dot1Q(prio=7)/IP()/Raw('x'*20)

   packet with vlan user priority 0 should be received by queue 0
   packet with vlan user priority 1 should be received by queue 16
   packet with vlan user priority 2 should be received by queue 32
   packet with vlan user priority 3 should be received by queue 48
   packet with vlan user priority 4 should be received by queue 64
   packet with vlan user priority 5 should be received by queue 80
   packet with vlan user priority 6 should be received by queue 96
   packet with vlan user priority 7 should be received by queue 112

Test Case: shaping for queue with 4 tc
======================================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 3-7 -n 4 --master-lcore=3 -- -i --nb-cores=4 --rxq=4 --txq=4 --disable-rss
    testpmd> vlan set filter off 0
    testpmd> vlan set filter off 1
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 4 pfc off
    testpmd> port config 1 dcb vt off 4 pfc off
    testpmd> port start all

2. Add root non leaf node::

    testpmd> add port tm nonleaf node 1 1000000 -1 0 1 0 -1 1 0 0

3. Add tc node::

    testpmd> add port tm nonleaf node 1 900000 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900001 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900002 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900003 1000000 0 1 1 -1 1 0 0

4. Add private shaper 1, 2, 3, 4 for tc node 0-4::

    testpmd> add port tm node shaper profile 1 0 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 1 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 2 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 3 0 0 25000000 0 0

5. Add queue leaf node::

    testpmd> add port tm leaf node 1 0 900000 0 1 2 0 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 1 900001 0 1 2 1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 2 900002 0 1 2 2 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 3 900003 0 1 2 3 0 0xffffffff 0 0

6. Commit the traffic management hierarchy on the port::

    testpmd> port tm hierarchy commit 1 no
    testpmd> port start all
    testpmd> start

7. Send four flows with user priority 0, 1, 2, 3,
   verify shaping each flow to 200Mbps(25MBps).

Test Case: shaping for queue with 8 tc
======================================
1. Start testpmd and set DCB::

    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 3-11 -n 4 --master-lcore=3 -- -i --nb-cores=8 --rxq=8 --txq=8 --disable-rss
    testpmd> vlan set filter off 0
    testpmd> vlan set filter off 1
    testpmd> port stop all
    testpmd> port config 0 dcb vt off 8 pfc off
    testpmd> port config 1 dcb vt off 8 pfc off
    testpmd> port start all

2. Add root non leaf node::

    testpmd> add port tm nonleaf node 1 1000000 -1 0 1 0 -1 1 0 0

3. Add tc node::

    testpmd> add port tm nonleaf node 1 900000 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900001 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900002 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900003 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900004 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900005 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900006 1000000 0 1 1 -1 1 0 0
    testpmd> add port tm nonleaf node 1 900007 1000000 0 1 1 -1 1 0 0

4. Add private shaper 1, 2, 3, 4, 5, 6, 7, 8, for tc node 0-7::

    testpmd> add port tm node shaper profile 1 0 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 1 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 2 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 3 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 4 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 5 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 6 0 0 25000000 0 0
    testpmd> add port tm node shaper profile 1 7 0 0 25000000 0 0

5. Add queue leaf node::

    testpmd> add port tm leaf node 1 0 900000 0 1 2 0 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 1 900001 0 1 2 1 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 2 900002 0 1 2 2 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 3 900003 0 1 2 3 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 4 900004 0 1 2 4 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 5 900005 0 1 2 5 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 6 900006 0 1 2 6 0 0xffffffff 0 0
    testpmd> add port tm leaf node 1 7 900007 0 1 2 7 0 0xffffffff 0 0

6. Commit the traffic management hierarchy on the port::

    testpmd> port tm hierarchy commit 1 no
    testpmd> port start all
    testpmd> start

7. Send four flows with user priority 0, 1, 2, 3, 4, 5, 6, 7, 8,
   verify shaping each flow to 200Mbps(25MBps).