rte_sched.h File Reference

#include <rte_compat.h>
#include <rte_mbuf.h>
#include <rte_meter.h>
#include "rte_red.h"
#include "rte_pie.h"

Go to the source code of this file.

Data Structures
struct	rte_sched_subport_stats
struct	rte_sched_queue_stats
struct	rte_sched_port_params

Macros
#define	RTE_SCHED_QUEUES_PER_PIPE   16
#define	RTE_SCHED_BE_QUEUES_PER_PIPE   4
#define	RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE   (RTE_SCHED_QUEUES_PER_PIPE - RTE_SCHED_BE_QUEUES_PER_PIPE + 1)
#define	RTE_SCHED_TRAFFIC_CLASS_BE   (RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE - 1)

Enumerations
enum	rte_sched_cman_mode { RTE_SCHED_CMAN_RED, RTE_SCHED_CMAN_PIE }

Functions
struct rte_sched_port *	rte_sched_port_config (struct rte_sched_port_params *params)
void	rte_sched_port_free (struct rte_sched_port *port)
int	rte_sched_subport_pipe_profile_add (struct rte_sched_port *port, uint32_t subport_id, struct rte_sched_pipe_params *params, uint32_t *pipe_profile_id)
__rte_experimental int	rte_sched_port_subport_profile_add (struct rte_sched_port *port, struct rte_sched_subport_profile_params *profile, uint32_t *subport_profile_id)
int	rte_sched_subport_config (struct rte_sched_port *port, uint32_t subport_id, struct rte_sched_subport_params *params, uint32_t subport_profile_id)
int	rte_sched_pipe_config (struct rte_sched_port *port, uint32_t subport_id, uint32_t pipe_id, int32_t pipe_profile)
uint32_t	rte_sched_port_get_memory_footprint (struct rte_sched_port_params *port_params, struct rte_sched_subport_params **subport_params)
int	rte_sched_subport_read_stats (struct rte_sched_port *port, uint32_t subport_id, struct rte_sched_subport_stats *stats, uint32_t *tc_ov)
int	rte_sched_queue_read_stats (struct rte_sched_port *port, uint32_t queue_id, struct rte_sched_queue_stats *stats, uint16_t *qlen)
void	rte_sched_port_pkt_write (struct rte_sched_port *port, struct rte_mbuf *pkt, uint32_t subport, uint32_t pipe, uint32_t traffic_class, uint32_t queue, enum rte_color color)
void	rte_sched_port_pkt_read_tree_path (struct rte_sched_port *port, const struct rte_mbuf *pkt, uint32_t *subport, uint32_t *pipe, uint32_t *traffic_class, uint32_t *queue)
int	rte_sched_port_enqueue (struct rte_sched_port *port, struct rte_mbuf **pkts, uint32_t n_pkts)
int	rte_sched_port_dequeue (struct rte_sched_port *port, struct rte_mbuf **pkts, uint32_t n_pkts)
__rte_experimental int	rte_sched_subport_tc_ov_config (struct rte_sched_port *port, uint32_t subport_id, bool tc_ov_enable)

Detailed Description

RTE Hierarchical Scheduler

The hierarchical scheduler prioritizes the transmission of packets from different users and traffic classes according to the Service Level Agreements (SLAs) defined for the current network node.

The scheduler supports thousands of packet queues grouped under a 5-level hierarchy:

1. Port:
   • Typical usage: output Ethernet port;
   • Multiple ports are scheduled in round robin order with equal priority;
2. Subport:
   • Typical usage: group of users;
   • Traffic shaping using the token bucket algorithm (one bucket per subport);
   • Upper limit enforced per traffic class at subport level;
   • Lower priority traffic classes able to reuse subport bandwidth currently unused by higher priority traffic classes of the same subport;
   • When any subport traffic class is oversubscribed (configuration time event), the usage of subport member pipes with high demand for that traffic class pipes is truncated to a dynamically adjusted value with no impact to low demand pipes;
3. Pipe:
   • Typical usage: individual user/subscriber;
   • Traffic shaping using the token bucket algorithm (one bucket per pipe);
4. Traffic class:
   • Traffic classes of the same pipe handled in strict priority order;
   • Upper limit enforced per traffic class at the pipe level;
   • Lower priority traffic classes able to reuse pipe bandwidth currently unused by higher priority traffic classes of the same pipe;
5. Queue:
   • Typical usage: queue hosting packets from one or multiple connections of same traffic class belonging to the same user;
   • Weighted Round Robin (WRR) is used to service the queues within same pipe lowest priority traffic class (best-effort).

Definition in file rte_sched.h.

Macro Definition Documentation

RTE_SCHED_QUEUES_PER_PIPE

#define RTE_SCHED_QUEUES_PER_PIPE   16

Congestion Management Maximum number of queues per pipe. Note that the multiple queues (power of 2) can only be assigned to lowest priority (best-effort) traffic class. Other higher priority traffic classes can only have one queue. Can not change.

See also

struct rte_sched_port_params

Examples:

examples/qos_sched/cfg_file.c, and examples/qos_sched/stats.c.

Definition at line 75 of file rte_sched.h.

RTE_SCHED_BE_QUEUES_PER_PIPE

#define RTE_SCHED_BE_QUEUES_PER_PIPE   4

Number of WRR queues for best-effort traffic class per pipe.

See also

struct rte_sched_pipe_params

Examples:

examples/ip_pipeline/cli.c, examples/qos_sched/cfg_file.c, and examples/qos_sched/stats.c.

Definition at line 81 of file rte_sched.h.

RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE

#define RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE   (RTE_SCHED_QUEUES_PER_PIPE - RTE_SCHED_BE_QUEUES_PER_PIPE + 1)

Number of traffic classes per pipe (as well as subport).

See also

struct rte_sched_subport_params

struct rte_sched_pipe_params

Examples:

examples/ip_pipeline/cli.c, examples/qos_sched/cfg_file.c, and examples/qos_sched/stats.c.

Definition at line 87 of file rte_sched.h.

RTE_SCHED_TRAFFIC_CLASS_BE

#define RTE_SCHED_TRAFFIC_CLASS_BE   (RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE - 1)

Best-effort traffic class ID Can not change.

Examples:

examples/qos_sched/app_thread.c, examples/qos_sched/cfg_file.c, and examples/qos_sched/stats.c.

Definition at line 93 of file rte_sched.h.

Enumeration Type Documentation

rte_sched_cman_mode

enum rte_sched_cman_mode

Congestion Management (CMAN) mode

This is used for controlling the admission of packets into a packet queue or group of packet queues on congestion.

The Random Early Detection (RED) algorithm works by proactively dropping more and more input packets as the queue occupancy builds up. When the queue is full or almost full, RED effectively works as tail drop. The Weighted RED algorithm uses a separate set of RED thresholds for each packet color.

Similar to RED, Proportional Integral Controller Enhanced (PIE) randomly drops a packet at the onset of the congestion and tries to control the latency around the target value. The congestion detection, however, is based on the queueing latency instead of the queue length like RED. For more information, refer RFC8033.

Enumerator
RTE_SCHED_CMAN_RED	Random Early Detection (RED)
RTE_SCHED_CMAN_PIE	Proportional Integral Controller Enhanced (PIE)

Definition at line 128 of file rte_sched.h.

Function Documentation

rte_sched_port_config()

struct rte_sched_port* rte_sched_port_config

(

struct rte_sched_port_params *

params

)

Hierarchical scheduler port configuration

Parameters

params

Port scheduler configuration parameter structure

Returns

Handle to port scheduler instance upon success or NULL otherwise.

Examples:

examples/ip_pipeline/tmgr.c, and examples/qos_sched/init.c.

rte_sched_port_free()

void rte_sched_port_free

(

struct rte_sched_port *

port

)

Hierarchical scheduler port free

Parameters

port	Handle to port scheduler instance. If port is NULL, no operation is performed.

Examples:

examples/ip_pipeline/tmgr.c.

rte_sched_subport_pipe_profile_add()

int rte_sched_subport_pipe_profile_add	(	struct rte_sched_port *	port,
		uint32_t	subport_id,
		struct rte_sched_pipe_params *	params,
		uint32_t *	pipe_profile_id
	)

Hierarchical scheduler pipe profile add

Parameters

port	Handle to port scheduler instance
subport_id	Subport ID
params	Pipe profile parameters
pipe_profile_id	Set to valid profile id when profile is added successfully.

Returns

0 upon success, error code otherwise

rte_sched_port_subport_profile_add()

__rte_experimental int rte_sched_port_subport_profile_add	(	struct rte_sched_port *	port,
		struct rte_sched_subport_profile_params *	profile,
		uint32_t *	subport_profile_id
	)

Warning

EXPERIMENTAL: this API may change without prior notice.

Hierarchical scheduler subport bandwidth profile add Note that this function is safe to use in runtime for adding new subport bandwidth profile as it doesn't have any impact on hierarchical structure of the scheduler.

Parameters

port	Handle to port scheduler instance
profile	Subport bandwidth profile
subport_profile_id	Subport profile id

Returns

0 upon success, error code otherwise

rte_sched_subport_config()

int rte_sched_subport_config	(	struct rte_sched_port *	port,
		uint32_t	subport_id,
		struct rte_sched_subport_params *	params,
		uint32_t	subport_profile_id
	)

Hierarchical scheduler subport configuration Note that this function is safe to use at runtime to configure subport bandwidth profile.

Parameters

port	Handle to port scheduler instance
subport_id	Subport ID
params	Subport configuration parameters. Must be non-NULL for first invocation (i.e initialization) for a given subport. Ignored (recommended value is NULL) for all subsequent invocation on the same subport.
subport_profile_id	ID of subport bandwidth profile

Returns

0 upon success, error code otherwise

Examples:

examples/ip_pipeline/tmgr.c, and examples/qos_sched/init.c.

rte_sched_pipe_config()

int rte_sched_pipe_config	(	struct rte_sched_port *	port,
		uint32_t	subport_id,
		uint32_t	pipe_id,
		int32_t	pipe_profile
	)

Hierarchical scheduler pipe configuration

Parameters

port	Handle to port scheduler instance
subport_id	Subport ID
pipe_id	Pipe ID within subport
pipe_profile	ID of subport-level pre-configured pipe profile

Returns

0 upon success, error code otherwise

Examples:

examples/ip_pipeline/tmgr.c, and examples/qos_sched/init.c.

rte_sched_port_get_memory_footprint()

uint32_t rte_sched_port_get_memory_footprint	(	struct rte_sched_port_params *	port_params,
		struct rte_sched_subport_params **	subport_params
	)

Hierarchical scheduler memory footprint size per port

Parameters

port_params	Port scheduler configuration parameter structure
subport_params	Array of subport parameter structures

Returns

Memory footprint size in bytes upon success, 0 otherwise

rte_sched_subport_read_stats()

int rte_sched_subport_read_stats	(	struct rte_sched_port *	port,
		uint32_t	subport_id,
		struct rte_sched_subport_stats *	stats,
		uint32_t *	tc_ov
	)

Hierarchical scheduler subport statistics read

Parameters

port	Handle to port scheduler instance
subport_id	Subport ID
stats	Pointer to pre-allocated subport statistics structure where the statistics counters should be stored
tc_ov	Pointer to pre-allocated RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE-entry array where the oversubscription status for each of the subport traffic classes should be stored.

Returns

0 upon success, error code otherwise

Examples:

examples/qos_sched/stats.c.

rte_sched_queue_read_stats()

int rte_sched_queue_read_stats	(	struct rte_sched_port *	port,
		uint32_t	queue_id,
		struct rte_sched_queue_stats *	stats,
		uint16_t *	qlen
	)

Hierarchical scheduler queue statistics read

Parameters

port	Handle to port scheduler instance
queue_id	Queue ID within port scheduler
stats	Pointer to pre-allocated subport statistics structure where the statistics counters should be stored
qlen	Pointer to pre-allocated variable where the current queue length should be stored.

Returns

0 upon success, error code otherwise

Examples:

examples/qos_sched/stats.c.

rte_sched_port_pkt_write()

void rte_sched_port_pkt_write	(	struct rte_sched_port *	port,
		struct rte_mbuf *	pkt,
		uint32_t	subport,
		uint32_t	pipe,
		uint32_t	traffic_class,
		uint32_t	queue,
		enum rte_color	color
	)

Scheduler hierarchy path write to packet descriptor. Typically called by the packet classification stage.

Parameters

port	Handle to port scheduler instance
pkt	Packet descriptor handle
subport	Subport ID
pipe	Pipe ID within subport
traffic_class	Traffic class ID within pipe (0 .. RTE_SCHED_TRAFFIC_CLASS_BE)
queue	Queue ID within pipe traffic class, 0 for high priority TCs, and 0 .. (RTE_SCHED_BE_QUEUES_PER_PIPE - 1) for best-effort TC
color	Packet color set

Examples:

examples/qos_sched/app_thread.c.

rte_sched_port_pkt_read_tree_path()

void rte_sched_port_pkt_read_tree_path	(	struct rte_sched_port *	port,
		const struct rte_mbuf *	pkt,
		uint32_t *	subport,
		uint32_t *	pipe,
		uint32_t *	traffic_class,
		uint32_t *	queue
	)

Scheduler hierarchy path read from packet descriptor (struct rte_mbuf). Typically called as part of the hierarchical scheduler enqueue operation. The subport, pipe, traffic class and queue parameters need to be pre-allocated by the caller.

Parameters

port	Handle to port scheduler instance
pkt	Packet descriptor handle
subport	Subport ID
pipe	Pipe ID within subport
traffic_class	Traffic class ID within pipe (0 .. RTE_SCHED_TRAFFIC_CLASS_BE)
queue	Queue ID within pipe traffic class, 0 for high priority TCs, and 0 .. (RTE_SCHED_BE_QUEUES_PER_PIPE - 1) for best-effort TC

rte_sched_port_enqueue()

int rte_sched_port_enqueue	(	struct rte_sched_port *	port,
		struct rte_mbuf **	pkts,
		uint32_t	n_pkts
	)

Hierarchical scheduler port enqueue. Writes up to n_pkts to port scheduler and returns the number of packets actually written. For each packet, the port scheduler queue to write the packet to is identified by reading the hierarchy path from the packet descriptor; if the queue is full or congested and the packet is not written to the queue, then the packet is automatically dropped without any action required from the caller.

Parameters

port	Handle to port scheduler instance
pkts	Array storing the packet descriptor handles
n_pkts	Number of packets to enqueue from the pkts array into the port scheduler

Returns

Number of packets successfully enqueued

Examples:

examples/qos_sched/app_thread.c.

rte_sched_port_dequeue()

int rte_sched_port_dequeue	(	struct rte_sched_port *	port,
		struct rte_mbuf **	pkts,
		uint32_t	n_pkts
	)

Hierarchical scheduler port dequeue. Reads up to n_pkts from the port scheduler and stores them in the pkts array and returns the number of packets actually read. The pkts array needs to be pre-allocated by the caller with at least n_pkts entries.

Parameters

port	Handle to port scheduler instance
pkts	Pre-allocated packet descriptor array where the packets dequeued from the port scheduler should be stored
n_pkts	Number of packets to dequeue from the port scheduler

Returns

Number of packets successfully dequeued and placed in the pkts array

Examples:

examples/qos_sched/app_thread.c.

rte_sched_subport_tc_ov_config()

__rte_experimental int rte_sched_subport_tc_ov_config	(	struct rte_sched_port *	port,
		uint32_t	subport_id,
		bool	tc_ov_enable
	)

Hierarchical scheduler subport traffic class oversubscription enable/disable. This function should be called at the time of subport initialization.

Parameters

port	Handle to port scheduler instance
subport_id	Subport ID
tc_ov_enable	Boolean flag to enable/disable TC OV

Returns

0 upon success, error code otherwise