7. RX/TX Callbacks Sample Application
7.1. Overview
The RX/TX Callbacks sample application is a packet forwarding application that demonstrates the use of user defined callbacks on received and transmitted packets. The application performs a simple latency check, using callbacks, to determine the time that packets spend within the application.
In the sample application, a user defined callback is applied to all received packets to add a timestamp. A separate callback is applied to all packets prior to transmission to calculate the elapsed time in CPU cycles.
If hardware timestamping is supported by the NIC, the sample application will also display the average latency. The packet was timestamped in hardware on top of the latency since the packet was received and processed by the RX callback.
7.2. Compiling the Application
To compile the sample application, see Compiling the Sample Applications.
The application is located in the rxtx_callbacks
sub-directory.
7.3. Running the Application
To run the example in a linux
environment:
./<build_dir>/examples/dpdk-rxtx_callbacks -l 1 -n 4 -- [-t]
Use -t to enable hardware timestamping. If not supported by the NIC, an error will be displayed.
Refer to DPDK Getting Started Guide for general information on running applications and the Environment Abstraction Layer (EAL) options.
7.4. Explanation
The rxtx_callbacks
application is mainly a simple forwarding application
based on the Basic Forwarding Sample Application. See that section of the documentation for more
details of the forwarding part of the application.
The sections below explain the additional RX/TX callback code.
7.4.1. The Main Function
The main()
function performs the application initialization and calls the
execution threads for each lcore. This function is effectively identical to
the main()
function explained in Basic Forwarding Sample Application.
The lcore_main()
function is also identical.
The main difference is in the user defined port_init()
function where the
callbacks are added. This is explained in the next section:
7.4.2. The Port Initialization Function
The main functional part of the port initialization is shown below with comments:
static inline int
port_init(uint16_t port, struct rte_mempool *mbuf_pool)
{
struct rte_eth_conf port_conf;
const uint16_t rx_rings = 1, tx_rings = 1;
uint16_t nb_rxd = RX_RING_SIZE;
uint16_t nb_txd = TX_RING_SIZE;
int retval;
uint16_t q;
struct rte_eth_dev_info dev_info;
struct rte_eth_rxconf rxconf;
struct rte_eth_txconf txconf;
if (!rte_eth_dev_is_valid_port(port))
return -1;
memset(&port_conf, 0, sizeof(struct rte_eth_conf));
retval = rte_eth_dev_info_get(port, &dev_info);
if (retval != 0) {
printf("Error during getting device (port %u) info: %s\n",
port, strerror(-retval));
return retval;
}
if (dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE)
port_conf.txmode.offloads |=
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
if (hw_timestamping) {
if (!(dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
printf("\nERROR: Port %u does not support hardware timestamping\n"
, port);
return -1;
}
port_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_TIMESTAMP;
rte_mbuf_dyn_rx_timestamp_register(&hwts_dynfield_offset, NULL);
if (hwts_dynfield_offset < 0) {
printf("ERROR: Failed to register timestamp field\n");
return -rte_errno;
}
}
retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &port_conf);
if (retval != 0)
return retval;
retval = rte_eth_dev_adjust_nb_rx_tx_desc(port, &nb_rxd, &nb_txd);
if (retval != 0)
return retval;
rxconf = dev_info.default_rxconf;
for (q = 0; q < rx_rings; q++) {
retval = rte_eth_rx_queue_setup(port, q, nb_rxd,
rte_eth_dev_socket_id(port), &rxconf, mbuf_pool);
if (retval < 0)
return retval;
}
txconf = dev_info.default_txconf;
txconf.offloads = port_conf.txmode.offloads;
for (q = 0; q < tx_rings; q++) {
retval = rte_eth_tx_queue_setup(port, q, nb_txd,
rte_eth_dev_socket_id(port), &txconf);
if (retval < 0)
return retval;
}
retval = rte_eth_dev_start(port);
if (retval < 0)
return retval;
if (hw_timestamping && ticks_per_cycle_mult == 0) {
uint64_t cycles_base = rte_rdtsc();
uint64_t ticks_base;
retval = rte_eth_read_clock(port, &ticks_base);
if (retval != 0)
return retval;
rte_delay_ms(100);
uint64_t cycles = rte_rdtsc();
uint64_t ticks;
rte_eth_read_clock(port, &ticks);
uint64_t c_freq = cycles - cycles_base;
uint64_t t_freq = ticks - ticks_base;
double freq_mult = (double)c_freq / t_freq;
printf("TSC Freq ~= %" PRIu64
"\nHW Freq ~= %" PRIu64
"\nRatio : %f\n",
c_freq * 10, t_freq * 10, freq_mult);
/* TSC will be faster than internal ticks so freq_mult is > 0
* We convert the multiplication to an integer shift & mult
*/
ticks_per_cycle_mult = (1 << TICKS_PER_CYCLE_SHIFT) / freq_mult;
}
struct rte_ether_addr addr;
retval = rte_eth_macaddr_get(port, &addr);
if (retval < 0) {
printf("Failed to get MAC address on port %u: %s\n",
port, rte_strerror(-retval));
return retval;
}
printf("Port %u MAC: %02"PRIx8" %02"PRIx8" %02"PRIx8
" %02"PRIx8" %02"PRIx8" %02"PRIx8"\n",
(unsigned)port,
RTE_ETHER_ADDR_BYTES(&addr));
retval = rte_eth_promiscuous_enable(port);
if (retval != 0)
return retval;
/* RX and TX callbacks are added to the ports. 8< */
rte_eth_add_rx_callback(port, 0, add_timestamps, NULL);
rte_eth_add_tx_callback(port, 0, calc_latency, NULL);
/* >8 End of RX and TX callbacks. */
return 0;
}
The RX and TX callbacks are added to the ports/queues as function pointers:
rte_eth_add_rx_callback(port, 0, add_timestamps, NULL);
rte_eth_add_tx_callback(port, 0, calc_latency, NULL);
More than one callback can be added and additional information can be passed
to callback function pointers as a void*
. In the examples above NULL
is used.
The add_timestamps()
and calc_latency()
functions are explained below.
7.4.3. The add_timestamps() Callback
The add_timestamps()
callback is added to the RX port and is applied to
all packets received:
static uint16_t
add_timestamps(uint16_t port __rte_unused, uint16_t qidx __rte_unused,
struct rte_mbuf **pkts, uint16_t nb_pkts,
uint16_t max_pkts __rte_unused, void *_ __rte_unused)
{
unsigned i;
uint64_t now = rte_rdtsc();
for (i = 0; i < nb_pkts; i++)
*tsc_field(pkts[i]) = now;
return nb_pkts;
}
The DPDK function rte_rdtsc()
is used to add a cycle count timestamp to
each packet (see the cycles section of the DPDK API Documentation for
details).
7.4.4. The calc_latency() Callback
The calc_latency()
callback is added to the TX port and is applied to all
packets prior to transmission:
static uint16_t
calc_latency(uint16_t port, uint16_t qidx __rte_unused,
struct rte_mbuf **pkts, uint16_t nb_pkts, void *_ __rte_unused)
{
uint64_t cycles = 0;
uint64_t queue_ticks = 0;
uint64_t now = rte_rdtsc();
uint64_t ticks;
unsigned i;
if (hw_timestamping)
rte_eth_read_clock(port, &ticks);
for (i = 0; i < nb_pkts; i++) {
cycles += now - *tsc_field(pkts[i]);
if (hw_timestamping)
queue_ticks += ticks - *hwts_field(pkts[i]);
}
latency_numbers.total_cycles += cycles;
if (hw_timestamping)
latency_numbers.total_queue_cycles += (queue_ticks
* ticks_per_cycle_mult) >> TICKS_PER_CYCLE_SHIFT;
latency_numbers.total_pkts += nb_pkts;
if (latency_numbers.total_pkts > (100 * 1000 * 1000ULL)) {
printf("Latency = %"PRIu64" cycles\n",
latency_numbers.total_cycles / latency_numbers.total_pkts);
if (hw_timestamping) {
printf("Latency from HW = %"PRIu64" cycles\n",
latency_numbers.total_queue_cycles
/ latency_numbers.total_pkts);
}
latency_numbers.total_cycles = 0;
latency_numbers.total_queue_cycles = 0;
latency_numbers.total_pkts = 0;
}
return nb_pkts;
}
The calc_latency()
function accumulates the total number of packets and
the total number of cycles used. Once more than 100 million packets have been
transmitted, the average cycle count per packet is printed out
and the counters are reset.