54. Event DMA Adapter Library

DPDK eventdev library provides event driven programming model with features to schedule events. DMA device library provides an interface to DMA poll mode drivers that support DMA operations. Event DMA adapter is intended to bridge between the event device and the DMA device.

Packet flow from DMA device to the event device can be accomplished using software and hardware based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to be used. The adapter uses an EAL service core function for software-based packet transfer and uses the eventdev PMD functions to configure hardware-based packet transfer between DMA device and the event device. DMA adapter uses a new event type called RTE_EVENT_TYPE_DMADEV to indicate the source of event.

Application can choose to submit a DMA operation directly to a DMA device or send it to a DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. Choice of mode can be specified while creating the adapter. In the former mode, it is the application’s responsibility to enable ingress packet ordering. In the latter mode, it is the adapter’s responsibility to enable ingress packet ordering.

54.1. Adapter Modes

54.1.1. RTE_EVENT_DMA_ADAPTER_OP_NEW mode

In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to an DMA device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events to the event device. This mode does not ensure ingress ordering as the application directly enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued from the adapter are treated as new events. The application has to specify event information (response information) which is needed to enqueue an event after the DMA operation is completed.

../_images/event_dma_adapter_op_new.svg

Fig. 54.1 Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode

54.1.2. RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode

In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode, if the event PMD and DMA PMD supports internal event port (RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD), the application should use rte_event_dma_adapter_enqueue() API to enqueue DMA operations as events to DMA adapter. If not, application retrieves DMA adapter’s event port using rte_event_dma_adapter_event_port_get() API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev using rte_event_enqueue_burst(). The adapter then dequeues the events and submits the DMA operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the event device.

Applications can use this mode when ingress packet ordering is needed. In this mode, events dequeued from the adapter will be treated as forwarded events. Application has to specify event information (response information) needed to enqueue the event after the DMA operation has completed.

../_images/event_dma_adapter_op_forward.svg

Fig. 54.2 Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode

54.2. API Overview

This section has a brief introduction to the event DMA adapter APIs. The application is expected to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the adapter instance.

54.2.1. Create an adapter instance

An adapter instance is created using rte_event_dma_adapter_create(). This function is called with event device to be associated with the adapter and port configuration for the adapter to setup an event port (if the adapter needs to use a service function).

Adapter can be started in RTE_EVENT_DMA_ADAPTER_OP_NEW or RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode.

enum rte_event_dma_adapter_mode mode;
struct rte_event_dev_info dev_info;
struct rte_event_port_conf conf;
uint8_t evdev_id;
uint8_t dma_id;
int ret;

ret = rte_event_dev_info_get(dma_id, &dev_info);

conf.new_event_threshold = dev_info.max_num_events;
conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);

rte_event_dma_adapter_create_ext() function can be used by the application to have a finer control on eventdev port allocation and setup. The rte_event_dma_adapter_create_ext() function is passed a callback function. The callback function is invoked if the adapter creates a service function and uses an event port for it. The callback is expected to fill the struct rte_event_dma_adapter_conf passed to it.

In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode, if the event PMD and DMA PMD supports internal event port (RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD), events with DMA operations should be enqueued to the DMA adapter using rte_event_dma_adapter_enqueue() API. If not, the event port created by the adapter can be retrieved using rte_event_dma_adapter_event_port_get() API. An application can use this event port to link with an event queue, on which it enqueues events towards the DMA adapter using rte_event_enqueue_burst().

uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
struct rte_event ev;
uint32_t cap;
int ret;

/* Fill in event info and update event_ptr with rte_dma_op */
memset(&ev, 0, sizeof(ev));
.
.
ev.event_ptr = op;

ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
        ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
} else {
        ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
        ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
        ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
        ev.queue_id = app_qid;
        ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
}

54.2.2. Event device configuration for service based adapter

When rte_event_dma_adapter_create() is used for creating adapter instance, rte_event_dev_config::nb_event_ports is automatically incremented, and event device is reconfigured with additional event port during service initialization. This event device reconfigure logic also increments the rte_event_dev_config::nb_single_link_event_port_queues parameter if the adapter event port config is of type RTE_EVENT_PORT_CFG_SINGLE_LINK.

Applications using this mode of adapter creation need not configure the event device with rte_event_dev_config::nb_event_ports and rte_event_dev_config::nb_single_link_event_port_queues parameters required for DMA adapter when the adapter is created using the above-mentioned API.

54.2.3. Querying adapter capabilities

The rte_event_dma_adapter_caps_get() function allows the application to query the adapter capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev are connected using internal HW port or not.

rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);

54.2.4. Adding vchan to the adapter instance

dmadev device ID and vchan are configured using dmadev APIs. For more information, see dmadev.

struct rte_dma_vchan_conf vchan_conf;
struct rte_dma_conf dev_conf;
uint8_t dev_id = 0;
uint16_t vchan = 0;

rte_dma_configure(dev_id, &dev_conf);
rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);

These dmadev ID and vchan are added to the instance using the rte_event_dma_adapter_vchan_add() API. The same is removed using rte_event_dma_adapter_vchan_del() API. If hardware supports RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability, event information must be passed to the add API.

uint32_t cap;
int ret;

ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
        struct rte_event event;

        rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
} else
        rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);

54.2.5. Configuring service function

If the adapter uses a service function, the application is required to assign a service core to the service function as show below.

uint32_t service_id;

if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
        rte_service_map_lcore_set(service_id, CORE_ID);

54.2.6. Set event response information

In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application specifies the dmadev ID and vchan ID in struct rte_event_dma_adapter_op and the event information (response information) needed to enqueue an event after the DMA operation has completed. The response information is specified in struct rte_event and appended to the struct rte_event_dma_adapter_op.

54.2.7. Start the adapter instance

The application calls rte_event_dma_adapter_start() to start the adapter. This function calls the start callbacks of the eventdev PMDs for hardware-based eventdev-dmadev connections and rte_service_run_state_set() to enable the service function if one exists.

rte_event_dma_adapter_start(id);

Note

The eventdev to which the event_dma_adapter is connected should be started before calling rte_event_dma_adapter_start().

54.2.8. Get adapter statistics

The rte_event_dma_adapter_stats_get() function reports counters defined in struct rte_event_dma_adapter_stats. The received packet and enqueued event counts are a sum of the counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by the service function, if one exists.

54.2.9. Set/Get adapter runtime configuration parameters

The runtime configuration parameters of adapter can be set/get using rte_event_dma_adapter_runtime_params_set() and rte_event_dma_adapter_runtime_params_get() respectively. The parameters that can be set/get are defined in struct rte_event_dma_adapter_runtime_params.