55. Event DMA Adapter Library
DPDK eventdev library provides event driven programming model with features to schedule events. DMA device library provides an interface to DMA poll mode drivers that support DMA operations. Event DMA adapter is intended to bridge between the event device and the DMA device.
Packet flow from DMA device to the event device can be accomplished
using software and hardware based transfer mechanisms.
The adapter queries an eventdev PMD to determine which mechanism to be used.
The adapter uses an EAL service core function for software-based packet transfer
and uses the eventdev PMD functions to configure hardware-based packet transfer
between DMA device and the event device.
DMA adapter uses a new event type called RTE_EVENT_TYPE_DMADEV
to indicate the source of event.
Application can choose to submit a DMA operation directly to a DMA device
or send it to a DMA adapter via eventdev
based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD
capability.
The first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW
) mode
and the second as the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD
) mode.
Choice of mode can be specified while creating the adapter.
In the former mode, it is the application’s responsibility to enable ingress packet ordering.
In the latter mode, it is the adapter’s responsibility to enable ingress packet ordering.
55.1. Adapter Modes
55.1.1. RTE_EVENT_DMA_ADAPTER_OP_NEW mode
In the RTE_EVENT_DMA_ADAPTER_OP_NEW
mode,
application submits DMA operations directly to an DMA device.
The adapter then dequeues DMA completions from the DMA device
and enqueues them as events to the event device.
This mode does not ensure ingress ordering
as the application directly enqueues to the dmadev without going through DMA/atomic stage.
In this mode, events dequeued from the adapter are treated as new events.
The application has to specify event information (response information)
which is needed to enqueue an event after the DMA operation is completed.
Fig. 55.1 Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW
mode
55.1.2. RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD
mode,
if the event PMD and DMA PMD supports internal event port
(RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD
),
the application should use rte_event_dma_adapter_enqueue()
API
to enqueue DMA operations as events to DMA adapter.
If not, application retrieves DMA adapter’s event port
using rte_event_dma_adapter_event_port_get()
API,
links its event queue to this port
and starts enqueuing DMA operations as events to eventdev
using rte_event_enqueue_burst()
.
The adapter then dequeues the events
and submits the DMA operations to the dmadev.
After the DMA operation is complete,
the adapter enqueues events to the event device.
Applications can use this mode when ingress packet ordering is needed. In this mode, events dequeued from the adapter will be treated as forwarded events. Application has to specify event information (response information) needed to enqueue the event after the DMA operation has completed.
Fig. 55.2 Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD
mode
55.2. API Overview
This section has a brief introduction to the event DMA adapter APIs. The application is expected to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the adapter instance.
55.2.1. Create an adapter instance
An adapter instance is created using rte_event_dma_adapter_create()
.
This function is called with event device
to be associated with the adapter and port configuration
for the adapter to setup an event port (if the adapter needs to use a service function).
Adapter can be started in RTE_EVENT_DMA_ADAPTER_OP_NEW
or RTE_EVENT_DMA_ADAPTER_OP_FORWARD
mode.
enum rte_event_dma_adapter_mode mode;
struct rte_event_dev_info dev_info;
struct rte_event_port_conf conf;
uint8_t evdev_id;
uint8_t dma_id;
int ret;
ret = rte_event_dev_info_get(dma_id, &dev_info);
conf.new_event_threshold = dev_info.max_num_events;
conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
rte_event_dma_adapter_create_ext()
function can be used by the application
to have a finer control on eventdev port allocation and setup.
The rte_event_dma_adapter_create_ext()
function is passed a callback function.
The callback function is invoked if the adapter creates a service function
and uses an event port for it.
The callback is expected to fill the struct rte_event_dma_adapter_conf
passed to it.
In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD
mode,
if the event PMD and DMA PMD supports internal event port
(RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD
),
events with DMA operations should be enqueued to the DMA adapter
using rte_event_dma_adapter_enqueue()
API.
If not, the event port created by the adapter can be retrieved
using rte_event_dma_adapter_event_port_get()
API.
An application can use this event port to link with an event queue,
on which it enqueues events towards the DMA adapter using rte_event_enqueue_burst()
.
uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
struct rte_event ev;
uint32_t cap;
int ret;
/* Fill in event info and update event_ptr with rte_dma_op */
memset(&ev, 0, sizeof(ev));
.
.
ev.event_ptr = op;
ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
} else {
ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
ev.queue_id = app_qid;
ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
}
55.2.2. Event device configuration for service based adapter
When rte_event_dma_adapter_create()
is used for creating adapter instance,
rte_event_dev_config::nb_event_ports
is automatically incremented,
and event device is reconfigured with additional event port during service initialization.
This event device reconfigure logic also
increments the rte_event_dev_config::nb_single_link_event_port_queues
parameter
if the adapter event port config is of type RTE_EVENT_PORT_CFG_SINGLE_LINK
.
Applications using this mode of adapter creation need not configure the event device
with rte_event_dev_config::nb_event_ports
and
rte_event_dev_config::nb_single_link_event_port_queues
parameters
required for DMA adapter when the adapter is created using the above-mentioned API.
55.2.3. Querying adapter capabilities
The rte_event_dma_adapter_caps_get()
function allows the application
to query the adapter capabilities for an eventdev and dmadev combination.
This API provides whether dmadev and eventdev are connected using internal HW port or not.
rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
55.2.4. Adding vchan to the adapter instance
dmadev device ID and vchan are configured using dmadev APIs. For more information, see dmadev.
struct rte_dma_vchan_conf vchan_conf;
struct rte_dma_conf dev_conf;
uint8_t dev_id = 0;
uint16_t vchan = 0;
rte_dma_configure(dev_id, &dev_conf);
rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);
These dmadev ID and vchan are added to the instance
using the rte_event_dma_adapter_vchan_add()
API.
The same is removed using rte_event_dma_adapter_vchan_del()
API.
If hardware supports RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND
capability,
event information must be passed to the add API.
uint32_t cap;
int ret;
ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
struct rte_event event;
rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
} else
rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
55.2.5. Configuring service function
If the adapter uses a service function, the application is required to assign a service core to the service function as show below.
uint32_t service_id;
if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
rte_service_map_lcore_set(service_id, CORE_ID);
55.2.6. Set event response information
In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD
/ RTE_EVENT_DMA_ADAPTER_OP_NEW
mode,
the application specifies the dmadev ID and vchan ID in struct rte_event_dma_adapter_op
and the event information (response information)
needed to enqueue an event after the DMA operation has completed.
The response information is specified in struct rte_event
and appended to the struct rte_event_dma_adapter_op
.
55.2.7. Start the adapter instance
The application calls rte_event_dma_adapter_start()
to start the adapter.
This function calls the start callbacks of the eventdev PMDs
for hardware-based eventdev-dmadev connections
and rte_service_run_state_set()
to enable the service function if one exists.
rte_event_dma_adapter_start(id);
Note
The eventdev to which the event_dma_adapter is connected should be started
before calling rte_event_dma_adapter_start()
.
55.2.8. Get adapter statistics
The rte_event_dma_adapter_stats_get()
function reports counters
defined in struct rte_event_dma_adapter_stats
.
The received packet and enqueued event counts are a sum of the counts
from the eventdev PMD callbacks if the callback is supported,
and the counts maintained by the service function, if one exists.
55.2.9. Set/Get adapter runtime configuration parameters
The runtime configuration parameters of adapter can be set/get using
rte_event_dma_adapter_runtime_params_set()
and
rte_event_dma_adapter_runtime_params_get()
respectively.
The parameters that can be set/get are defined in
struct rte_event_dma_adapter_runtime_params
.