capability - Testbed Capabilities

Testbed capabilities.

This module provides a protocol that defines the common attributes of test cases and suites and support for test environment capabilities.

Many test cases are testing features not available on all hardware. On the other hand, some test cases or suites may not need the most complex topology available.

The module allows developers to mark test cases or suites a requiring certain hardware capabilities or a particular topology with the requires() decorator.

There are differences between hardware and topology capabilities:

• Hardware capabilities are assumed to not be required when not specified.

• However, some topology is always available, so each test case or suite is assigned a default topology if no topology is specified in the decorator.

The module also allows developers to mark test cases or suites as requiring certain hardware capabilities with the requires() decorator.

Examples

from framework.test_suite import TestSuite, func_test
from framework.testbed_model.capability import TopologyType, requires
# The whole test suite (each test case within) doesn't require any links.
@requires(topology_type=TopologyType.no_link)
@func_test
class TestHelloWorld(TestSuite):
    def hello_world_single_core(self):
    ...

from framework.test_suite import TestSuite, func_test
from framework.testbed_model.capability import NicCapability, requires
class TestPmdBufferScatter(TestSuite):
    # only the test case requires the SCATTERED_RX_ENABLED capability
    # other test cases may not require it
    @requires(NicCapability.SCATTERED_RX_ENABLED)
    @func_test
    def test_scatter_mbuf_2048(self):

class Capability

Bases: ABC

The base class for various capabilities.

The same capability should always be represented by the same object, meaning the same capability required by different test cases or suites should point to the same object.

Example

test_case1 and test_case2 each require capability1 and in both instances, capability1 should point to the same capability object.

It is up to the subclasses how they implement this.

The instances are used in sets so they must be hashable.

capabilities_to_check: ClassVar[set[typing_extensions.Self]] = {}

A set storing the capabilities whose support should be checked.

register_to_check() → Callable[[SutNode, Topology], set[typing_extensions.Self]]

Register the capability to be checked for support.

Returns:

The callback function that checks the support of capabilities of the particular subclass which should be called after all capabilities have been registered.

Return type:

Callable[[SutNode, Topology], set[typing_extensions.Self]]

add_to_required(test_case_or_suite: type[framework.testbed_model.capability.TestProtocol]) → None

Add the capability instance to the required test case or suite’s capabilities.

Parameters:

test_case_or_suite (type[framework.testbed_model.capability.TestProtocol]) – The test case or suite among whose required capabilities to add this instance.

abstract classmethod get_supported_capabilities(sut_node: SutNode, topology: Topology) → set[typing_extensions.Self]

Get the support status of each registered capability.

Each subclass must implement this method and return the subset of supported capabilities of capabilities_to_check.

Parameters:

• sut_node (SutNode) – The SUT node of the current test run.

• topology (Topology) – The topology of the current test run.

Returns:

The supported capabilities.

Return type:

set[typing_extensions.Self]

class DecoratedNicCapability

Bases: Capability

A wrapper around NicCapability.

New instances should be created with the create_unique() class method to ensure there are no duplicate instances.

nic_capability

The NIC capability that defines each instance.

Type:

framework.remote_session.testpmd_shell.NicCapability

capability_fn

The capability retrieval function of nic_capability.

Type:

collections.abc.Callable[[framework.remote_session.testpmd_shell.TestPmdShell, collections.abc.MutableSet[framework.remote_session.testpmd_shell.NicCapability], collections.abc.MutableSet[framework.remote_session.testpmd_shell.NicCapability]], None]

capability_decorator

The decorator function of nic_capability. This function will wrap capability_fn.

Type:

collections.abc.Callable[[collections.abc.Callable[[Concatenate[framework.remote_session.testpmd_shell.TestPmdShell, ~P]], Any]], collections.abc.Callable[[Concatenate[framework.remote_session.testpmd_shell.TestPmdShell, ~P]], Any]] | None

classmethod get_unique(nic_capability: NicCapability) → DecoratedNicCapability

Get the capability uniquely identified by nic_capability.

This is a factory method that implements a quasi-enum pattern. The instances of this class are stored in an internal class variable, _unique_capabilities.

If an instance identified by nic_capability doesn’t exist, it is created and added to _unique_capabilities. If it exists, it is returned so that a new identical instance is not created.

Parameters:

nic_capability (NicCapability) – The NIC capability.

Returns:

The capability uniquely identified by nic_capability.

Return type:

DecoratedNicCapability

classmethod get_supported_capabilities(sut_node: SutNode, topology: Topology) → set[framework.testbed_model.capability.DecoratedNicCapability]

Overrides get_supported_capabilities().

The capabilities are first sorted by decorators, then reduced into a single function which is then passed to the decorator. This way we execute each decorator only once. Each capability is first checked whether it’s supported/unsupported before executing its capability_fn so that each capability is retrieved only once.

__init__(nic_capability: NicCapability, capability_fn: Callable[[TestPmdShell, MutableSet[NicCapability], MutableSet[NicCapability]], None], capability_decorator: collections.abc.Callable[[collections.abc.Callable[[Concatenate[framework.remote_session.testpmd_shell.TestPmdShell, P]], Any]], collections.abc.Callable[[Concatenate[framework.remote_session.testpmd_shell.TestPmdShell, P]], Any]] | None) → None

class TopologyCapability

Bases: Capability

A wrapper around TopologyType.

Each test case must be assigned a topology. It could be done explicitly; the implicit default is default, which this class defines as equal to two_links.

Test case topology may be set by setting the topology for the whole suite. The priority in which topology is set is as follows:

1. The topology set using the requires() decorator with a test case,

2. The topology set using the requires() decorator with a test suite,

3. The default topology if the decorator is not used.

The default topology of test suite (i.e. when not using the decorator or not setting the topology with the decorator) does not affect the topology of test cases.

New instances should be created with the create_unique() class method to ensure there are no duplicate instances.

topology_type

The topology type that defines each instance.

Type:

framework.testbed_model.topology.TopologyType

classmethod get_unique(topology_type: TopologyType) → TopologyCapability

Get the capability uniquely identified by topology_type.

This is a factory method that implements a quasi-enum pattern. The instances of this class are stored in an internal class variable, _unique_capabilities.

If an instance identified by topology_type doesn’t exist, it is created and added to _unique_capabilities. If it exists, it is returned so that a new identical instance is not created.

Parameters:

topology_type (TopologyType) – The topology type.

Returns:

The capability uniquely identified by topology_type.

Return type:

TopologyCapability

classmethod get_supported_capabilities(sut_node: SutNode, topology: Topology) → set[framework.testbed_model.capability.TopologyCapability]

Overrides get_supported_capabilities().

set_required(test_case_or_suite: type[framework.testbed_model.capability.TestProtocol]) → None

The logic for setting the required topology of a test case or suite.

Decorators are applied on methods of a class first, then on the class. This means we have to modify test case topologies when processing the test suite topologies. At that point, the test case topologies have been set by the requires() decorator. The test suite topology only affects the test case topologies if not default.

__init__(topology_type: TopologyType) → None

class TestProtocol

Bases: Protocol

Common test suite and test case attributes.

skip: ClassVar[bool] = False

Whether to skip the test case or suite.

skip_reason: ClassVar[str] = ''

The reason for skipping the test case or suite.

topology_type: ClassVar[TopologyCapability] = TopologyType.two_links

The topology type of the test case or suite.

required_capabilities: ClassVar[set[framework.testbed_model.capability.Capability]] = {}

The capabilities the test case or suite requires in order to be executed.

classmethod get_test_cases() → list[type['TestCase']]

Get test cases. Should be implemented by subclasses containing test cases.

Raises:

NotImplementedError – The subclass does not implement the method.

__init__(*args, **kwargs)

requires(*nic_capabilities: NicCapability, topology_type: TopologyType = TopologyType.two_links) → Callable[[type[framework.testbed_model.capability.TestProtocol]], type[framework.testbed_model.capability.TestProtocol]]

A decorator that adds the required capabilities to a test case or test suite.

Parameters:

• nic_capabilities (NicCapability) – The NIC capabilities that are required by the test case or test suite.

• topology_type (TopologyType) – The topology type the test suite or case requires.

Returns:

The decorated test case or test suite.

Return type:

Callable[[type[framework.testbed_model.capability.TestProtocol]], type[framework.testbed_model.capability.TestProtocol]]

get_supported_capabilities(sut_node: SutNode, topology_config: Topology, capabilities_to_check: set[framework.testbed_model.capability.Capability]) → set[framework.testbed_model.capability.Capability]

Probe the environment for capabilities_to_check and return the supported ones.

Parameters:

• sut_node (SutNode) – The SUT node to check for capabilities.

• topology_config (Topology) – The topology config to check for capabilities.

• capabilities_to_check (set[framework.testbed_model.capability.Capability]) – The capabilities to check.

Returns:

The capabilities supported by the environment.

Return type:

set[framework.testbed_model.capability.Capability]