test_result - Test Results Records

Record and process DTS results.

The results are recorded in a hierarchical manner:

Each result may contain many intermediate steps, e.g. there are multiple ResultNodes in a ResultNode.

The --output command line argument and the DTS_OUTPUT_DIR environment variable modify the directory where the files with results will be stored.

class Result

Bases: IntEnum

The possible states that a setup, a teardown or a test case may end up in.

PASS = 1
SKIP = 2
BLOCK = 3
FAIL = 4
ERROR = 5
__new__(value)
class ResultLeaf

Bases: BaseModel

Class representing a result in the results tree.

A leaf node that can contain the results for a TestSuite, test_suite.TestCase or a DTS execution step.

result

The actual result.

Type:

framework.test_result.Result

reason

The reason of the result.

Type:

framework.exception.DTSError | None

ExecutionStep

Predefined execution steps.

alias of Literal[‘setup’, ‘teardown’]

class ResultNode

Bases: BaseModel

Class representing a node in the tree of results.

Each node contains a label and a list of children, which can be either ResultNode, or ResultLeaf. This node is serialized as a dictionary of the children. The key of each child is either result in the case of a ResultLeaf, or it is the value of label.

label

The name of the node.

Type:

str

children

A list of either ResultNode or ResultLeaf.

Type:

list[Union[framework.test_result.ResultNode, framework.test_result.ResultLeaf]]

parent

The parent node, if any.

Type:

Optional[framework.test_result.ResultNode]

add_child(label: str) → ResultNode

Creates and append a child node to the model.

Parameters:

label (str) – The name of the node.

mark_result_as(result: Result, ex: Exception | None = None) → None

Mark result for the current step.

Parameters:

  • result (Result) – The result of the current step.

  • ex (Exception | None) – The exception if any occurred. If this is not an instance of DTSError, it is wrapped with an InternalError.

mark_step_as(step: Literal['setup', 'teardown'], result: Result, ex: Exception | None = None) → None

Mark an execution step with the given result.

Parameters:

  • step (Literal['setup', 'teardown']) – Step to mark, e.g.: setup, teardown.

  • result (Result) – The result of the execution step.

  • ex (Exception | None) – The exception if any occurred. If this is not an instance of DTSError, it is wrapped with an InternalError.

serialize_model() → dict[str, Any]

Serializes model output.

get_overall_result() → ResultLeaf

The overall result of the underlying results.

make_summary() → Counter[Result]

Make the summary of the underlying results while ignoring special nodes.

print_results(file: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, indent_level: int = 0, indent_width: int = 2) → None

Print the results in a textual tree format.

class TestRunResult

Bases: BaseModel

Class representing the root node of the results tree.

Root node of the model containing metadata about the DPDK version, ports, compiler and DTS execution results.

sut_session_info

The SUT node OS session information.

Type:

framework.testbed_model.os_session.OSSessionInfo | None

dpdk_build_info

The DPDK build information.

Type:

framework.remote_session.dpdk.DPDKBuildInfo | None

ports

The ports that were used in the test run.

Type:

list[dict[str, str]] | None

test_suites

The test suites containing the results of DTS execution.

Type:

framework.test_result.ResultNode

execution_errors

A list of errors that occur during DTS execution.

Type:

list[framework.exception.DTSError]

serialize_errors(execution_errors: list[framework.exception.DTSError]) → list[str]

Serialize errors as plain text.

add_error(ex: Exception) → None

Add an execution error to the test run result.

property summary: dict[str, int]

The test cases result summary.

property return_code: int

Gather all the errors and return a code by highest severity.

print_summary(file: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None

Print out the textual summary.

dump_json(file: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, /, indent: int = 4) → None

Dump the results as JSON.

process() → int

Process and store all the results, and return the resulting exit code.