os_session - OS-aware Remote Session ABC

OS-aware remote session.

DPDK supports multiple different operating systems, meaning it can run on these different operating systems. This module defines the common API that OS-unaware layers use and translates the API into OS-aware calls/utility usage.

Note

Running commands with administrative privileges requires OS awareness. This is the only layer that’s aware of OS differences, so this is where non-privileged command get converted to privileged commands.

Example

A user wishes to remove a directory on a remote SutNode. The SutNode object isn’t aware what OS the node is running - it delegates the OS translation logic to main_session. The SUT node calls remove_remote_dir() with a generic, OS-unaware path and the main_session translates that to rm -rf if the node’s OS is Linux and other commands for other OSs. It also translates the path to match the underlying OS.

class FilePermissions

Bases: Flag

The permissions for a file and/or directory.

OTHERS_EXECUTE = 1
OTHERS_WRITE = 2
OTHERS_READ = 4
GROUP_EXECUTE = 8
GROUP_WRITE = 16
GROUP_READ = 32
OWNER_EXECUTE = 64
OWNER_WRITE = 128
OWNER_READ = 256
to_octal() → str

Convert this flag to an octal representation.

class OSSessionInfo

Bases: object

Supplemental OS session information.

os_name

The name of the running operating system of the Node.

Type:

str

os_version

The version of the running operating system of the Node.

Type:

str

kernel_version

The kernel version of the running operating system of the Node.

Type:

str

__init__(os_name: str, os_version: str, kernel_version: str) → None
class OSSession

Bases: ABC

OS-unaware to OS-aware translation API definition.

The OSSession classes create a remote session to a DTS node and implement OS specific behavior. There a few control methods implemented by the base class, the rest need to be implemented by subclasses.

name

The name of the session.

Type:

str

remote_session

The remote session maintaining the connection to the node.

Type:

framework.remote_session.remote_session.RemoteSession

interactive_session

The interactive remote session maintaining the connection to the node.

Type:

framework.remote_session.interactive_remote_session.InteractiveRemoteSession

__init__(node_config: NodeConfiguration, name: str, logger: DTSLogger)

Initialize the OS-aware session.

Connect to the node right away and also create an interactive remote session.

Parameters:

  • node_config (NodeConfiguration) – The test run configuration of the node to connect to.

  • name (str) – The name of the session.

  • logger (DTSLogger) – The logger instance this session will use.

is_alive() → bool

Check whether the underlying remote session is still responding.

send_command(command: str, timeout: float = 15, privileged: bool = False, verify: bool = False, env: dict | None = None) → CommandResult

An all-purpose API for OS-agnostic commands.

This can be used for an execution of a portable command that’s executed the same way on all operating systems, such as Python.

The --timeout command line argument and the DTS_TIMEOUT environment variable configure the timeout of command execution.

Parameters:

  • command (str) – The command to execute.

  • timeout (float) – Wait at most this long in seconds for command execution to complete.

  • privileged (bool) – Whether to run the command with administrative privileges.

  • verify (bool) – If True, will check the exit code of the command.

  • env (dict | None) – A dictionary with environment variables to be used with the command execution.

Raises:

RemoteCommandExecutionError – If verify is True and the command failed.

close() → None

Close the underlying remote session.

abstract get_remote_tmp_dir() → PurePath

Get the path of the temporary directory of the remote OS.

Returns:

The absolute path of the temporary directory.

Return type:

PurePath

abstract get_dpdk_build_env_vars(arch: Architecture) → dict

Create extra environment variables needed for the target architecture.

Different architectures may require different configuration, such as setting 32-bit CFLAGS.

Returns:

A dictionary with keys as environment variables.

Return type:

dict

abstract join_remote_path(*args: str | pathlib.PurePath) → PurePath

Join path parts using the path separator that fits the remote OS.

Parameters:

args (str | pathlib.PurePath) – Any number of paths to join.

Returns:

The resulting joined path.

Return type:

PurePath

abstract remote_path_exists(remote_path: str | pathlib.PurePath) → bool

Check whether remote_path exists on the remote system.

Parameters:

remote_path (str | pathlib.PurePath) – The path to check.

Returns:

True if the path exists, False otherwise.

Return type:

bool

abstract create_tmp_dir(template: str = 'dts.XXXXX') → PurePath

Create a temporary directory on the remote node.

Parameters:

template (str) – The template to use for the name of the directory. “X”s are treated as placeholder.

Returns:

The path to the created directory.

Return type:

PurePath

abstract copy_from(source_file: str | pathlib.PurePath, destination_dir: str | pathlib.Path) → None

Copy a file from the remote node to the local filesystem.

Copy source_file from the remote node associated with this remote session to destination_dir on the local filesystem.

Parameters:

  • source_file (str | pathlib.PurePath) – The file on the remote node.

  • destination_dir (str | pathlib.Path) – The directory path on the local filesystem where the source_file will be saved.

abstract copy_to(source_file: str | pathlib.Path, destination_dir: str | pathlib.PurePath) → None

Copy a file from local filesystem to the remote node.

Copy source_file from local filesystem to destination_dir on the remote node associated with this remote session.

Parameters:

  • source_file (str | pathlib.Path) – The file on the local filesystem.

  • destination_dir (str | pathlib.PurePath) – The directory path on the remote Node where the source_file will be saved.

abstract copy_dir_from(source_dir: str | pathlib.PurePath, destination_dir: str | pathlib.Path, compress_format: TarCompressionFormat = TarCompressionFormat.none, exclude: str | list[str] | None = None) → None

Copy a directory from the remote node to the local filesystem.

Copy source_dir from the remote node associated with this remote session to destination_dir on the local filesystem. The new local directory will be created at destination_dir path.

Example

source_dir = ‘/remote/path/to/source’ destination_dir = ‘/local/path/to/destination’ compress_format = TarCompressionFormat.xz

The method will:
  1. Create a tarball from source_dir, resulting in:

    ‘/remote/path/to/source.tar.xz’,

  2. Copy ‘/remote/path/to/source.tar.xz’ to

    ‘/local/path/to/destination/source.tar.xz’,

  3. Extract the contents of the tarball, resulting in:

    ‘/local/path/to/destination/source/’,

  4. Remove the tarball after extraction

    (‘/local/path/to/destination/source.tar.xz’).

Final Path Structure:

‘/local/path/to/destination/source/’

Parameters:

  • source_dir (str | pathlib.PurePath) – The directory on the remote node.

  • destination_dir (str | pathlib.Path) – The directory path on the local filesystem.

  • compress_format (TarCompressionFormat) – The compression format to use. Defaults to no compression.

  • exclude (str | list[str] | None) – Patterns for files or directories to exclude from the tarball. These patterns are used with tar’s –exclude option.

abstract copy_dir_to(source_dir: str | pathlib.Path, destination_dir: str | pathlib.PurePath, compress_format: TarCompressionFormat = TarCompressionFormat.none, exclude: str | list[str] | None = None) → None

Copy a directory from the local filesystem to the remote node.

Copy source_dir from the local filesystem to destination_dir on the remote node associated with this remote session. The new remote directory will be created at destination_dir path.

Example

source_dir = ‘/local/path/to/source’ destination_dir = ‘/remote/path/to/destination’ compress_format = TarCompressionFormat.xz

The method will:
  1. Create a tarball from source_dir, resulting in:

    ‘/local/path/to/source.tar.xz’,

  2. Copy ‘/local/path/to/source.tar.xz’ to

    ‘/remote/path/to/destination/source.tar.xz’,

  3. Extract the contents of the tarball, resulting in:

    ‘/remote/path/to/destination/source/’,

  4. Remove the tarball after extraction

    (‘/remote/path/to/destination/source.tar.xz’).

Final Path Structure:

‘/remote/path/to/destination/source/’

Parameters:

  • source_dir (str | pathlib.Path) – The directory on the local filesystem.

  • destination_dir (str | pathlib.PurePath) – The directory path on the remote node.

  • compress_format (TarCompressionFormat) – The compression format to use. Defaults to no compression.

  • exclude (str | list[str] | None) – Patterns for files or directories to exclude from the tarball. These patterns are used with fnmatch.fnmatch to filter out files.

abstract change_permissions(remote_path: PurePath, permissions: FilePermissions, recursive: bool = False) → None

Change the permissions of the given path.

abstract remove_remote_file(remote_file_path: str | pathlib.PurePath, force: bool = True) → None

Remove remote file, by default remove forcefully.

Parameters:

  • remote_file_path (str | pathlib.PurePath) – The file path to remove.

  • force (bool) – If True, ignore all warnings and try to remove at all costs.

abstract remove_remote_dir(remote_dir_path: str | pathlib.PurePath, recursive: bool = True, force: bool = True) → None

Remove remote directory, by default remove recursively and forcefully.

Parameters:

  • remote_dir_path (str | pathlib.PurePath) – The directory path to remove.

  • recursive (bool) – If True, also remove all contents inside the directory.

  • force (bool) – If True, ignore all warnings and try to remove at all costs.

abstract create_remote_tarball(remote_dir_path: str | pathlib.PurePath, compress_format: TarCompressionFormat = TarCompressionFormat.none, exclude: str | list[str] | None = None) → PurePosixPath

Create a tarball from the contents of the specified remote directory.

This method creates a tarball containing all files and directories within remote_dir_path. The tarball will be saved in the directory of remote_dir_path and will be named based on remote_dir_path.

Parameters:

  • remote_dir_path (str | pathlib.PurePath) – The directory path on the remote node.

  • compress_format (TarCompressionFormat) – The compression format to use. Defaults to no compression.

  • exclude (str | list[str] | None) – Patterns for files or directories to exclude from the tarball. These patterns are used with tar’s –exclude option.

Returns:

The path to the created tarball on the remote node.

Return type:

PurePosixPath

abstract create_directory(path: PurePath) → None

Create a directory at a specified path.

abstract extract_remote_tarball(remote_tarball_path: str | pathlib.PurePath, destination_path: str | pathlib.PurePath, strip_root_dir: bool = False) → None

Extract remote tarball in the given path.

Parameters:

  • remote_tarball_path (str | pathlib.PurePath) – The tarball path on the remote node.

  • destination_path (str | pathlib.PurePath) – The location the tarball will be extracted to.

  • strip_root_dir (bool) – If True and the root of the tarball is a folder, strip it and extract its contents only.

abstract is_remote_dir(remote_path: PurePath) → bool

Check if the remote_path is a directory.

Parameters:

remote_path (PurePath) – The path to the remote tarball.

Returns:

If True the remote_path is a directory, otherwise False.

Return type:

bool

abstract is_remote_tarfile(remote_tarball_path: PurePath) → bool

Check if the remote_tarball_path is a tar archive.

Parameters:

remote_tarball_path (PurePath) – The path to the remote tarball.

Returns:

If True the remote_tarball_path is a tar archive, otherwise False.

Return type:

bool

abstract get_tarball_top_dir(remote_tarball_path: str | pathlib.PurePath) → str | pathlib.PurePosixPath | None

Get the top directory of the remote tarball.

Examines the contents of a tarball located at the given remote_tarball_path and determines the top-level directory. If all files and directories in the tarball share the same top-level directory, that directory name is returned. If the tarball contains multiple top-level directories or is empty, the method return None.

Parameters:

remote_tarball_path (str | pathlib.PurePath) – The path to the remote tarball.

Returns:

The top directory of the tarball. If there are multiple top directories or the tarball is empty, returns None.

Return type:

str | pathlib.PurePosixPath | None

abstract build_dpdk(env_vars: dict, meson_args: MesonArgs, remote_dpdk_dir: str | pathlib.PurePath, remote_dpdk_build_dir: str | pathlib.PurePath, rebuild: bool = False, timeout: float = 1200) → None

Build DPDK on the remote node.

An extracted DPDK tarball must be present on the node. The build consists of two steps:

meson setup <meson args> remote_dpdk_dir remote_dpdk_build_dir
ninja -C remote_dpdk_build_dir

The --compile-timeout command line argument and the DTS_COMPILE_TIMEOUT environment variable configure the timeout of DPDK build.

Parameters:

  • env_vars (dict) – Use these environment variables when building DPDK.

  • meson_args (MesonArgs) – Use these meson arguments when building DPDK.

  • remote_dpdk_dir (str | pathlib.PurePath) – The directory on the remote node where DPDK will be built.

  • remote_dpdk_build_dir (str | pathlib.PurePath) – The target build directory on the remote node.

  • rebuild (bool) – If True, do a subsequent build with meson configure instead of meson setup.

  • timeout (float) – Wait at most this long in seconds for the build execution to complete.

abstract get_dpdk_version(version_path: str | pathlib.PurePath) → str

Inspect the DPDK version on the remote node.

Parameters:

version_path (str | pathlib.PurePath) – The path to the VERSION file containing the DPDK version.

Returns:

The DPDK version.

Return type:

str

abstract get_remote_cpus() → list[framework.testbed_model.cpu.LogicalCore]

Get the list of LogicalCores on the remote node.

Parameters:

use_first_core – If False, the first physical core won’t be used.

Returns:

The logical cores present on the node.

Return type:

list[framework.testbed_model.cpu.LogicalCore]

abstract kill_cleanup_dpdk_apps(dpdk_prefix_list: Iterable[str]) → None

Kill and cleanup all DPDK apps.

Parameters:

dpdk_prefix_list (Iterable[str]) – Kill all apps identified by dpdk_prefix_list. If dpdk_prefix_list is empty, attempt to find running DPDK apps to kill and clean.

abstract get_dpdk_file_prefix(dpdk_prefix: str) → str

Make OS-specific modification to the DPDK file prefix.

Parameters:

dpdk_prefix (str) – The OS-unaware file prefix.

Returns:

The OS-specific file prefix.

Return type:

str

abstract setup_hugepages(number_of: int, hugepage_size: int, force_first_numa: bool) → None

Configure hugepages on the node.

Get the node’s Hugepage Size, configure the specified count of hugepages if needed and mount the hugepages if needed.

Parameters:

  • number_of (int) – Configure this many hugepages.

  • hugepage_size (int) – Configure hugepages of this size.

  • force_first_numa (bool) – If True, configure just on the first numa node.

abstract get_compiler_version(compiler_name: str) → str

Get installed version of compiler used for DPDK.

Parameters:

compiler_name (str) – The name of the compiler executable.

Returns:

The compiler’s version.

Return type:

str

abstract get_node_info() → OSSessionInfo

Collect additional information about the node.

Returns:

Node information.

Return type:

OSSessionInfo

abstract get_arch_info() → str

Discover CPU architecture of the remote host.

Returns:

Remote host CPU architecture.

Return type:

str

abstract get_port_info(pci_address: str) → PortInfo

Get port information.

Returns:

An instance of PortInfo.

Raises:

ConfigurationError – If the port could not be found.

Return type:

PortInfo

abstract bind_ports_to_driver(ports: list[framework.testbed_model.port.Port], driver_name: str) → None

Bind ports to the given driver_name.

Parameters:

  • ports (list[framework.testbed_model.port.Port]) – The list of the ports to bind to the driver.

  • driver_name (str) – The name of the driver to bind the ports to.

Send operating system specific command for bringing up link on node interfaces.

Parameters:

ports (Iterable[Port]) – The ports to apply the link up command to.

abstract configure_port_mtu(mtu: int, port: Port) → None

Configure mtu on port.

Parameters:

  • mtu (int) – Desired MTU value.

  • port (Port) – Port to set mtu on.

abstract create_vfs(pf_port: Port) → None

Creates virtual functions for pf_port.

Checks how many virtual functions (VFs) pf_port supports, and creates that number of VFs on the port.

Parameters:

pf_port (Port) – The port to create virtual functions on.

Raises:

  • InternalError – If the number of VFs is greater than 0 but less than the

  • maximum for pf_port. –

abstract delete_vfs(pf_port: Port) → None

Deletes virtual functions for pf_port.

Checks how many virtual functions (VFs) pf_port supports, and deletes that number of VFs on the port.

Parameters:

pf_port (Port) – The port to delete virtual functions on.

Raises:

  • InternalError – If the number of VFs is greater than 0 but less than the

  • maximum for pf_port. –

abstract get_pci_addr_of_vfs(pf_port: Port) → list[str]

Find the PCI addresses of all virtual functions (VFs) on the port pf_port.

Parameters:

pf_port (Port) – The port to find the VFs on.

Returns:

A list containing all of the PCI addresses of the VFs on the port. If the port has no VFs then the list will be empty.

Return type:

list[str]