rte_ptr_compress.h File Reference

#include <stdint.h>
#include <inttypes.h>
#include <rte_bitops.h>
#include <rte_branch_prediction.h>
#include <rte_common.h>
#include <rte_debug.h>
#include <rte_vect.h>

Go to the source code of this file.

Macros
#define	RTE_PTR_COMPRESS_BITS_NEEDED_FOR_POINTER_WITHIN_RANGE(mem_length)
#define	RTE_PTR_COMPRESS_BIT_SHIFT_FROM_ALIGNMENT(alignment)   ((alignment) == 0 ? 0 : rte_ctz64((uint64_t)alignment))
#define	RTE_PTR_COMPRESS_CAN_COMPRESS_16_SHIFT(mem_length, obj_alignment)
#define	RTE_PTR_COMPRESS_CAN_COMPRESS_32_SHIFT(mem_length, obj_alignment)

Functions
static __rte_always_inline void	rte_ptr_compress_32_shift (void *ptr_base, void *const *src_table, uint32_t *dest_table, size_t n, uint8_t bit_shift)
static __rte_always_inline void	rte_ptr_decompress_32_shift (void *ptr_base, uint32_t const *src_table, void **dest_table, size_t n, uint8_t bit_shift)
static __rte_always_inline void	rte_ptr_compress_16_shift (void *ptr_base, void *const *src_table, uint16_t *dest_table, size_t n, uint8_t bit_shift)
static __rte_always_inline void	rte_ptr_decompress_16_shift (void *ptr_base, uint16_t const *src_table, void **dest_table, size_t n, uint8_t bit_shift)

Detailed Description

Pointer compression and decompression functions.

When passing arrays full of pointers between threads, memory containing the pointers is copied multiple times which is especially costly between cores. These functions allow us to compress the pointers.

Compression takes advantage of the fact that pointers are usually located in a limited memory region. We compress them by converting them to offsets from a base memory address. Offsets can be stored in fewer bytes.

The compression functions come in two varieties: 32-bit and 16-bit.

To determine how many bits are needed to compress the pointer, calculate the biggest offset possible (highest value pointer - base pointer) and shift the value right according to alignment (shift by exponent of the power of 2 of alignment: aligned by 4 - shift by 2, aligned by 8 - shift by 3, etc.). The resulting value must fit in either 32 or 16 bits. You may use the macros provided in this file to do it programmatically.

For usage example and further explanation please see this library's documentation in the programming guide.

Definition in file rte_ptr_compress.h.

Macro Definition Documentation

RTE_PTR_COMPRESS_BITS_NEEDED_FOR_POINTER_WITHIN_RANGE

#define RTE_PTR_COMPRESS_BITS_NEEDED_FOR_POINTER_WITHIN_RANGE

(

mem_length

)

Value:

(((uint64_t)mem_length) < 2 ? 1 : \
        (sizeof(uint64_t) * CHAR_BIT - \
         rte_clz64((uint64_t)mem_length - 1)))

Calculate how many bits are required to store pointers within a given memory region as offsets. This can help decide which pointer compression functions can be used.

Parameters

mem_length

Length of the memory region the pointers are constrained to.

Returns

Number of bits required to store a value.

Definition at line 56 of file rte_ptr_compress.h.

RTE_PTR_COMPRESS_BIT_SHIFT_FROM_ALIGNMENT

#define RTE_PTR_COMPRESS_BIT_SHIFT_FROM_ALIGNMENT

(

alignment

)

((alignment) == 0 ? 0 : rte_ctz64((uint64_t)alignment))

Calculate how many bits in the address can be dropped without losing any information thanks to the alignment of the address.

Parameters

alignment

Memory alignment.

Returns

Size of shift allowed without dropping any information from the pointer.

Definition at line 70 of file rte_ptr_compress.h.

RTE_PTR_COMPRESS_CAN_COMPRESS_16_SHIFT

#define RTE_PTR_COMPRESS_CAN_COMPRESS_16_SHIFT	(		mem_length,
			obj_alignment
	)

Value:

((RTE_PTR_COMPRESS_BITS_NEEDED_FOR_POINTER_WITHIN_RANGE(mem_length) - \
    RTE_PTR_COMPRESS_BIT_SHIFT_FROM_ALIGNMENT(obj_alignment)) <= 16 ? 1 : 0)

Determine if rte_ptr_compress_16_shift can be used to compress pointers that contain addresses of memory objects whose memory is aligned by a given amount and contained in a given memory region.

Parameters

mem_length	The length of the memory region that contains the objects pointed to.
obj_alignment	The alignment of objects pointed to.

Returns

1 if function can be used, 0 otherwise.

Definition at line 85 of file rte_ptr_compress.h.

RTE_PTR_COMPRESS_CAN_COMPRESS_32_SHIFT

#define RTE_PTR_COMPRESS_CAN_COMPRESS_32_SHIFT	(		mem_length,
			obj_alignment
	)

Value:

((RTE_PTR_COMPRESS_BITS_NEEDED_FOR_POINTER_WITHIN_RANGE(mem_length) - \
    RTE_PTR_COMPRESS_BIT_SHIFT_FROM_ALIGNMENT(obj_alignment)) <= 32 ? 1 : 0)

Determine if rte_ptr_compress_32_shift can be used to compress pointers that contain addresses of memory objects whose memory is aligned by a given amount and contained in a given memory region.

Parameters

mem_length	The length of the memory region that contains the objects pointed to.
obj_alignment	The alignment of objects pointed to.

Returns

1 if function can be used, 0 otherwise.

Definition at line 101 of file rte_ptr_compress.h.

Function Documentation

rte_ptr_compress_32_shift()

static __rte_always_inline void rte_ptr_compress_32_shift ( void *  ptr_base, void *const *  src_table, uint32_t *  dest_table, size_t  n, uint8_t  bit_shift  )

static

Compress pointers into 32-bit offsets from base pointer.

Note

It is programmer's responsibility to ensure the resulting offsets fit into 32 bits. Alignment of the structures pointed to by the pointers allows us to drop bits from the offsets. This is controlled by the bit_shift parameter. This means that if structures are aligned by 8 bytes they must be within 32GB of the base pointer. If there is no such alignment guarantee they must be within 4GB.

Parameters

ptr_base	A pointer used to calculate offsets of pointers in src_table.
src_table	A pointer to an array of pointers.
dest_table	A pointer to an array of compressed pointers returned by this function.
n	The number of objects to compress, must be strictly positive.
bit_shift	Byte alignment of memory pointed to by the pointers allows for bits to be dropped from the offset and hence widen the memory region that can be covered. This controls how many bits are right shifted.

Definition at line 129 of file rte_ptr_compress.h.

rte_ptr_decompress_32_shift()

static __rte_always_inline void rte_ptr_decompress_32_shift ( void *  ptr_base, uint32_t const *  src_table, void **  dest_table, size_t  n, uint8_t  bit_shift  )

static

Decompress pointers from 32-bit offsets from base pointer.

Parameters

ptr_base	A pointer which was used to calculate offsets in src_table.
src_table	A pointer to an array to compressed pointers.
dest_table	A pointer to an array of decompressed pointers returned by this function.
n	The number of objects to decompress, must be strictly positive.
bit_shift	Byte alignment of memory pointed to by the pointers allows for bits to be dropped from the offset and hence widen the memory region that can be covered. This controls how many bits are left shifted when pointers are recovered from the offsets.

Definition at line 190 of file rte_ptr_compress.h.

rte_ptr_compress_16_shift()

static __rte_always_inline void rte_ptr_compress_16_shift ( void *  ptr_base, void *const *  src_table, uint16_t *  dest_table, size_t  n, uint8_t  bit_shift  )

static

Compress pointers into 16-bit offsets from base pointer.

Note

It is programmer's responsibility to ensure the resulting offsets fit into 16 bits. Alignment of the structures pointed to by the pointers allows us to drop bits from the offsets. This is controlled by the bit_shift parameter. This means that if structures are aligned by 8 bytes they must be within 256KB of the base pointer. If there is no such alignment guarantee they must be within 64KB.

Parameters

ptr_base	A pointer used to calculate offsets of pointers in src_table.
src_table	A pointer to an array of pointers.
dest_table	A pointer to an array of compressed pointers returned by this function.
n	The number of objects to compress, must be strictly positive.
bit_shift	Byte alignment of memory pointed to by the pointers allows for bits to be dropped from the offset and hence widen the memory region that can be covered. This controls how many bits are right shifted.

Definition at line 254 of file rte_ptr_compress.h.

rte_ptr_decompress_16_shift()

static __rte_always_inline void rte_ptr_decompress_16_shift ( void *  ptr_base, uint16_t const *  src_table, void **  dest_table, size_t  n, uint8_t  bit_shift  )

static

Decompress pointers from 16-bit offsets from base pointer.

Parameters

ptr_base	A pointer which was used to calculate offsets in src_table.
src_table	A pointer to an array to compressed pointers.
dest_table	A pointer to an array of decompressed pointers returned by this function.
n	The number of objects to decompress, must be strictly positive.
bit_shift	Byte alignment of memory pointed to by the pointers allows for bits to be dropped from the offset and hence widen the memory region that can be covered. This controls how many bits are left shifted when pointers are recovered from the offsets.

Definition at line 298 of file rte_ptr_compress.h.