xopto.mcbase.mcutil.buffer module

class BufferAllocation(offset: int, size: int, shape: tuple, dtype: numpy.dtype, owner: any, initializer: Optional[numpy.ndarray] = None, allocator: Optional[xopto.mcbase.mcutil.buffer.BufferAllocator] = None, download=True)[source]

Bases: object

Buffer allocation instance.

Parameters

  • offset (int) – Offset of the first element of the allocated buffer from the start of the common memory buffer (in number of elements).

  • size (int) – Size of the allocated buffer in number of items.

  • shape (tuple) – Shape of the allocated buffer.

  • dtype (np.dtype) – Numpy data type of the allocated buffer.

  • owner (any) – Owner of the allocation.

  • initializer (np.ndarray or None) – Initializer for the allocation.

  • allocator (BufferAllocator) – Buffer allocator that created this buffer.

  • download (bool) – Set to True if the data buffer should be downloaded after executing the kernel. The downloaded data will be passed to the update_data() of the owner.

property allocator: xopto.mcbase.mcutil.buffer.BufferAllocator

Allocation allocator.

property download: bool

Download required after execution.

property dtype: numpy.dtype

Numpy data type as dtype.

free()[source]

Clear the allocation.

property initializer: numpy.ndarray

Buffer initializer.

property offset: int

Offset of the buffer start.

property owner: any

Allocation owner.

property shape: tuple

Buffer shape.

property size: int

Size of the allocated buffer.

class BufferAllocator(dtype=<class 'numpy.float64'>)[source]

Bases: object

allocate(owner: any, shape: Tuple[int], download=True) → int[source]

Allocate a new data buffer for the owner. The allocations cannot be cleared individually. Each owner can allocate multiple buffers.

Parameters

  • owner (any) – Owner of the allocated buffer.

  • shape (tuple) – Shape of the data buffer.

  • download (bool) – Set to True if the data buffer should be downloaded after executing the kernel. The downloaded data will be passed to the update_data() of the owner.

Returns

allocation – Buffer allocation object.

Return type

BufferAllocation

allocations(owner: any) → List[dict][source]

Get a list of existing allocations for the owner.

Parameters

owner (any) – Allocation owner.

Returns

allocation –

List of allocation dicts with the following keys:
  • size: int

    Allocation buffer size.

  • shape: tuple

    Allocation buffer shape

  • offset: int

    Offset of the first buffer element in the common data buffer array.

  • owner: any

    Allocation owner.

The returned value is an empty list if no allocations for the given owner are found.

Return type

List[dict]

clear()[source]

Remove all allocations.

create_buffer(mc: xopto.mcbase.mcobject.McObject, out: Optional[numpy.ndarray] = None) → numpy.ndarray[source]

Create a new numpy buffer or return the existing buffer if the size and type match the requirements.

Parameters

  • mc (McObject) – Monte Carlo simulator instance.

  • out (np.ndarray) – Optional existing buffer array.

Returns

buffer – Existing buffer if passed as an input argument and matches the required size and data type, else a new buffer

Return type

np.ndarray

defragment()[source]

Defragment allocations. Note that this will change the location of buffers.

property dtype: numpy.dtype

Allocator data type.

extract(data: numpy.ndarray, owner: any) → List[numpy.ndarray][source]

Extract buffers of the owner from the full memory buffer.

Parameters

  • data (np.ndarray) – Common data buffer.

  • owner (any) – Find and return buffers that were allocated by this owner.

Returns

owner_buffers – Buffers that were allocated by the owner.

Return type

List[np.ndarray]

free(allocation: xopto.mcbase.mcutil.buffer.BufferAllocation)[source]

Free a buffer allocation. This will not change the total buffer size. The memory space will be left unused.

property size: int

Current buffer size.

class BufferAllocators[source]

Bases: object

Manages multiple general Buffer allocators of type BufferAllocator.

allocator(dtype: numpy.dtype) → xopto.mcbase.mcutil.buffer.BufferAllocator[source]

Return allocator for the given buffer data type. Allocators are created on the first demand/use.

Parameters

dtype (np.dtype) – Numpy data type of the buffer allocator.

Returns

allocator – Buffer allocator.

Return type

ContiguousNumpyAllocator

Note

Allocators can be retrieved by the [] operator that takes the numpy data type of the allocator.

class NumpyAllocator(dtype, size=1000000)[source]

Bases: object

Manages temporary allocations of contiguous numpy-based buffers. The initial size of the buffer grows as required by the allocations. The internal buffer is never released.

Parameters

  • dtype (np.dtype) – Data type of the allocator.

  • size (int) – Initial minimum size of the internal buffer created on the first allocation.

allocate(size: int) → numpy.ndarray[source]

Allocates a buffer with the given size.

Parameters

size (int) – Size of the allocated buffer in number of buffer items (NOT bytes).

Returns

buffer – Allocated numpy buffer

Return type

np.ndarray

available() → int[source]

Returns the size available for allocation before a new internal buffer allocation is required.

clear()[source]

Release all allocations made by allocate().

Note

The internal numpy buffer is not released, only the internal buffer manager will clear all allocations.

property size: int

Currently allocated size.

class NumpyAllocators(size: int = 0, dtypes: Optional[List[numpy.dtype]] = None)[source]

Bases: object

Manages temporary allocations of contiguous numpy-based buffers. The initial size of the internal numpy buffers grows as required by the allocations but is never released (reduced in size).

If a list of data types (dtypes) is passed to the constructor, only buffers of the listed types can be allocated (restricted allocator).

Parameters

  • size (int) – Initial minimum size of the internal buffers created on the first allocation.

  • dtype (np.dtype) – Data types of the allocator. If None, any data type is allowed.

allocate(dtype: numpy.dtype, size: int) → numpy.ndarray[source]

Allocates a buffer of the given data type and size. Raises TypeError if allocating a data type that is not allowed.

Parameters

  • dtype (np.dtype) – Data type of the buffer.

  • size (int) – Size of the allocated buffer in number of buffer items (NOT bytes).

Returns

buffer – Allocated numpy buffer

Return type

np.ndarray

available(dtype=<class 'numpy.dtype'>) → int[source]

Returns the size available for allocation before a new internal buffer allocation is required for the given data type.

Parameters

dtype (np.dtype) – Data type of the allocations.

Returns

size – Total buffer size available before a new internal buffer allocation is required.

Return type

int

release(dtype: Optional[numpy.dtype] = None)[source]

Release all allocations made by allocate().

Parameters

dtype (np.dtype) – If not None, only the allocations of the specified data type will be cleared. If None, all allocations are cleared.

Note

The internal numpy buffers are not released, only the internal buffer manager will clear all allocations.

size(dtype: numpy.dtype) → int[source]

Total size of all allocations of the given data type.

Parameters

dtype (np.dtype) – Data type of the allocations.

Returns

size – Total size of the allocations for the given data type.

Return type

int

class RestrictedBufferAllocators(dtypes: List[numpy.dtype])[source]

Bases: xopto.mcbase.mcutil.buffer.BufferAllocators

Manages multiple general Buffer allocators of type BufferAllocator. The buffer types are restricted to a list of types passed to the constructor.

allocator(dtype: numpy.dtype) → xopto.mcbase.mcutil.buffer.BufferAllocator[source]

Return allocator for the given buffer data type. Allocators are created on the first demand/use.

Parameters

dtype (np.dtype) – Numpy data type of the buffer allocator.

Returns

allocator – Buffer allocator.

Return type

ContiguousNumpyAllocator

Note

Allocators can be retrieved by the [] operator that takes the numpy data type of the allocator.