Data arrays

struct mts_array_t

mts_array_t manages n-dimensional arrays used as data in a block or tensor map. The array itself if opaque to this library and can come from multiple sources: Rust program, a C/C++ program, a Fortran program, Python with numpy or torch. The data does not have to live on CPU, or even on the same machine where this code is executed.

This struct contains a C-compatible manual implementation of a virtual table (vtable, i.e. trait in Rust, pure virtual class in C++); allowing manipulation of the array in an opaque way.

WARNING: all function implementations MUST be thread-safe, and can be called from multiple threads at the same time. The mts_array_t itself might be moved from one thread to another.

Public Members

void *ptr

User-provided data should be stored here, it will be passed as the first parameter to all function pointers below.

mts_status_t (*origin)(const void *array, mts_data_origin_t *origin)

This function needs to store the “data origin” for this array in origin. Users of mts_array_t should register a single data origin with mts_register_data_origin, and use it for all compatible arrays.

mts_status_t (*data)(void *array, double **data)

Get a pointer to the underlying data storage.

This function is allowed to fail if the data is not accessible in RAM, not stored as 64-bit floating point values, or not stored as a C-contiguous array.

mts_status_t (*shape)(const void *array, const uintptr_t **shape, uintptr_t *shape_count)

Get the shape of the array managed by this mts_array_t in the *shape pointer, and the number of dimension (size of the *shape array) in *shape_count.

mts_status_t (*reshape)(void *array, const uintptr_t *shape, uintptr_t shape_count)

Change the shape of the array managed by this mts_array_t to the given shape. shape_count must contain the number of elements in the shape array

mts_status_t (*swap_axes)(void *array, uintptr_t axis_1, uintptr_t axis_2)

Swap the axes axis_1 and axis_2 in this array.

mts_status_t (*create)(const void *array, const uintptr_t *shape, uintptr_t shape_count, struct mts_array_t *new_array)

Create a new array with the same options as the current one (data type, data location, etc.) and the requested shape; and store it in new_array. The number of elements in the shape array should be given in shape_count.

The new array should be filled with zeros.

mts_status_t (*copy)(const void *array, struct mts_array_t *new_array)

Make a copy of this array and return the new array in new_array.

The new array is expected to have the same data origin and parameters (data type, data location, etc.)

void (*destroy)(void *array)

Remove this array and free the associated memory. This function can be set to NULL is there is no memory management to do.

mts_status_t (*move_samples_from)(void *output, const void *input, const struct mts_sample_mapping_t *samples, uintptr_t samples_count, uintptr_t property_start, uintptr_t property_end)

Set entries in the output array (the current array) taking data from the input array. The output array is guaranteed to be created by calling mts_array_t::create with one of the arrays in the same block or tensor map as the input.

The samples array of size samples_count indicate where the data should be moved from input to output.

This function should copy data from input[samples[i].input, ..., :] to array[samples[i].output, ..., property_start:property_end] for i up to samples_count. All indexes are 0-based.

mts_status_t mts_register_data_origin(const char *name, mts_data_origin_t *origin)

Register a new data origin with the given name. Calling this function multiple times with the same name will give the same mts_data_origin_t.

Parameters:

  • name – name of the data origin as an UTF-8 encoded NULL-terminated string

  • origin – pointer to an mts_data_origin_t where the origin will be stored

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_get_data_origin(mts_data_origin_t origin, char *buffer, uintptr_t buffer_size)

Get the name used to register a given data origin in the given buffer

Parameters:

  • origin – pre-registered data origin

  • buffer – buffer to be filled with the data origin name. The origin name will be written as an UTF-8 encoded, NULL-terminated string

  • buffer_size – size of the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.