Miscellaneous

Version number

const char *mts_version(void)

Get the runtime version of the metatensor library as a string.

This version follows the <major>.<minor>.<patch>[-<dev>] format.

METATENSOR_VERSION

Macro containing the compile-time version of metatensor, as a string

METATENSOR_VERSION_MAJOR

Macro containing the compile-time major version number of metatensor, as an integer

METATENSOR_VERSION_MINOR

Macro containing the compile-time minor version number of metatensor, as an integer

METATENSOR_VERSION_PATCH

Macro containing the compile-time patch version number of metatensor, as an integer

Error handling

const char *mts_last_error(void)

Get the last error message that was created on the current thread.

Returns:

the last error message, as a NULL-terminated string

void mts_disable_panic_printing(void)

Disable printing of the message to stderr when some Rust code reach a panic.

All panics from Rust code are caught anyway and translated to an error status code, and the message is stored and accessible through mts_last_error. To print the error message and Rust backtrace anyway, users can set the RUST_BACKTRACE environment variable to 1.

typedef int32_t mts_status_t

Status type returned by all functions in the C API.

The value 0 (MTS_SUCCESS) is used to indicate successful operations, positive values are used by this library to indicate errors, while negative values are reserved for users of this library to indicate their own errors in callbacks.

MTS_SUCCESS

Status code used when a function succeeded

MTS_INVALID_PARAMETER_ERROR

Status code used when a function got an invalid parameter

MTS_BUFFER_SIZE_ERROR

Status code used when a memory buffer is too small to fit the requested data

MTS_INTERNAL_ERROR

Status code used when there was an internal error, i.e. there is a bug inside metatensor itself

Serialization

Tensors

struct mts_tensormap_t *mts_tensormap_load(const char *path, mts_create_array_callback_t create_array)

Load a tensor map from the file at the given path.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_tensormap_free.

TensorMap are serialized using numpy’s .npz format, i.e. a ZIP file without compression (storage method is STORED), where each file is stored as a .npy array. Both the ZIP and NPY format are well documented:

We add other restriction on top of these formats when saving/loading data. First, Labels instances are saved as structured array, see the labels module for more information. Only 32-bit integers are supported for Labels, and only 64-bit floats are supported for data (values and gradients).

Second, the path of the files in the archive also carry meaning. The keys of the TensorMap are stored in /keys.npy, and then different blocks are stored as

/  blocks / <block_id>  / values / samples.npy
                        / values / components  / 0.npy
                                               / <...>.npy
                                               / <n_components>.npy
                        / values / properties.npy
                        / values / data.npy

                        # optional sections for gradients, one by parameter
                        /   gradients / <parameter> / samples.npy
                                                    /   components  / 0.npy
                                                                    / <...>.npy
                                                                    / <n_components>.npy
                                                    /   data.npy
Parameters:
  • path – path to the file as a NULL-terminated UTF-8 string

  • create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor map, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_tensormap_save(const char *path, const struct mts_tensormap_t *tensor)

Save a tensor map to the file at the given path.

If the file already exists, it is overwritten.

Parameters:
  • path – path to the file as a NULL-terminated UTF-8 string

  • tensor – tensor map to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

struct mts_tensormap_t *mts_tensormap_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, mts_create_array_callback_t create_array)

Load a tensor map from the given in-memory buffer.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_tensormap_free.

Parameters:
  • buffer – buffer containing a previously serialized mts_tensormap_t

  • buffer_count – number of elements in the buffer

  • create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor map, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_tensormap_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_tensormap_t *tensor)

Save a tensor map to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation.

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:
  • buffer – pointer to the buffer the tensor will be stored to, which can change due to reallocations.

  • buffer_count – pointer to the buffer size on input, number of written bytes on output

  • realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.

  • realloc – function that allows to grow the buffer allocation

  • tensor – tensor map that will saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

typedef mts_status_t (*mts_create_array_callback_t)(const uintptr_t *shape, uintptr_t shape_count, struct mts_array_t *array)

Function pointer to create a new mts_array_t when de-serializing tensor maps.

This function gets the shape of the array (the shape contains shape_count elements) and should fill array with a new valid mts_array_t or return non-zero mts_status_t.

The newly created array should contains 64-bit floating points (double) data, and live on CPU, since metatensor will use mts_array_t.data to get the data pointer and write to it.

typedef uint8_t *(*mts_realloc_buffer_t)(void *user_data, uint8_t *ptr, uintptr_t new_size)

Function pointer to grow in-memory buffers for mts_tensormap_save_buffer and mts_labels_save_buffer.

This function takes an existing pointer in ptr and a new length in new_size, and grow the allocation. If the pointer is NULL, it should create a new allocation. If it is unable to allocate memory, it should return a NULL pointer. This follows the API of the standard C function realloc, with an additional parameter user_data that can be used to hold custom data.

Blocks

struct mts_block_t *mts_block_load(const char *path, mts_create_array_callback_t create_array)

Load a tensor block from the file at the given path.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_block_free.

See mts_tensormap_load for more information about the format used to serialize the data.

Parameters:
  • path – path to the file as a NULL-terminated UTF-8 string

  • create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated block, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_block_save(const char *path, const struct mts_block_t *block)

Save a tensor block to the file at the given path.

If the file already exists, it is overwritten.

Parameters:
  • path – path to the file as a NULL-terminated UTF-8 string

  • block – tensor block to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

struct mts_block_t *mts_block_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, mts_create_array_callback_t create_array)

Load a tensor block from the given in-memory buffer.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_block_free.

Parameters:
  • buffer – buffer containing a previously serialized mts_block_t

  • buffer_count – number of elements in the buffer

  • create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor block, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_block_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_block_t *block)

Save a tensor block to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation.

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:
  • buffer – pointer to the buffer the block will be stored to, which can change due to reallocations.

  • buffer_count – pointer to the buffer size on input, number of written bytes on output

  • realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.

  • realloc – function that allows to grow the buffer allocation

  • block – tensor block that will saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

Labels

mts_status_t mts_labels_load(const char *path, struct mts_labels_t *labels)

Load labels from the file at the given path.

Labels are stored using numpy’s NPY format, so the file will typically use the .npy extension.

This function allocates memory which must be released mts_labels_free when you don’t need it anymore.

Parameters:
  • path – path to the file as a NULL-terminated UTF-8 string

  • labels – pointer to empty Labels

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_labels_save(const char *path, struct mts_labels_t labels)

Save labels to the file at the given path.

If the file already exists, it is overwritten.

Parameters:
  • path – path to the file as a NULL-terminated UTF-8 string

  • labels – Labels to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_labels_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, struct mts_labels_t *labels)

Load labels from the given in-memory buffer.

This function allocates memory which must be released mts_labels_free when you don’t need it anymore.

Parameters:
  • buffer – buffer containing a previously serialized mts_labels_t

  • buffer_count – number of elements in the buffer

  • labels – pointer to empty Labels

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_labels_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, struct mts_labels_t labels)

Save labels to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation (0 if *buffer is NULL).

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:
  • buffer – pointer to the buffer the tensor will be stored to, which can change due to reallocations.

  • buffer_count – pointer to the buffer size on input, number of written bytes on output

  • realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.

  • realloc – function that allows to grow the buffer allocation

  • labels – Labels that will saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.