Miscellaneous¶

Version number¶

const char *mts_version(void)¶

Get the runtime version of the metatensor library as a string.

This version follows the <major>.<minor>.<patch>[-<dev>] format.

METATENSOR_VERSION¶: Macro containing the compile-time version of metatensor, as a string

METATENSOR_VERSION_MAJOR¶: Macro containing the compile-time major version number of metatensor, as an integer

METATENSOR_VERSION_MINOR¶: Macro containing the compile-time minor version number of metatensor, as an integer

METATENSOR_VERSION_PATCH¶: Macro containing the compile-time patch version number of metatensor, as an integer

Error handling¶

const char *mts_last_error(void)¶

Get the last error message that was created on the current thread.

Returns:: the last error message, as a NULL-terminated string

void mts_disable_panic_printing(void)¶

Disable printing of the message to stderr when some Rust code reach a panic.

All panics from Rust code are caught anyway and translated to an error status code, and the message is stored and accessible through mts_last_error. To print the error message and Rust backtrace anyway, users can set the RUST_BACKTRACE environment variable to 1.

typedef int32_t mts_status_t¶

Status type returned by all functions in the C API.

The value 0 (MTS_SUCCESS) is used to indicate successful operations, positive values are used by this library to indicate errors, while negative values are reserved for users of this library to indicate their own errors in callbacks.

MTS_SUCCESS¶: Status code used when a function succeeded

MTS_INVALID_PARAMETER_ERROR¶: Status code used when a function got an invalid parameter

MTS_BUFFER_SIZE_ERROR¶: Status code used when a memory buffer is too small to fit the requested data

MTS_INTERNAL_ERROR¶: Status code used when there was an internal error, i.e. there is a bug inside metatensor itself

Serialization¶

Tensors¶

mts_tensormap_save(): serialize and save a mts_tensormap_t to a file
mts_tensormap_load(): load serialized mts_tensormap_t from a file
mts_tensormap_save_buffer(): serialize and save a mts_tensormap_t to a in-memory buffer
mts_tensormap_load_buffer(): load serialized mts_tensormap_t from a in-memory buffer

struct mts_tensormap_t *mts_tensormap_load(const char *path, mts_create_array_callback_t create_array)¶

Load a tensor map from the file at the given path.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_tensormap_free.

TensorMap are serialized using numpy’s .npz format, i.e. a ZIP file without compression (storage method is STORED), where each file is stored as a .npy array. Both the ZIP and NPY format are well documented:

We add other restriction on top of these formats when saving/loading data. First, Labels instances are saved as structured array, see the labels module for more information. Only 32-bit integers are supported for Labels, and only 64-bit floats are supported for data (values and gradients).

Second, the path of the files in the archive also carry meaning. The keys of the TensorMap are stored in /keys.npy, and then different blocks are stored as

/  blocks / <block_id>  / values / samples.npy
                        / values / components  / 0.npy
                                               / <...>.npy
                                               / <n_components>.npy
                        / values / properties.npy
                        / values / data.npy

                        # optional sections for gradients, one by parameter
                        /   gradients / <parameter> / samples.npy
                                                    /   components  / 0.npy
                                                                    / <...>.npy
                                                                    / <n_components>.npy
                                                    /   data.npy

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor map, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_tensormap_save(const char *path, const struct mts_tensormap_t *tensor)¶

Save a tensor map to the file at the given path.

If the file already exists, it is overwritten.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
tensor – tensor map to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

struct mts_tensormap_t *mts_tensormap_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, mts_create_array_callback_t create_array)¶

Load a tensor map from the given in-memory buffer.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_tensormap_free.

Parameters:

buffer – buffer containing a previously serialized mts_tensormap_t
buffer_count – number of elements in the buffer
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor map, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_tensormap_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_tensormap_t *tensor)¶

Save a tensor map to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation.

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:

buffer – pointer to the buffer the tensor will be stored to, which can change due to reallocations.
buffer_count – pointer to the buffer size on input, number of written bytes on output
realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.
realloc – function that allows to grow the buffer allocation
tensor – tensor map that will saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

typedef mts_status_t (*mts_create_array_callback_t)(const uintptr_t *shape, uintptr_t shape_count, struct mts_array_t *array)¶

Function pointer to create a new mts_array_t when de-serializing tensor maps.

This function gets the shape of the array (the shape contains shape_count elements) and should fill array with a new valid mts_array_t or return non-zero mts_status_t.

The newly created array should contains 64-bit floating points (double) data, and live on CPU, since metatensor will use mts_array_t.data to get the data pointer and write to it.

typedef uint8_t *(*mts_realloc_buffer_t)(void *user_data, uint8_t *ptr, uintptr_t new_size)¶

Function pointer to grow in-memory buffers for mts_tensormap_save_buffer and mts_labels_save_buffer.

This function takes an existing pointer in ptr and a new length in new_size, and grow the allocation. If the pointer is NULL, it should create a new allocation. If it is unable to allocate memory, it should return a NULL pointer. This follows the API of the standard C function realloc, with an additional parameter user_data that can be used to hold custom data.

Blocks¶

mts_block_save(): serialize and save a mts_block_t to a file
mts_block_load(): load serialized mts_block_t from a file
mts_block_save_buffer(): serialize and save a mts_block_t to a in-memory buffer
mts_block_load_buffer(): load serialized mts_block_t from a in-memory buffer

struct mts_block_t *mts_block_load(const char *path, mts_create_array_callback_t create_array)¶

Load a tensor block from the file at the given path.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_block_free.

See mts_tensormap_load for more information about the format used to serialize the data.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated block, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_block_save(const char *path, const struct mts_block_t *block)¶

Save a tensor block to the file at the given path.

If the file already exists, it is overwritten.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
block – tensor block to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

struct mts_block_t *mts_block_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, mts_create_array_callback_t create_array)¶

Load a tensor block from the given in-memory buffer.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_block_free.

Parameters:

buffer – buffer containing a previously serialized mts_block_t
buffer_count – number of elements in the buffer
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor block, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_block_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_block_t *block)¶

Save a tensor block to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation.

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:

buffer – pointer to the buffer the block will be stored to, which can change due to reallocations.
buffer_count – pointer to the buffer size on input, number of written bytes on output
realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.
realloc – function that allows to grow the buffer allocation
block – tensor block that will saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

Labels¶

mts_labels_save(): serialize and save a mts_labels_t to a file
mts_labels_load(): load serialized mts_labels_t from a file
mts_labels_save_buffer(): serialize and save a mts_labels_t to a in-memory buffer
mts_labels_load_buffer(): load serialized mts_labels_t from a in-memory buffer
mts_tensormap_load(): create the Rust-side data for the labels

mts_status_t mts_labels_load(const char *path, struct mts_labels_t *labels)¶

Load labels from the file at the given path.

Labels are stored using numpy’s NPY format, so the file will typically use the .npy extension.

This function allocates memory which must be released mts_labels_free when you don’t need it anymore.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
labels – pointer to empty Labels

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_labels_save(const char *path, struct mts_labels_t labels)¶

Save labels to the file at the given path.

If the file already exists, it is overwritten.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
labels – Labels to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_labels_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, struct mts_labels_t *labels)¶

Load labels from the given in-memory buffer.

This function allocates memory which must be released mts_labels_free when you don’t need it anymore.

Parameters:

buffer – buffer containing a previously serialized mts_labels_t
buffer_count – number of elements in the buffer
labels – pointer to empty Labels

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

mts_status_t mts_labels_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, struct mts_labels_t labels)¶

Save labels to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation (0 if *buffer is NULL).

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:

buffer – pointer to the buffer the tensor will be stored to, which can change due to reallocations.
buffer_count – pointer to the buffer size on input, number of written bytes on output
realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.
realloc – function that allows to grow the buffer allocation
labels – Labels that will saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.