Miscellaneous¶

Version number¶

const char *mts_version(void)¶

Get the runtime version of the metatensor library as a string.

This version follows the <major>.<minor>.<patch>[-<dev>] format.

METATENSOR_VERSION¶: Macro containing the compile-time version of metatensor, as a string

METATENSOR_VERSION_MAJOR¶: Macro containing the compile-time major version number of metatensor, as an integer

METATENSOR_VERSION_MINOR¶: Macro containing the compile-time minor version number of metatensor, as an integer

METATENSOR_VERSION_PATCH¶: Macro containing the compile-time patch version number of metatensor, as an integer

Error handling¶

mts_status_t mts_last_error(const char **message, const char **origin, void **data)¶

Get the last error message that was created on the current thread.

Parameters:

message – if not NULL, this will be set to the last error message, as a NULL-terminated string
origin – if not NULL, this will be set to the origin of the last error, as a NULL-terminated string
data – if not NULL, this will be set to the custom data of the last error

Returns:

The status code of this operation.

mts_status_t mts_set_last_error(const char *message, const char *origin, void *data, void (*data_deleter)(void*))¶

Set the last error message for the current thread.

This is useful when the error is created in a callback provided by the user of metatensor.

Parameters:

message – the error message to set, as a NULL-terminated string
origin – the origin of the error, as a NULL-terminated string. This can be used to check if the data field was defined by the same code that is calling mts_last_error when retrieving custom data.
data – an arbitrary pointer that can be retrieved later with mts_last_error.
data_deleter – deleter for the custom data, this function will be called with data as argument when the last error is replaced by another one.

void mts_disable_panic_printing(void)¶

Disable printing of the message to stderr when some Rust code reaches a panic.

All panics from Rust code are caught anyway and translated to an error status code, and the message is stored and accessible through mts_last_error. To print the error message and Rust backtrace anyway, users can set the RUST_BACKTRACE environment variable to 1.

enum mts_status_t¶

Status type returned by all functions in the C API.

The value 0 (MTS_SUCCESS) is used to indicate successful operations.

Values:

enumerator MTS_SUCCESS¶: Status code used when a function succeeded

enumerator MTS_INVALID_PARAMETER_ERROR¶: Status code used when a function got an invalid parameter

enumerator MTS_IO_ERROR¶: Status code indicating I/O error when loading/writing mts_tensormap_t to a file

enumerator MTS_SERIALIZATION_ERROR¶: Status code indicating errors in the serialization format when loading/writing mts_tensormap_t to a file

enumerator MTS_BUFFER_SIZE_ERROR¶: Status code used when a memory buffer is too small to fit the requested data

enumerator MTS_CALLBACK_ERROR¶: Status code indicating errors that come from callbacks provided by the user of metatensor. The error message and arbitrary custom data can be stored using mts_set_last_error inside the callback, and retrieved later with mts_last_error.

enumerator MTS_INTERNAL_ERROR¶: Status code used when there was an internal error, i.e. there is a bug inside metatensor itself

Serialization¶

Tensors¶

mts_tensormap_save(): serialize and save a mts_tensormap_t to a file
mts_tensormap_load(): load a serialized mts_tensormap_t from a file
mts_tensormap_save_buffer(): serialize and save a mts_tensormap_t to a in-memory buffer
mts_tensormap_load_buffer(): load a serialized mts_tensormap_t from an in-memory buffer

struct mts_tensormap_t *mts_tensormap_load(const char *path, mts_create_array_callback_t create_array)¶

Load a tensor map from the file at the given path.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_tensormap_free.

TensorMap are serialized using numpy’s NPZ format, i.e. a ZIP file without compression (storage method is STORED), where each file is stored as a .npy array. Both the ZIP and NPY format are well documented:

We add other restriction on top of these formats when saving/loading data. First, Labels instances are saved as structured array, see the labels module for more information.

Second, the path of the files in the archive also carry meaning. The keys of the TensorMap are stored in /keys.npy, and then different blocks are stored as

/  blocks / <block_id>  / values / samples.npy
                        / values / components  / 0.npy
                                               / <...>.npy
                                               / <n_components>.npy
                        / values / properties.npy
                        / values / data.npy

                        # optional sections for gradients, one by parameter
                        /   gradients / <parameter> / samples.npy
                                                    /   components  / 0.npy
                                                                    / <...>.npy
                                                                    / <n_components>.npy
                                                    /   data.npy

Finally, the info (i.e. global metadata) of the TensorMap is stored inside the info.json file, unless empty.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor map, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_tensormap_save(const char *path, const struct mts_tensormap_t *tensor)¶

Save a tensor map to the file at the given path.

If the file already exists, it is overwritten. The recommended file extension when saving data is .mts, to prevent confusion with generic .npz files.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
tensor – tensor map to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

struct mts_tensormap_t *mts_tensormap_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, mts_create_array_callback_t create_array)¶

Load a tensor map from the given in-memory buffer.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_tensormap_free.

Parameters:

buffer – buffer containing a previously serialized mts_tensormap_t
buffer_count – number of elements in the buffer
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor map, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_tensormap_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_tensormap_t *tensor)¶

Save a tensor map to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation.

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:

buffer – pointer to the buffer the tensor will be stored to, which can change due to reallocations.
buffer_count – pointer to the buffer size on input, number of written bytes on output
realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.
realloc – function that allows to grow the buffer allocation
tensor – tensor map that will be saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

typedef mts_status_t (*mts_create_array_callback_t)(const uintptr_t *shape, uintptr_t shape_count, DLDataType dtype, struct mts_array_t *array)¶

Function pointer to create a new mts_array_t when de-serializing tensor maps.

This function gets the shape of the array (the shape contains shape_count elements) and the dtype (a DLDataType describing the element type), and should fill array with a new valid mts_array_t or return non-zero mts_status_t.

The newly created array should live on CPU, since metatensor will use mts_array_t.as_dlpack to get the data pointer and write to it.

This function should return MTS_SUCCESS on success, or MTS_CALLBACK_ERROR on failure. In case of failure, the implementation should call mts_set_last_error with an appropriate error message before returning.

typedef uint8_t *(*mts_realloc_buffer_t)(void *user_data, uint8_t *ptr, uintptr_t new_size)¶

Function pointer to grow in-memory buffers for mts_tensormap_save_buffer and mts_labels_save_buffer.

This function takes an existing pointer in ptr and a new length in new_size, and grow the allocation. If the pointer is NULL, it should create a new allocation. If it is unable to allocate memory, it should return a NULL pointer. This follows the API of the standard C function realloc, with an additional parameter user_data that can be used to hold custom data.

Blocks¶

mts_block_save(): serialize and save a mts_block_t to a file
mts_block_load(): load a serialized mts_block_t from a file
mts_block_save_buffer(): serialize and save a mts_block_t to a in-memory buffer
mts_block_load_buffer(): load a serialized mts_block_t from a in-memory buffer

struct mts_block_t *mts_block_load(const char *path, mts_create_array_callback_t create_array)¶

Load a tensor block from the file at the given path.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_block_free.

See mts_tensormap_load for more information about the format used to serialize the data.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated block, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_block_save(const char *path, const struct mts_block_t *block)¶

Save a tensor block to the file at the given path.

If the file already exists, it is overwritten. The recommended file extension when saving data is .mts, to prevent confusion with generic .npz files.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
block – tensor block to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

struct mts_block_t *mts_block_load_buffer(const uint8_t *buffer, uintptr_t buffer_count, mts_create_array_callback_t create_array)¶

Load a tensor block from the given in-memory buffer.

Arrays for the values and gradient data will be created with the given create_array callback, and filled by this function with the corresponding data.

The memory allocated by this function should be released using mts_block_free.

Parameters:

buffer – buffer containing a previously serialized mts_block_t
buffer_count – number of elements in the buffer
create_array – callback function that will be used to create data arrays inside each block

Returns:

A pointer to the newly allocated tensor block, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_block_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_block_t *block)¶

Save a tensor block to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation.

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:

buffer – pointer to the buffer the block will be stored to, which can change due to reallocations.
buffer_count – pointer to the buffer size on input, number of written bytes on output
realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.
realloc – function that allows to grow the buffer allocation
block – tensor block that will be saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

Labels¶

mts_labels_save(): serialize and save a mts_labels_t to a file
mts_labels_load(): load a serialized mts_labels_t from a file
mts_labels_save_buffer(): serialize and save a mts_labels_t to an in-memory buffer
mts_labels_load_buffer(): load serialized mts_labels_t from an in-memory buffer
mts_tensormap_load(): create the Rust-side data for the labels

const struct mts_labels_t *mts_labels_load(const char *path)¶

Load labels from the file at the given path.

This function allocates memory which must be released with mts_labels_free when you don’t need it anymore.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string

Returns:

A pointer to the newly allocated labels, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_labels_save(const char *path, const struct mts_labels_t *labels)¶

Save labels to the file at the given path.

If the file already exists, it is overwritten. The recommended file extension when saving data is .mts, to prevent confusion with generic .npz files.

Parameters:

path – path to the file as a NULL-terminated UTF-8 string
labels – pointer to labels to save to the file

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.

const struct mts_labels_t *mts_labels_load_buffer(const uint8_t *buffer, uintptr_t buffer_count)¶

Load labels from the given in-memory buffer.

This function allocates memory which must be released with mts_labels_free when you don’t need it anymore.

Parameters:

buffer – buffer containing a previously serialized mts_labels_t
buffer_count – number of elements in the buffer

Returns:

A pointer to the newly allocated labels, or a NULL pointer in case of error. In case of error, you can use mts_last_error() to get the error message.

mts_status_t mts_labels_save_buffer(uint8_t **buffer, uintptr_t *buffer_count, void *realloc_user_data, mts_realloc_buffer_t realloc, const struct mts_labels_t *labels)¶

Save labels to an in-memory buffer.

On input, *buffer should contain the address of a starting buffer (which can be NULL) and *buffer_count should contain the size of the allocation (0 if *buffer is NULL).

On output, *buffer will contain the serialized data, and *buffer_count the total number of written bytes (which might be less than the allocation size).

Users of this function are responsible for freeing the *buffer when they are done with it, using the function matching the realloc callback.

Parameters:

buffer – pointer to the buffer the tensor will be stored to, which can change due to reallocations.
buffer_count – pointer to the buffer size on input, number of written bytes on output
realloc_user_data – custom data for the realloc callback. This will be passed as the first argument to realloc as-is.
realloc – function that allows to grow the buffer allocation
labels – pointer to labels that will be saved to the buffer

Returns:

The status code of this operation. If the status is not MTS_SUCCESS, you can use mts_last_error() to get the full error message.