Neural Network¶
Modules¶
- class metatensor.learn.nn.ModuleMap(in_keys: Labels, modules: List[Module], out_properties: List[Labels] | None = None)[source]¶
A class that imitates
torch.nn.ModuleDict. In its forward function the module at position i given on construction by :param modules: is applied to the tensor block that corresponding to the`i`th key in :param in_keys:.- Parameters:
in_keys (Labels) – A
metatensor.Labelsobject with the keys of the module map that are assumed to be in the input tensor map in theforward()function.modules (List[Module]) – A sequence of modules applied in the
forward()function on the inputTensorMap. Each module corresponds to oneLabelsEntryin :param in_keys: that determines on whichTensorBlockthe module is applied on. :param modules: and :param in_keys: must match in length.out_properties (List[Labels] | None) –
A list of labels that is used to determine the properties labels of the output. Because a module could change the number of properties, the labels of the properties cannot be persevered. By default the output properties are relabeled using Labels.range with “_” as key.
>>> import torch >>> import numpy as np >>> from copy import deepcopy >>> from metatensor import Labels, TensorBlock, TensorMap >>> from metatensor.learn.nn import ModuleMap
Create simple block
>>> block_1 = TensorBlock( ... values=torch.tensor( ... [ ... [1.0, 2.0, 4.0], ... [3.0, 5.0, 6.0], ... ] ... ), ... samples=Labels( ... ["system", "atom"], ... np.array( ... [ ... [0, 0], ... [0, 1], ... ] ... ), ... ), ... components=[], ... properties=Labels(["properties"], np.array([[0], [1], [2]])), ... ) >>> block_2 = TensorBlock( ... values=torch.tensor( ... [ ... [5.0, 8.0, 2.0], ... [1.0, 2.0, 8.0], ... ] ... ), ... samples=Labels( ... ["system", "atom"], ... np.array( ... [ ... [0, 0], ... [0, 1], ... ] ... ), ... ), ... components=[], ... properties=Labels(["properties"], np.array([[3], [4], [5]])), ... ) >>> keys = Labels(names=["key"], values=np.array([[0], [1]])) >>> tensor = TensorMap(keys, [block_1, block_2])
Create modules
>>> linear = torch.nn.Linear(3, 1, bias=False) >>> with torch.no_grad(): ... _ = linear.weight.copy_(torch.tensor([1.0, 1.0, 1.0])) >>> modules = [linear, deepcopy(linear)] >>> # you could also extend the module by some nonlinear activation function
Create ModuleMap from this ModucDict and apply it
>>> module_map = ModuleMap(tensor.keys, modules) >>> out = module_map(tensor) >>> out TensorMap with 2 blocks keys: key 0 1 >>> out[0].values tensor([[ 7.], [14.]], grad_fn=<MmBackward0>) >>> out[1].values tensor([[15.], [11.]], grad_fn=<MmBackward0>)
Let’s look at the metadata
>>> tensor[0] TensorBlock samples (2): ['system', 'atom'] components (): [] properties (3): ['properties'] gradients: None >>> out[0] TensorBlock samples (2): ['system', 'atom'] components (): [] properties (1): ['_'] gradients: None
It got completely lost because we cannot know in general what the output is. You can add in the initialization of the ModuleMap a TensorMap that contains the intended output Labels.
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- classmethod from_module(in_keys: Labels, module: Module, out_properties: List[Labels] | None = None)[source]¶
A wrapper around one
torch.nn.Moduleapplying the same type of module on each tensor block.- Parameters:
in_keys (Labels) – A
metatensor.Labelsobject that determines the keys of the module map that are ass the TensorMaps that are assumed to be in the input tensor map in theforward()function.module (Module) – The module that is applied on each block.
out_properties (List[Labels] | None) – A list of labels that is used to determine the properties labels of the output. Because a module could change the number of properties, the labels of the properties cannot be persevered. By default the output properties are relabeled using Labels.range with “_” as key.
>>> import torch >>> import numpy as np >>> from metatensor import Labels, TensorBlock, TensorMap >>> block_1 = TensorBlock( ... values=torch.tensor( ... [ ... [1.0, 2.0, 4.0], ... [3.0, 5.0, 6.0], ... ] ... ), ... samples=Labels( ... ["system", "atom"], ... np.array( ... [ ... [0, 0], ... [0, 1], ... ] ... ), ... ), ... components=[], ... properties=Labels(["properties"], np.array([[0], [1], [2]])), ... ) >>> block_2 = TensorBlock( ... values=torch.tensor( ... [ ... [5.0, 8.0, 2.0], ... [1.0, 2.0, 8.0], ... ] ... ), ... samples=Labels( ... ["system", "atom"], ... np.array( ... [ ... [0, 0], ... [0, 1], ... ] ... ), ... ), ... components=[], ... properties=Labels(["properties"], np.array([[0], [1], [2]])), ... ) >>> keys = Labels(names=["key"], values=np.array([[0], [1]])) >>> tensor = TensorMap(keys, [block_1, block_2]) >>> linear = torch.nn.Linear(3, 1, bias=False) >>> with torch.no_grad(): ... _ = linear.weight.copy_(torch.tensor([1.0, 1.0, 1.0])) >>> # you could also extend the module by some nonlinear activation function >>> from metatensor.learn.nn import ModuleMap >>> module_map = ModuleMap.from_module(tensor.keys, linear) >>> out = module_map(tensor) >>> out[0].values tensor([[ 7.], [14.]], grad_fn=<MmBackward0>) >>> out[1].values tensor([[15.], [11.]], grad_fn=<MmBackward0>)
- forward(tensor: TensorMap) TensorMap[source]¶
Apply the modules on each block in
tensor.tensormust have the same set of keys as the modules used to initialize thisModuleMap.
- get_module(key: LabelsEntry)[source]¶
- Parameters:
key (LabelsEntry) – key of module which should be returned
- Return module:
returns he torch.nn.Module corresponding to the :param key:
- property in_keys: Labels¶
A list of labels that defines the initialized keys with corresponding modules of this module map.
- class metatensor.learn.nn.Sequential(in_keys: Labels, *args: List[ModuleMap])[source]¶
A sequential model that applies a list of ModuleMaps to the input in order.
- Parameters:
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.Linear(in_keys: Labels, in_features: int | List[int], out_features: List[int] | int | None = None, out_properties: List[Labels] | None = None, *, bias: bool = True, device: device | None = None, dtype: dtype | None = None)[source]¶
Module similar to
torch.nn.Linearthat works withmetatensor.torch.TensorMap.Applies a linear transformation to each block of a
TensorMappassed to its forward method, indexed by :param in_keys:.Refer to the :py:class`torch.nn.Linear` documentation for a more detailed description of the other parameters.
Each parameter is passed as a single value of its expected type, which is used as the parameter for all blocks.
- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.in_features (int | List[int]) –
intorlistofint, the number of input features for each block. If passed as a single value, the same feature size is taken for all blocks.out_features (List[int] | int | None) –
intorlintofint, the number of output features for each block. If passed as a single value, the same feature size is taken for all blocks.out_properties (List[Labels] | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default the output properties are relabeled using Labels.range. If provided, :param out_features: can be inferred and need not be provided.
bias (bool)
device (device | None)
dtype (dtype | None)
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.EquivariantLinear(in_keys: Labels, in_features: int | List[int], out_features: List[int] | int | None = None, out_properties: List[Labels] | None = None, invariant_keys: Labels | None = None, *, bias: bool = True, device: device | None = None, dtype: dtype | None = None)[source]¶
Module similar to
torch.nn.Linearthat works with equivariantmetatensor.torch.TensorMapobjects.Applies a linear transformation to each block of a
TensorMappassed to its forward method, indexed by :param in_keys:.Refer to the :py:class`torch.nn.Linear` documentation for a more detailed description of the other parameters.
For
EquivariantLinear, by contrast toLinear, the parameter :param bias: is only applied to modules corresponding to invariant blocks, i.e. keys in :param in_keys: that correspond to the selection in :param invariant_keys:.- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.in_features (int | List[int]) –
intorlistofint, the number of input features for each block. If passed as a single value, the same feature size is taken for all blocks.out_features (List[int] | int | None) –
intorlintofint, the number of output features for each block. If passed as a single value, the same feature size is taken for all blocks.out_properties (List[Labels] | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default the output properties are relabeled using Labels.range. If provided, :param out_features: can be inferred and need not be provided.
invariant_keys (Labels | None) – a
Labelsobject that is used to select the invariant keys fromin_keys. If not provided, the invariant keys are assumed to be those where key dimensions["o3_lambda", "o3_sigma"]are equal to[0, 1].bias (bool)
device (device | None)
dtype (dtype | None)
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.Tanh(in_keys: Labels, out_properties: Labels | None = None)[source]¶
Module similar to
torch.nn.Tanhthat works withmetatensor.torch.TensorMapobjects.Applies a hyperbolic tangent transformation to each block of a
TensorMappassed to its forward method, indexed by :param in_keys:.Refer to the :py:class`torch.nn.Tanh` documentation for a more detailed description of the parameters.
- Parameters:
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.InvariantTanh(in_keys: Labels, out_properties: Labels | None = None, invariant_keys: Labels | None = None)[source]¶
Module similar to
torch.nn.Tanhthat works withmetatensor.torch.TensorMapobjects, applying the transformation only to the invariant blocks.Applies a hyperbolic tangent transformation to each invariant block of a
TensorMappassed to itsforward()method. These are indexed by the keys in :param in_keys: that correspond to the selection passed in :param invariant_keys:.Refer to the :py:class`torch.nn.Tanh` documentation for a more detailed description of the parameters.
- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.out_properties (Labels | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default the output properties are relabeled using Labels.range.
invariant_keys (Labels | None) – a
Labelsobject that is used to select the invariant keys fromin_keys. If not provided, the invariant keys are assumed to be those where key dimensions["o3_lambda", "o3_sigma"]are equal to[0, 1].
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.ReLU(in_keys: Labels, out_properties: Labels | None = None, *, in_place: bool = False)[source]¶
Module similar to
torch.nn.ReLUthat works withmetatensor.torch.TensorMapobjects.Applies a rectified linear unit transformation transformation to each block of a
TensorMappassed to its forward method, indexed by :param in_keys:.Refer to the :py:class`torch.nn.ReLU` documentation for a more detailed description of the parameters.
- Parameters:
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.InvariantReLU(in_keys: Labels, out_properties: Labels | None = None, invariant_keys: Labels | None = None, *, in_place: bool = False)[source]¶
Module similar to
torch.nn.ReLUthat works withmetatensor.torch.TensorMapobjects, applying the transformation only to the invariant blocks.Applies a rectified linear unit transformation to each invariant block of a
TensorMappassed to itsforward()method. These are indexed by the keys in :param in_keys: that correspond to the selection passed in :param invariant_keys:.Refer to the :py:class`torch.nn.ReLU` documentation for a more detailed description of the parameters.
- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.out_properties (Labels | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default the output properties are relabeled using Labels.range.
invariant_keys (Labels | None) – a
Labelsobject that is used to select the invariant keys fromin_keys. If not provided, the invariant keys are assumed to be those where key dimensions["o3_lambda", "o3_sigma"]are equal to[0, 1].in_place (bool)
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.SiLU(in_keys: Labels, out_properties: Labels | None = None, *, in_place: bool = False)[source]¶
Module similar to
torch.nn.SiLUthat works withmetatensor.torch.TensorMapobjects.Applies a sigmoid linear unit transformation transformation to each block of a
TensorMappassed to its forward method, indexed by :param in_keys:.Refer to the :py:class`torch.nn.SiLU` documentation for a more detailed description of the parameters.
- Parameters:
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.InvariantSiLU(in_keys: Labels, out_properties: Labels | None = None, invariant_keys: Labels | None = None, *, in_place: bool = False)[source]¶
Module similar to
torch.nn.SiLUthat works withmetatensor.torch.TensorMapobjects, applying the transformation only to the invariant blocks.Applies a sigmoid linear unit transformation to each invariant block of a
TensorMappassed to itsforward()method. These are indexed by the keys in :param in_keys: that correspond to the selection passed in :param invariant_keys:.Refer to the :py:class`torch.nn.SiLU` documentation for a more detailed description of the parameters.
- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.out_properties (Labels | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default the output properties are relabeled using Labels.range.
invariant_keys (Labels | None) – a
Labelsobject that is used to select the invariant keys fromin_keys. If not provided, the invariant keys are assumed to be those where key dimensions["o3_lambda", "o3_sigma"]are equal to[0, 1].in_place (bool)
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.LayerNorm(in_keys: Labels, in_features: List[int], out_properties: List[Labels] | None = None, *, eps: float = 1e-05, elementwise_affine: bool = True, bias: bool = True, mean: bool = True, device: device | None = None, dtype: dtype | None = None)[source]¶
Module similar to
torch.nn.LayerNormthat works withmetatensor.torch.TensorMapobjects.Applies a layer normalization to each block of a
TensorMappassed to itsforward()method, indexed by :param in_keys:.The main difference from
torch.nn.LayerNormis that there is no normalized_shape parameter. Instead, the standard deviation and mean (if applicable) are calculated over all dimensions except the samples (first) dimension of eachTensorBlock.The extra parameter :param mean: controls whether or not the mean over these dimensions is subtracted from the input tensor in the transformation.
Refer to the :py:class`torch.nn.LayerNorm` documentation for a more detailed description of the other parameters.
Each parameter is passed as a single value of its expected type, which is used as the parameter for all blocks.
- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.in_features (List[int]) – list of int, the number of features in the input tensor for each block indexed by the keys in :param in_keys:. If passed as a single value, the same number of features is assumed for all blocks.
out_properties (List[Labels] | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default (if none) the output properties are relabeled using Labels.range.
eps (float)
elementwise_affine (bool)
bias (bool)
mean (bool)
device (device | None)
dtype (dtype | None)
- Mean bool:
whether or not to subtract the mean over all dimensions except the samples (first) dimension of each block of the input passed to
forward().
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.InvariantLayerNorm(in_keys: Labels, in_features: List[int], out_properties: List[Labels] | None = None, invariant_keys: Labels | None = None, *, eps: float = 1e-05, elementwise_affine: bool = True, bias: bool = True, mean: bool = True, device: device | None = None, dtype: dtype | None = None)[source]¶
Module similar to
torch.nn.LayerNormthat works withmetatensor.torch.TensorMapobjects, applying the transformation only to the invariant blocks.Applies a layer normalization to each invariant block of a
TensorMappassed toforward()method. These are indexed by the keys in :param in_keys: that correspond to the selection passed in :param invariant_keys:.The main difference from
torch.nn.LayerNormis that there is no normalized_shape parameter. Instead, the standard deviation and mean (if applicable) are calculated over all dimensions except the samples (first) dimension of eachTensorBlock.The extra parameter :param mean: controls whether or not the mean over these dimensions is subtracted from the input tensor in the transformation.
Refer to the :py:class`torch.nn.LayerNorm` documentation for a more detailed description of the other parameters.
Each parameter is passed as a single value of its expected type, which is used as the parameter for all blocks.
- Parameters:
in_keys (Labels) –
Labels, the keys that are assumed to be in the input tensor map in theforward()method.in_features (List[int]) – list of int, the number of features in the input tensor for each block indexed by the keys in :param in_keys:. If passed as a single value, the same number of features is assumed for all blocks.
out_properties (List[Labels] | None) – list of :py:class`Labels` (optional), the properties labels of the output. By default (if none) the output properties are relabeled using Labels.range.
invariant_keys (Labels | None) – a
Labelsobject that is used to select the invariant keys fromin_keys. If not provided, the invariant keys are assumed to be those where key dimensions["o3_lambda", "o3_sigma"]are equal to[0, 1].eps (float)
elementwise_affine (bool)
bias (bool)
mean (bool)
device (device | None)
dtype (dtype | None)
- Mean bool:
whether or not to subtract the mean over all dimensions except the samples (first) dimension of each block of the input passed to
forward().
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- class metatensor.learn.nn.EquivariantTransformation(modules: List[Module], in_keys: Labels, in_features: int | List[int], out_features: List[int] | int | None = None, out_properties: List[Labels] | None = None, invariant_keys: Labels | None = None)[source]¶
A custom
torch.nn.Modulethat applies an arbitrary shape- and equivariance-preserving transformation to an inputTensorMap.For invariant blocks (specified with
invariant_keys), the respective transformation contained in :param modules: is applied as is. For covariant blocks, an invariant multiplier is created, applying the transformation to the norm of the block over the component dimensions.- Parameters:
modules (List[Module]) – a
listoftorch.nn.Modulecontaining the transformations to be applied to each block indexed by :param in_keys:. Transformations for invariant and covariant blocks differ. See above.in_keys (Labels) –
Labels, the keys that are assumed to be in the inputTensorMapin theforward()method.in_features (int | List[int]) –
listofint, the number of features in the input tensor for each block indexed by the keys in :param in_keys:. If passed as a single value, the same number of features is assumed for all blocks.out_properties (List[Labels] | None) –
listof :py:class`Labels` (optional), the properties labels of the output. By default the output properties are relabeled using Labels.range.invariant_keys (Labels | None) –
a
Labelsobject that is used to select the invariant keys fromin_keys. If not provided, the invariant keys are assumed to be those where key dimensions["o3_lambda", "o3_sigma"]are equal to[0, 1].>>> import torch >>> import numpy as np >>> from metatensor import Labels, TensorBlock, TensorMap >>> from metatensor.learn.nn import EquivariantTransformation
Define a dummy invariant TensorBlock
>>> block_1 = TensorBlock( ... values=torch.randn(2, 1, 3), ... samples=Labels( ... ["system", "atom"], ... np.array( ... [ ... [0, 0], ... [0, 1], ... ] ... ), ... ), ... components=[Labels(["o3_mu"], np.array([[0]]))], ... properties=Labels(["properties"], np.array([[0], [1], [2]])), ... )
Define a dummy covariant TensorBlock
>>> block_2 = TensorBlock( ... values=torch.randn(2, 3, 3), ... samples=Labels( ... ["system", "atom"], ... np.array( ... [ ... [0, 0], ... [0, 1], ... ] ... ), ... ), ... components=[Labels(["o3_mu"], np.array([[-1], [0], [1]]))], ... properties=Labels(["properties"], np.array([[3], [4], [5]])), ... )
Create a TensorMap containing the dummy TensorBlocks
>>> keys = Labels(names=["o3_lambda"], values=np.array([[0], [1]])) >>> tensor = TensorMap(keys, [block_1, block_2])
Define the transformation to apply to the TensorMap
>>> modules = [torch.nn.Tanh(), torch.nn.Tanh()] >>> in_features = [len(tensor.block(key).properties) for key in tensor.keys]
Define the EquivariantTransformation module
>>> transformation = EquivariantTransformation( ... modules, ... tensor.keys, ... in_features, ... out_properties=[tensor.block(key).properties for key in tensor.keys], ... invariant_keys=Labels( ... ["o3_lambda"], np.array([0], dtype=np.int64).reshape(-1, 1) ... ), ... )
The output metadata are the same as the input
>>> transformation(tensor) TensorMap with 2 blocks keys: o3_lambda 0 1 >>> transformation(tensor)[0] TensorBlock samples (2): ['system', 'atom'] components (1): ['o3_mu'] properties (3): ['properties'] gradients: None
Initialize internal Module state, shared by both nn.Module and ScriptModule.