Constraints API
These are the functions that form the basic interface of GeoTorch. They all provide a common interface to the spaces implemented in GeoTorch. For a finer control over the underlying implementation of each space, take a look at the different manifolds.
- geotorch.symmetric(module, tensor_name='weight', lower=True)[source]
Adds a symmetric parametrization to the matrix
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) so that \(X^\intercal = X\).If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(30, 30) >>> geotorch.symmetric(layer, "weight") >>> torch.allclose(layer.weight, layer.weight.T) True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
lower (bool) – Optional. Uses the lower triangular part of the matrix to parametrize the matrix. Default:
True
- geotorch.skew(module, tensor_name='weight', lower=True)[source]
Adds a skew-symmetric parametrization to the matrix
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) so that \(X^\intercal = -X\).If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(30, 30) >>> geotorch.skew(layer, "weight") >>> torch.allclose(layer.weight, -layer.weight.T) True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
lower (bool) – Optional. Uses the lower triangular part of the matrix to parametrize the matrix. Default:
True
- geotorch.sphere(module, tensor_name='weight', radius=1.0, embedded=False)[source]
Adds a spherical parametrization to the vector (or tensor)
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(v\) so that \(\lVert v \rVert = 1\).If the tensor has more than one dimension, the parametrization will be applied to the last dimension.
Examples:
>>> layer = nn.Linear(20, 30) >>> geotorch.sphere(layer, "bias") >>> torch.norm(layer.bias) tensor(1.) >>> geotorch.sphere(layer, "weight") # Make the columns unit norm >>> torch.allclose(torch.norm(layer.weight, dim=-1), torch.ones(30)) True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
radius (float) – Optional. Radius of the sphere. It has to be positive. Default: 1.
embedded (bool) – Optional. Chooses between the implementation of the sphere using the exponential map (
embedded=False
) and that using the projection from the ambient space (embedded=True
) Default.True
- geotorch.orthogonal(module, tensor_name='weight', triv='expm')[source]
Adds an orthogonal parametrization to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) so that \(X^\intercal X = \operatorname{I}\).If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 30) >>> geotorch.orthogonal(layer, "weight") >>> torch.norm(layer.weight.T @ layer.weight - torch.eye(20,20)) tensor(4.8488e-05) >>> layer = nn.Conv2d(20, 40, 3, 3) # Make the kernels orthogonal >>> geotorch.orthogonal(layer, "weight") >>> torch.norm(layer.weight.transpose(-2, -1) @ layer.weight - torch.eye(3,3)) tensor(1.2225e-05)
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
triv (str or callable) – Optional. A map that maps a skew-symmetric matrix to an orthogonal matrix. It can be the exponential of matrices or the cayley transform passing
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.almost_orthogonal(module, tensor_name='weight', lam=0.1, f='sin', triv='expm')[source]
Adds an almost orthogonal parametrization to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will have its singular values in the interval \([1-\texttt{lam}, 1+\texttt{lam}]\)If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 30) >>> geotorch.almost_orthogonal(layer, "weight", 0.5) >>> S = torch.linalg.svd(layer.weight).S >>> all(S >= 0.5 and S <= 1.5) True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
lam (float) – Radius of the interval for the singular values. A float in the interval \([0, 1]\). Default:
0.1
f (str or callable or pair of callables) –
Optional. Either:
One of
["scaled_sigmoid", "tanh", "sin"]
A callable that maps real numbers to the interval \([-1, 1]\)
A pair of callables such that the first maps the real numbers to \([-1, 1]\) and the second is a (right) inverse of the first
Default:
"sin"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.grassmannian(module, tensor_name='weight', triv='expm')[source]
Adds an parametrization to the tensor
module.tensor_name
so that the result represents a subspace. If the initial matrix was of size \(n \times k\) the parametrized matrix will represent a subspace of dimension \(k\) of \(\mathbb{R}^n\).When accessing
module.tensor_name
, the module will return the parametrized version \(X\) so that \(X\) represents \(k\) orthogonal vectors of \(\mathbb{R}^n\) that span the subspace. That is, the resulting matrix will be orthogonal, \(X^\intercal X = \operatorname{I}\).If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Note
Even though this space resembles that generated by
geotorch.orthogonal()
, it is actually a subspace of that, as every subspace can be represented by many different basis of vectors that span it.Examples:
>>> layer = nn.Linear(20, 30) >>> geotorch.grassmannian(layer, "weight") >>> torch.norm(layer.weight.t() @ layer.weight - torch.eye(20,20)) tensor(1.8933e-05) >>> layer = nn.Conv2d(20, 40, 3, 3) # Make the kernels represent subspaces >>> geotorch.grassmannian(layer, "weight") >>> torch.norm(layer.weight.transpose(-2, -1) @ layer.weight - torch.eye(3,3)) tensor(8.3796-06)
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
triv (str or callable) – Optional. A map that maps a skew-symmetric matrix to an orthogonal matrix. It can be the exponential of matrices or the cayley transform passing
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.low_rank(module, tensor_name, rank, triv='expm')[source]
Adds a low rank parametrization to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will have rank at mostrank
.If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 30) >>> geotorch.low_rank(layer, "weight", 4) >>> list(torch.linalg.svd(layer.weight).S > 1e-7).count(True) <= 4 True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.fixed_rank(module, tensor_name, rank, f='softplus', triv='expm')[source]
Adds a fixed rank parametrization to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will have rank equal torank
.If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 30) >>> geotorch.fixed_rank(layer, "weight", 5) >>> list(torch.linalg.svd(layer.weight).S > 1e-7).count(True) 5
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
f (str or callable or pair of callables) –
Optional. Either:
"softplus"
A callable that maps real numbers to the interval \((0, \infty)\)
A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default:
"softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.invertible(module, tensor_name='weight', f='softplus', triv='expm')[source]
Adds an invertibility constraint to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will have positive determinant and, in particular, it will be invertible.If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 20) >>> geotorch.invertible(layer, "weight") >>> torch.det(layer.weight) > 0.0 True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
f (str or callable or pair of callables) –
Optional. Either:
"softplus"
A callable that maps real numbers to the interval \((0, \infty)\)
A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default:
"softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.sln(module, tensor_name='weight', f='softplus', triv='expm')[source]
Adds a constraint of having determinant one to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will have determinant equal to 1.If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 20) >>> geotorch.sln(layer, "weight") >>> torch.det(layer.weight) tensor(1.0000)
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
f (str or callable or pair of callables) –
Optional. Either:
"softplus"
A callable that maps real numbers to the interval \((0, \infty)\)
A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default:
"softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.positive_definite(module, tensor_name='weight', f='softplus', triv='expm')[source]
Adds a positive definiteness constraint to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will be symmetric and with positive eigenvaluesIf the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 20) >>> geotorch.positive_definite(layer, "weight") >>> (torch.linalg.eigvalsh(layer.weight) > 0.0).all() tensor(True)
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
f (str or callable or pair of callables) –
Optional. Either:
"softplus"
A callable that maps real numbers to the interval \((0, \infty)\)
A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default:
"softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.positive_semidefinite(module, tensor_name='weight', triv='expm')[source]
Adds a positive definiteness constraint to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will be symmetric and with non-negative eigenvaluesIf the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 20) >>> geotorch.positive_semidefinite(layer, "weight") >>> L = torch.linalg.eigvalsh(layer.weight) >>> L[L.abs() < 1e-7] = 0.0 # Round errors >>> (L >= 0.0).all() tensor(True)
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default:
"weight"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.positive_semidefinite_low_rank(module, tensor_name, rank, triv='expm')[source]
Adds a positive definiteness constraint to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will be symmetric and with non-negative eigenvalues and at mostrank
of them non-zero.If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 20) >>> geotorch.positive_semidefinite_low_rank(layer, "weight", 5) >>> L = torch.linalg.eigvalsh(layer.weight) >>> L[L.abs() < 1e-7] = 0.0 # Round errors >>> (L >= 0.0).all() tensor(True) >>> list(L > 0.0).count(True) <= 5 True
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- geotorch.positive_semidefinite_fixed_rank(module, tensor_name, rank, f='softplus', triv='expm')[source]
Adds a positive definiteness constraint to the tensor
module.tensor_name
.When accessing
module.tensor_name
, the module will return the parametrized version \(X\) which will be symmetric and with non-negative eigenvalues and exactlyrank
of them non-zero.If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.
Examples:
>>> layer = nn.Linear(20, 20) >>> geotorch.positive_semidefinite_fixed_rank(layer, "weight", 5) >>> L = torch.linalg.eigvalsh(layer.weight) >>> L[L.abs() < 1e-7] = 0.0 # Round errors >>> (L >= 0.0).all() tensor(True) >>> list(L > 0.0).count(True) 5
- Parameters
module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
f (str or callable or pair of callables) –
Optional. Either:
"softplus"
A callable that maps real numbers to the interval \((0, \infty)\)
A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default:
"softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"