Constraints API

These are the functions that form the basic interface of GeoTorch. They all provide a common interface to the spaces implemented in GeoTorch. For a finer control over the underlying implementation of each space, take a look at the different manifolds.

geotorch.symmetric(module, tensor_name='weight', lower=True)[source]

Adds a symmetric parametrization to the matrix module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) so that \(X^\intercal = X\).

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(30, 30)
>>> geotorch.symmetric(layer, "weight")
>>> torch.allclose(layer.weight, layer.weight.T)
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
lower (bool) – Optional. Uses the lower triangular part of the matrix to parametrize the matrix. Default: True

geotorch.skew(module, tensor_name='weight', lower=True)[source]

Adds a skew-symmetric parametrization to the matrix module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) so that \(X^\intercal = -X\).

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(30, 30)
>>> geotorch.skew(layer, "weight")
>>> torch.allclose(layer.weight, -layer.weight.T)
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
lower (bool) – Optional. Uses the lower triangular part of the matrix to parametrize the matrix. Default: True

geotorch.sphere(module, tensor_name='weight', radius=1.0, embedded=False)[source]

Adds a spherical parametrization to the vector (or tensor) module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(v\) so that \(\lVert v \rVert = 1\).

If the tensor has more than one dimension, the parametrization will be applied to the last dimension.

Examples:

>>> layer = nn.Linear(20, 30)
>>> geotorch.sphere(layer, "bias")
>>> torch.norm(layer.bias)
tensor(1.)
>>> geotorch.sphere(layer, "weight")  # Make the columns unit norm
>>> torch.allclose(torch.norm(layer.weight, dim=-1), torch.ones(30))
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
radius (float) – Optional. Radius of the sphere. It has to be positive. Default: 1.
embedded (bool) – Optional. Chooses between the implementation of the sphere using the exponential map (embedded=False) and that using the projection from the ambient space (embedded=True) Default. True

geotorch.orthogonal(module, tensor_name='weight', triv='expm')[source]

Adds an orthogonal parametrization to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) so that \(X^\intercal X = \operatorname{I}\).

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 30)
>>> geotorch.orthogonal(layer, "weight")
>>> torch.norm(layer.weight.T @ layer.weight - torch.eye(20,20))
tensor(4.8488e-05)

>>> layer = nn.Conv2d(20, 40, 3, 3)  # Make the kernels orthogonal
>>> geotorch.orthogonal(layer, "weight")
>>> torch.norm(layer.weight.transpose(-2, -1) @ layer.weight - torch.eye(3,3))
tensor(1.2225e-05)

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
triv (str or callable) – Optional. A map that maps a skew-symmetric matrix to an orthogonal matrix. It can be the exponential of matrices or the cayley transform passing ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.almost_orthogonal(module, tensor_name='weight', lam=0.1, f='sin', triv='expm')[source]

Adds an almost orthogonal parametrization to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will have its singular values in the interval \([1-\texttt{lam}, 1+\texttt{lam}]\)

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 30)
>>> geotorch.almost_orthogonal(layer, "weight", 0.5)
>>> S = torch.linalg.svd(layer.weight).S
>>> all(S >= 0.5 and S <= 1.5)
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
lam (float) – Radius of the interval for the singular values. A float in the interval \([0, 1]\). Default: 0.1
f (str or callable or pair of callables) –
Optional. Either:
- One of ["scaled_sigmoid", "tanh", "sin"]
- A callable that maps real numbers to the interval \([-1, 1]\)
- A pair of callables such that the first maps the real numbers to \([-1, 1]\) and the second is a (right) inverse of the first
Default: "sin"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.grassmannian(module, tensor_name='weight', triv='expm')[source]

Adds an parametrization to the tensor module.tensor_name so that the result represents a subspace. If the initial matrix was of size \(n \times k\) the parametrized matrix will represent a subspace of dimension \(k\) of \(\mathbb{R}^n\).

When accessing module.tensor_name, the module will return the parametrized version \(X\) so that \(X\) represents \(k\) orthogonal vectors of \(\mathbb{R}^n\) that span the subspace. That is, the resulting matrix will be orthogonal, \(X^\intercal X = \operatorname{I}\).

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Note

Even though this space resembles that generated by geotorch.orthogonal(), it is actually a subspace of that, as every subspace can be represented by many different basis of vectors that span it.

Examples:

>>> layer = nn.Linear(20, 30)
>>> geotorch.grassmannian(layer, "weight")
>>> torch.norm(layer.weight.t() @ layer.weight - torch.eye(20,20))
tensor(1.8933e-05)

>>> layer = nn.Conv2d(20, 40, 3, 3)  # Make the kernels represent subspaces
>>> geotorch.grassmannian(layer, "weight")
>>> torch.norm(layer.weight.transpose(-2, -1) @ layer.weight - torch.eye(3,3))
tensor(8.3796-06)

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
triv (str or callable) – Optional. A map that maps a skew-symmetric matrix to an orthogonal matrix. It can be the exponential of matrices or the cayley transform passing ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.low_rank(module, tensor_name, rank, triv='expm')[source]

Adds a low rank parametrization to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will have rank at most rank.

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 30)
>>> geotorch.low_rank(layer, "weight", 4)
>>> list(torch.linalg.svd(layer.weight).S > 1e-7).count(True) <= 4
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.fixed_rank(module, tensor_name, rank, f='softplus', triv='expm')[source]

Adds a fixed rank parametrization to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will have rank equal to rank.

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 30)
>>> geotorch.fixed_rank(layer, "weight", 5)
>>> list(torch.linalg.svd(layer.weight).S > 1e-7).count(True)
5

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
f (str or callable or pair of callables) –
Optional. Either:
- "softplus"
- A callable that maps real numbers to the interval \((0, \infty)\)
- A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default: "softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.invertible(module, tensor_name='weight', f='softplus', triv='expm')[source]

Adds an invertibility constraint to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will have positive determinant and, in particular, it will be invertible.

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 20)
>>> geotorch.invertible(layer, "weight", 5)
>>> torch.det(layer.weight) > 0.0
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
f (str or callable or pair of callables) –
Optional. Either:
- "softplus"
- A callable that maps real numbers to the interval \((0, \infty)\)
- A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default: "softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.positive_definite(module, tensor_name='weight', f='softplus', triv='expm')[source]

Adds a positive definiteness constraint to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will be symmetric and with positive eigenvalues

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 20)
>>> geotorch.positive_definite(layer, "weight")
>>> (torch.linalg.eigvalsh(layer.weight) > 0.0).all()
tensor(True)

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
f (str or callable or pair of callables) –
Optional. Either:
- "softplus"
- A callable that maps real numbers to the interval \((0, \infty)\)
- A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default: "softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.positive_semidefinite(module, tensor_name='weight', triv='expm')[source]

Adds a positive definiteness constraint to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will be symmetric and with non-negative eigenvalues

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 20)
>>> geotorch.positive_semidefinite(layer, "weight")
>>> L = torch.linalg.eigvalsh(layer.weight)
>>> L[L.abs() < 1e-7] = 0.0  # Round errors
>>> (L >= 0.0).all()
tensor(True)

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied. Default: "weight"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.positive_semidefinite_low_rank(module, tensor_name, rank, triv='expm')[source]

Adds a positive definiteness constraint to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will be symmetric and with non-negative eigenvalues and at most rank of them non-zero.

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 20)
>>> geotorch.positive_semidefinite_low_rank(layer, "weight", 5)
>>> L = torch.linalg.eigvalsh(layer.weight)
>>> L[L.abs() < 1e-7] = 0.0  # Round errors
>>> (L >= 0.0).all()
tensor(True)
>>> list(L > 0.0).count(True) <= 5
True

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

geotorch.positive_semidefinite_fixed_rank(module, tensor_name, rank, f='softplus', triv='expm')[source]

Adds a positive definiteness constraint to the tensor module.tensor_name.

When accessing module.tensor_name, the module will return the parametrized version \(X\) which will be symmetric and with non-negative eigenvalues and exactly rank of them non-zero.

If the tensor has more than two dimensions, the parametrization will be applied to the last two dimensions.

Examples:

>>> layer = nn.Linear(20, 20)
>>> geotorch.positive_semidefinite_fixed_rank(layer, "weight", 5)
>>> L = torch.linalg.eigvalsh(layer.weight)
>>> L[L.abs() < 1e-7] = 0.0  # Round errors
>>> (L >= 0.0).all()
tensor(True)
>>> list(L > 0.0).count(True)
5

Parameters

module (nn.Module) – module on which to register the parametrization
tensor_name (string) – name of the parameter, buffer, or parametrization on which the parametrization will be applied
rank (int) – Rank of the matrix. It has to be less than the minimum of the two dimensions of the matrix
f (str or callable or pair of callables) –
Optional. Either:
- "softplus"
- A callable that maps real numbers to the interval \((0, \infty)\)
- A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default: "softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"