Positive Definite Matrices
\(\operatorname{PSD}(n)\) is the manifold of positive definite matrices.
It is realized via an eigenvalue-like factorization:
where we have identified the vector \(\Lambda\) with a diagonal matrix in \(\mathbb{R}^{n \times n}\). The function \(f\colon \mathbb{R} \to (0, \infty)\) is applied element-wise to the diagonal. By default, the softmax function is used
where we use a small \(\varepsilon > 0\) for numerical stability.
Note
For practical applications, it is more convenient to use the class geotorch.PSSD
, unless the positive definiteness condition is essential. This is because geotorch.PSSD
is less restrictive, and most of the times it will converge to a max-rank solution anyway, although in the optimization process there might be times when the matrix might become almost singular.
- class geotorch.PSD(size, f='softplus', triv='expm')[source]
Manifold of symmetric positive definite matrices
- Parameters
size (torch.size) – Size of the tensor to be parametrized
f (str or callable or pair of callables) –
Optional. Either:
"softplus"
A callable that maps real numbers to the interval \((0, \infty)\)
A pair of callables such that the first maps the real numbers to \((0, \infty)\) and the second is a (right) inverse of the first
Default:
"softplus"
triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(Q\) in the eigenvalue decomposition. It can be one of
["expm", "cayley"]
or a custom callable. Default:"expm"
- sample(init_=<function xavier_normal_>, eps=5e-06)
Returns a randomly sampled matrix on the manifold as
\[WW^\intercal \qquad W_{i,j} \sim \texttt{init_}\]If the sampled matrix has more than self.rank small singular values, the smallest ones are clamped to be at least
eps
in absolute value.The output of this method can be used to initialize a parametrized tensor as:
>>> layer = nn.Linear(20, 20) >>> M = PSSD(layer.weight.size()) >>> geotorch.register_parametrization(layer, "weight", M) >>> layer.weight = M.sample()
- Parameters
init_ (callable) – Optional. A function that takes a tensor and fills it in place according to some distribution. See torch.init. Default:
torch.nn.init.xavier_normal_
eps (float) – Optional. Minimum eigenvalue of the sampled matrix. Default:
5e-6
- in_manifold(X, eps=1e-06)
Checks that a matrix is in the manifold.
- Parameters
X (torch.Tensor) – The matrix or batch of matrices of shape
(*, n, n)
to check.eps (float) – Optional. Threshold at which the singular values are considered to be zero. Default:
1e-6