Low Rank Matrices

\(\operatorname{LowRank}(n,k,r)\) is the algebraic variety of matrices of rank less or equal to \(r\), for a given \(r \leq \min\{n, k\}\):

\[\operatorname{LowRank}(n,k,r) = \{X \in \mathbb{R}^{n\times k}\:\mid\:\operatorname{rank}(X) \leq r\}\]

It is realized via an SVD-like factorization:

\[\begin{split}\begin{align*} \pi \colon \operatorname{St}(n,r) \times \mathbb{R}^r \times \operatorname{St}(k, r) &\to \operatorname{LowRank}(n,k,r) \\ (U, \Sigma, V) &\mapsto U\Sigma V^\intercal \end{align*}\end{split}\]

where we have identified the vector \(\Sigma\) with a diagonal matrix in \(\mathbb{R}^{r \times r}\).

class geotorch.LowRank(size, rank, triv='expm')[source]

Variety of the matrices of rank \(r\) or less.

Parameters

  • size (torch.size) – Size of the tensor to be parametrized

  • rank (int) – Rank of the matrices. It has to be less or equal to \(\min(\texttt{size}[-1], \texttt{size}[-2])\)

  • triv (str or callable) – Optional. A map that maps skew-symmetric matrices onto the orthogonal matrices surjectively. This is used to optimize the \(U\) and \(V\) in the SVD. It can be one of ["expm", "cayley"] or a custom callable. Default: "expm"

sample(init_=<function xavier_normal_>, factorized=False)[source]

Returns a randomly sampled matrix on the manifold by sampling a matrix according to init_ and projecting it onto the manifold.

The output of this method can be used to initialize a parametrized tensor that has been parametrized with this or any other manifold as:

>>> layer = nn.Linear(20, 20)
>>> M = LowRank(layer.weight.size(), rank=6)
>>> geotorch.register_parametrization(layer, "weight", M)
>>> layer.weight = M.sample()
Parameters

init_ (callable) – Optional. A function that takes a tensor and fills it in place according to some distribution. See torch.init. Default: torch.nn.init.xavier_normal_

in_manifold(X, eps=1e-05)[source]

Checks that a given matrix is in the manifold.

Parameters

  • X (torch.Tensor or tuple) – The input matrix or matrices of shape (*, n, k).

  • eps (float) – Optional. Threshold at which the singular values are considered to be zero Default: 1e-5