Parameterizing Cholesky factors of full rank covariances

1

So I’m trying to parameterize a set of full-rank covariance or precision matrices for, eg., Gaussian mixture models or mixture density networks.

I want to define torch.nn.Parameters for the (log) diagonals and lower triangular parts of the Cholesky factors respectively, and then package those parameters together into the Cholesky factors (and then covariance or precision matrices).

In tensorflow I can do this easily with tf.distributions.fill_triangular(), but as far as I can tell no such function exists in torch. What’s the easiest way to achieve something equivalent in torch?

1 Like
2

You can use the same trick described in tensorflow fill_triangular, but converted to PyTorch, here is an example for the lower triangular of a 3x3 matrix:

x = torch.arange(6) + 1 
xc = torch.cat([x[3:], x.flip(dims=[0])])
y = xc.view(3, 3)
torch.tril(y)

tensor([[4, 0, 0],
        [6, 5, 0], 
        [3, 2, 1]])
1 Like