Sampling from a MultivariateNormal distribution?

Hello,

I’d like to know if there is a way to sample several variables from a Multivariate Normal distribution in a batch fashion ?

For instance, given a tensor of size [n,3], I tried to stack n covariance matrices of size [3,3] but I get an error when i try to sample.

I couldn’t figure out whether there’s a specific way to define the distribution or it is just not possible. Has anyone faced this yet ?
Thanks!

1 Like

Hi Mehdi,

I just ran into the same problem with an error

RuntimeError: Lapack Error in potrf : the leading minor of order 2 is not positive definite at /Users/soumith/minicondabuild3/conda-bld/pytorch_1524589329605/work/aten/src/TH/generic/THTensorLapack.c:617

and managed to solve it by realizing the covariance matrices I used first were not positive definite. By using symmetric matrices as covariance matrices and stacking them w.r.t. axis 0, I got it to work.

You can try the example below.

import torch
from torch.distributions.multivariate_normal import MultivariateNormal

mean = torch.Tensor([[1, 2, 3], [4, 5, 6]])
cov1 = torch.eye(3)
cov2 = torch.Tensor([[1, 1, 1], [1, 2, 2], [1, 2, 3]])
cov = torch.stack([cov1, cov2], 0)
distrib = MultivariateNormal(loc=mean, covariance_matrix=cov)
distrib.rsample()

As suggested by the source code comments, since they just perform a Cholesky decomposition to an n-by-n covariance matrix to get a scale_tril tensor of size n, it might be a better practice to pass in a tensor of size n directly when possible.

5 Likes