Learning to Sample from Complex Distribution (High-Dimensional)

1

Hello everyone,

I would like a NN to learn how to sample from a complex distribution in high-dimensions (e.g. 10-100K dimensions).

I guess that would be possible via learning parameters of a multivariate normal distribution e.g. MultivariateNormal

However, in high-dimensional problems, the parameter-size becomes too much and this becomes memory inefficient.

Can you please suggest to me an alternative if it exists in the framework or another method in order for achieving this?

Note: The scalability of the method is the most crucial aspect for me, the resulting expressiveness is not that important.

Thank you very much.

Sincerely,
Kamer

2

Hi Kamer!

I don’t think that neural networks are a natural approach for sampling
from high-dimensional distributions – not that I know what your
specific use case is.

Your distribution is described by a probability-destiny function that is
a function of the 10-100K coordinates of your high-dimensional space.
Can you compute reasonably efficiently the changes in the PDF as
you take “simple” steps that move you about in that space?

If so, the “standard” approach for sampling from such a distribution
is a Markov-chain Monte Carlo.

Best.

K. Frank

3

Hello Frank,

Thank you for your response. I was also recently looking into MCMC sampling methods and will look deeper after your advise. Normalizing Flows seems to be the most popular approach but I am unsure if they’re scalable enough for my use-case or would need too much additional parameters to be trained.

Sincerely,
Kamer