I’m trying to work my way through Mixtures Density Network, and while I managed to figure it out for a single feature for the output, I’m stuck when it comes to several features. Among my problems, here’s the one concerning the Normal distribution class.
Here’s an example:
import torch
from torch.distributions import Normal
means = torch.zeros(1,2)
stds = torch.ones(1,2)
dist = Normal(means, stds)
print('Sample is size {}'.format(dist.sample().shape))
print('Log prob of {} = {}'.format(torch.zeros(1,2), dist.log_prob(torch.zeros(1,2))))
Outputs:
Sample is size torch.Size([1, 2])
Log prob of tensor([[ 0., 0.]]) = tensor([[-0.9189, -0.9189]])
Shouldn’t the log probability be of a single dimension ? Am I (certainly but what) doing something wrong ?
Never mind, I figured it out !
I think that Normal generates one normal distribution per feature. Hence, you have as many log_probs as you have distributions.
However, in the case of multi-features, here’s what I propose:
Either keep Normal, but sum log probs (or multiply probs) to get the probability you seek given multifeatures
Use a Multivariate Normal, which avoids you to gather the probabilites yourself.
If anyone has other insights, or if I’ve made a mistake, I’d be delighted to hear it !