# How to correctly use a normal distribution?

Let’s look at this small code:

``````import torch
import torch.nn as nn
from torch.distributions import Normal

class Proba(nn.Module):
def __init__(self, size):
super(Proba, self).__init__()
self.mu = nn.Linear(size, size)

def forward(self, obs):
mu = self.mu(obs)
dist = Normal(mu, 0.2)
log_prob = dist.log_prob(obs)
return log_prob.mean()

proba = Proba(10)

# fixed observation to reconstruct:
obs = torch.rand(10)

# want to optimize the probability to reconstruct obs:
if __name__=="__main__":

for i in range(500):
loss = -proba(obs)
loss.backward()
optimizer.step()

if i%100==0:
# should never be larger than 0:
print("log_prob of obs =", -loss.item())
``````

The goal of this code is to reconstruct a signal “obs” by training a linear model that overfits an element-wise normal distribution with a fixed small standard deviation (0.2).

However, after ~500 iterations, the predicted log_prob starts being positive, while log_probabilites should never be positive.

Is something wrong in the code and the utilization of `distributions.Normal`?

Okay, I’ve just been through the source code and understood that it’s actually the log of the density, and not the log of a probability.

It makes sense since the measure of the event is null. I was rather expecting the proba of being in the interval [mu-|obs|, mu+|obs|]

Hi @alexis-jacq by looking at your question, is this a way to fit probabilistic models in pytorch, say estima the mean and variance for a normal likelyhood regression model?

@alexis-jacq Basically, I am trying to replicate this code from Keras to Pytorch:

``````def NLL(y, distr):
return -distr.log_prob(y) #A

def my_dist(params):
return tfd.Normal(loc=params, scale=1)

# set the sd to the fixed value 1
inputs = Input(shape=(1,))
params = Dense(1)(inputs)
dist = tfp.layers.DistributionLambda(my_dist)(params)

model_sd_1 = Model(inputs=inputs, outputs=dist)