Hi,
I’m adapting the DCGAN models from the PyTorch examples and had a problem saving and loading the parameters.
Specifically, when I saved the parameters and then read them back in at a later stage, I would get a different output for a consistent input.
I debugged and found what I believed to be the problem, however I’m not sure of the cause. I am using a loop to define hidden layers of the MLP (I’m not using the conv nets):
self.nhl = 4
self.nh = [2, 2, 2, 2]
self.fc = []
for i in range(self.nhl-1):
self.fc.append(nn.Linear(self.nh[i], self.nh[i+1]))
which gave the following output on repeated executions of the program (with a model saved earlier):
jordan [ src ] $ python main.py
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.558199584484
hidden layer --> 0.203323662281
output --> 0.305809795856 -0.139149516821 0.538217186928
jordan [ src ] $ python main.py
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.45568972826
hidden layer --> 0.0742047131062
output --> 0.425929844379 -0.366338938475 0.542093634605
jordan [ src ] $ python main.py
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.0
hidden layer --> 0.460717827082
output --> 0.345742940903 -0.0932856351137 0.694804787636
Note that the outputs at each of the layers changes on each execution (I confirmed the parameters loaded are all consistent on each iteration).
Forward is implemented as:
x = F.relu(self.input(_input))
print 'First layer -->', x.data[0][0], x.data[0][1]
for i in range(self.nhl-1):
x = F.relu(self.fc[i]((x)))
print 'hidden layer -->', x.data[0][0]
output = self.output(x)
If I unroll the loops then it produces consistent output. Is there a bug in my loop or is there a reason you can’t use loops to define hidden layers?