# Model produces random output when using loops to define hidden layers

**URL:** <https://discuss.pytorch.org/t/model-produces-random-output-when-using-loops-to-define-hidden-layers/265>\
**Category:** Uncategorized\
**Created:** [January 30, 2017, 4:37am UTC](https://discuss.pytorch.org/t/model-produces-random-output-when-using-loops-to-define-hidden-layers/265 "2017-01-30T04:37:28Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jordan\_Campbell](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jordan_campbell/32/91_2.png) [@Jordan\_Campbell](https://discuss.pytorch.org/u/Jordan_Campbell)\
**Post date:** [January 30, 2017, 4:37am UTC](https://discuss.pytorch.org/t/model-produces-random-output-when-using-loops-to-define-hidden-layers/265/1 "2017-01-30T04:37:28Z")

</div>

Hi,

I’m adapting the DCGAN models from the PyTorch examples and had a problem saving and loading the parameters.

Specifically, when I saved the parameters and then read them back in at a later stage, I would get a different output for a consistent input.

I debugged and found what I believed to be the problem, however I’m not sure of the cause. I am using a loop to define hidden layers of the MLP (I’m not using the conv nets):

```
self.nhl = 4
self.nh = [2, 2, 2, 2]
self.fc = []
for i in range(self.nhl-1):
    self.fc.append(nn.Linear(self.nh[i], self.nh[i+1]))

```

which gave the following output on repeated executions of the program (with a model saved earlier):

```
 jordan [src] $ python main.py 

First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.558199584484
hidden layer --> 0.203323662281
output --> 0.305809795856 -0.139149516821 0.538217186928

 jordan [src] $ python main.py 
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.45568972826
hidden layer --> 0.0742047131062
output --> 0.425929844379 -0.366338938475 0.542093634605
 
jordan [src] $ python main.py 
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.0
hidden layer --> 0.460717827082
output --> 0.345742940903 -0.0932856351137 0.694804787636

```

Note that the outputs at each of the layers changes on each execution (I confirmed the parameters loaded are all consistent on each iteration).

Forward is implemented as:

```
    x = F.relu(self.input(_input))

    print 'First layer -->', x.data[0][0], x.data[0][1]
    for i in range(self.nhl-1):
        x = F.relu(self.fc[i]((x)))
        print 'hidden layer -->', x.data[0][0]

    output = self.output(x)

```

If I unroll the loops then it produces consistent output. Is there a bug in my loop or is there a reason you can’t use loops to define hidden layers?

---

<div class="post-metadata">

**Author:** ![NgPDat](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/7feea3/32.png) [@NgPDat](https://discuss.pytorch.org/u/NgPDat)\
**Post date:** [January 30, 2017, 9:27am UTC](https://discuss.pytorch.org/t/model-produces-random-output-when-using-loops-to-define-hidden-layers/265/2 "2017-01-30T09:27:27Z")

</div>

Hi,  
I think the reason is a nn.Module does not recognize any module/parameter in a Python list. You can check this by `print(model)`  
Therefore, any parameters for those modules won’t show up when you call `model.parameters()`. I guess it also mess up the saving process, leading to the problem you described above.  
Some solution can be found in this thread: [List of nn.Module in a nn.Module](https://discuss.pytorch.org/t/list-of-nn-module-in-a-nn-module/219)

---

<div class="post-metadata">

**Author:** ![Jordan\_Campbell](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jordan_campbell/32/91_2.png) [@Jordan\_Campbell](https://discuss.pytorch.org/u/Jordan_Campbell)\
**Post date:** [January 30, 2017, 9:32am UTC](https://discuss.pytorch.org/t/model-produces-random-output-when-using-loops-to-define-hidden-layers/265/3 "2017-01-30T09:32:19Z")

</div>

Hi @NgPDat, thanks - that helps a lot!