What the difference of directly forward and separately forward of Sequential?

autograd
1

Hi,
I want to use a Sequential to build a model but i just use each of them to froward? what the difference of them?
e.g.,
m = Sequential(nn.Conv2d(),nn.ReLU(),nn.Conv2d(),nn.ReLU())
what the difference of the following of
o = m(x)
and
o = m[1](m[0](x))
o = m[3](m[2](o))
do the backward have some influences or other difference?

Thank a lot.

2

If I understand your question correctly. There is no difference in using them together or separately. Here is a quote from sowmith.

3

Thanks for your reply, and i mean that i define the network by sequential and in forward process, i want to get intermediate outputs, so i use o1=m[0](m[1](x)) o2=m[3](m[2](o1)) and output o1, and o2. I want make sure that the process whether the same as o=m(x)（including forward , backward, weight update ect.）

4

Yeah what happens in the background is same.

5

ok,thanks a lot. By the way, do the eval mode fix the running_mean and var and weight and bias of BatchNorm2d, or only fix the mean and var? When fixing the bn, is fix all the 4 or only mean and var, thanks.

7

In eval BatchNorm is using the running estimates, but doesn’t update them (if track_running_stats=True which is the default).