Can I get the ouput of a layer that sits inside nn.Sequential?

1

Say,

class Model(nn.Module):
  def __init__(self):
    super().__init__()
    self.net = nn.Sequential(nn.Conv2d(3, 3, 1, 1, 0), 
                             nn.Conv2d(3, 3, 1, 1, 0))

  def forward(self, x):
    for name, layer in self.net.named_modules():
      x = layer(x)

Now the thing is .named_modlues() will give me
first the whole thing
then the Sequential (so now my code has already propagated the input through the entire network)
And then the two convs.

I just want the layers to iterate over one by one.

One hardcoded solution is to skip over the first 2. which is not a good solution.
Another is to, check if the layer is present in a list of acceptable layers and then pass the x through it.

Any other solutions?

2 
>>> import torch
>>> import torch.nn as nn
>>> s = nn.Sequential()
>>> s.add_module("conv", nn.Conv2d(1, 1, 4))
>>> s.conv
Conv2d(1, 1, kernel_size=(4, 4), stride=(1, 1))
>>>

You can do this. Also, you should not be looping inside your custom forward() method. The Sequential class will take care of passing outputs to inputs of the subsequent layers. Just return the result of self.net(x)

3

I am sorry but I think you misunderstood my question.
I want the output of the first conv not the output after it has passed through all the layers. That’s why its necessary to loop.
In general, I want the output of the intermediate layers in a nn.Sequential as the question says.

4

Yep! I understand.

If you change your code to be like this:

import torch
import torch.nn as nn

class Model(nn.Sequential):
  def __init__(self):
    super().__init__()
    [self.add_module("conv_{}".format(n), nn.Conv2d(3, 3, 1, 1, 0)) for n in range(2)]

m = Model()

print(m)
print(m.conv_0)

You will have a Sequential model that implicitly chains the output of conv_0 to the input of conv_1 so there is no need for the loop in the forward method.

If you want to get a layer, it will be available as a property that matches the string passed for the module name in add_module. So in this case m.conv_0 is how you would get at the first layer without looping and matching the name.

You can call it directly on some input like m.conv_0(torch.randn(1, 3, 4, 4)). Hope that helps!

If you want to keep something closer to what you have, you could also do:

from collections import OrderedDict as odict
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super().__init__()

        layers = odict(
            [
                ("conv_0", nn.Conv2d(3, 3, 1, 1, 0)),
                ("conv_1", nn.Conv2d(3, 3, 1, 1, 0))
            ]
        )

        self.net = nn.Sequential(layers)

    def forward(self, x):
        return self.net(x)

m = Model()

print(m.net.conv_0)

But it seems weird to use a Module to wrap a single Sequential container. Why not inherit from nn.Sequential directly?

1 Like
5

Yeah, you are right. Thanks I will update my code.

1 Like