nn.Sequential(*layers) forward: with multiple inputs Error

HI, I have a toy densenet model (e.g. only one dense block, a copy from official implementation) with TWO variables (x,y) for the denseblock. However, there is an error " result = self.forward(*input, **kwargs) TypeError: forward() takes 2 positional arguments but 3 were given". This model works for only one variable (x) . Currenty, I can still not figure out the reason and was wondering if there is something wrong in the nn.Sequential(*layers) part? Thanks in advance!


class BasicBlock(nn.Module):
    def __init__(self, in_planes, out_planes):
        super(BasicBlock, self).__init__()
        self.bn1 = nn.BatchNorm2d(in_planes)
        self.relu = nn.ReLU(inplace=True)
        self.conv1 = nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=1, padding=1, bias=False)

    def forward(self, x,y):
        out1 = self.conv1(self.relu(self.bn1(x)))
        out2 = y
        return torch.cat([x, out1], 1), out2


class DenseBlock(nn.Module):
    def __init__(self, nb_layers, in_planes, growth_rate, block):
        super(DenseBlock, self).__init__()
        self.layer = self._make_layer(block, in_planes, growth_rate, nb_layers)
    def _make_layer(self, block, in_planes, growth_rate, nb_layers):
        layers = []
        for i in range(nb_layers):
            layers.append(block(in_planes+i*growth_rate, growth_rate))
        return nn.Sequential(*layers)
        # return block(in_planes+growth_rate, growth_rate)  #It works by replacing the whole "for" loop  with this line, but we can only obtain one block rather than nb_layes blocks. 
    def forward(self, x,y):
        return self.layer(x,y)

class DenseNet3(nn.Module):
    def __init__(self, n=2, growth_rate=8):
        super(DenseNet3, self).__init__()
        in_planes = growth_rate
        block = BasicBlock

        # 1st conv before any dense block
        self.conv1 = nn.Conv2d(3, in_planes, kernel_size=5, stride=1,
                               padding=0, bias=False)
        # 1st block
        self.block1 = DenseBlock(n, in_planes, growth_rate, block)
        in_planes = int(in_planes+n*growth_rate)

        ####
        # 2st block ...
        # 3st block ...

    def forward(self, x,y):
        x = self.conv1(x)
        out1, out2 = self.block1(x,y)
        # 2nd, 3rd blocks ...
        return  out1, out2

Error: File “/home/…/densenet.py”, line 33, in forward
return self.layer(x,y) (in class DenseBlock)
File “/home/…/miniconda3/envs/Detector/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 489, in call
result = self.forward(*input, **kwargs)
TypeError: forward() takes 2 positional arguments but 3 were given

1 Like

Generally, the forward() function can be defined to have multiple input arguments. However, I think it is not possible to pass multiple arguments to models wrapped by Sequential() function.

Furthermore, I see that in the following code from the BasicBlock:

the input tensor y is just passed along as output without any changes. So, in fact you would not even need to pass y along with x.

Thanks very much for your reply! The variable ‘y’ in BasicBlock is just an example to better describe my confusion. I was wondering if there is an alternative to solve this problem (passing multiple arguments through a wrapped model). Otherwise, we have to do this in a relatively stupid way.

I see. So, I found a workaround solution, you can put the two inputs in a dictionary. I tried that with a dictionary, but it might be also possible to that with list. Here is a working example:

class BasicBlock(nn.Module):
    def __init__(self, in_planes_1, in_planes_2, out_planes):
        super(BasicBlock, self).__init__()
        self.relu = nn.ReLU(inplace=True)
        self.conv1 = nn.Conv2d(in_planes_1, out_planes, kernel_size=3, stride=1, padding=1, bias=False)
        self.conv2 = nn.Conv2d(in_planes_2, out_planes, kernel_size=3, stride=1, padding=1, bias=False)
    def forward(self, x):
        x1, x2 = x[0], x[1]
        out1 = self.conv1(self.relu(x1))
        out2 = self.conv2(self.relu(x2))
        return {0:torch.cat([x1, out1], 1), 1:torch.cat([x2, out2], 1)}

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.main = nn.Sequential(
            BasicBlock(3, 1, 8),
            BasicBlock(11, 9, 16))
    def forward(self, x):
        return self.main(x)

Note that I have made the input and output consistent using the dictionary.

Now we create a model:

>>> m = MyModel()
>>> m
MyModel(
  (main): Sequential(
    (0): BasicBlock(
      (relu): ReLU(inplace)
      (conv1): Conv2d(3, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (conv2): Conv2d(1, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    )
    (1): BasicBlock(
      (relu): ReLU(inplace)
      (conv1): Conv2d(11, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (conv2): Conv2d(9, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    )
  )
)

and now we pass the two arguments as follows:

>>> t1 = torch.randn(10, 3, 28, 28)
>>> t2 = torch.randn(10, 1, 28, 28)
>>> output = m({0:t1, 1:t2})
>>> output[0].shape, output[1].shape
>>> type(output)
<class 'dict'>
>>> output.keys()
dict_keys([0, 1])
(torch.Size([10, 27, 28, 28]), torch.Size([10, 25, 28, 28]))
5 Likes

Hi, vmirly1,

Cool! Thanks for sharing your ideas!

Cheers,
yclin

Hi,
I don’t know if it is a good way of doing it, but it was working for my simple usage (note that all my models I use in it have *args ,**kwargs in their forward definition to allow other layers to use the additional arguments):

from torch import nn


class CombineModel(nn.Sequential):
    """ Class to combine multiple models. Sequential allowing multiple inputs."""

    def __init__(self, *args):
        super(CombineModel, self).__init__(*args)

    def forward(self, x, *args, **kwargs):
        for i, module in enumerate(self):
            if i == 0:
                x = module(x, *args, **kwargs)
            else:
                x = module(*x, **kwargs)
            if not isinstance(x, tuple) and i != len(self) - 1:
                x = (x,)
        return x

I’d be interested in your opinion, please let me know if I’m doing something completely wrong or unusable.

2 Likes

Hi, turpaultn, thanks very much! Personally I prefer the way vmirly1 proposed by using a dictionary, which seems nicer (because of the keys).