nn.Sequential() of the children of a model has different output than that model

1

I am loading a Resnet18 architecture from torchvision.models. To do some testing, just with the original torchvision.model.resnet18(), we get this:

resnet18 = torchvision.models.resnet18()
resnet18(
    torch.tensor(np.random.uniform(-42, 42, (16, 3, 128, 128))).float()
)

With the output
torch.Size([16, 1000]).

However, when I do the following:

resnet18_ls = nn.Sequential(
    *list(resnet18.children())
)
resnet18_ls(
    torch.tensor(np.random.uniform(-42, 42, (16, 3, 128, 128))).float()   
)

I get the runtime error below:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-73-2262642ca1ec> in <module>()
      3 )
      4 resnet18_ls(
----> 5     torch.tensor(np.random.uniform(-42, 42, (16, 3, 128, 128))).float()
      6 )

4 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in linear(input, weight, bias)
   1846     if has_torch_function_variadic(input, weight, bias):
   1847         return handle_torch_function(linear, (input, weight, bias), input, weight, bias=bias)
-> 1848     return torch._C._nn.linear(input, weight, bias)
   1849 
   1850 

RuntimeError: mat1 and mat2 shapes cannot be multiplied (8192x1 and 512x1000)

But aren’t these the same network? I understand that the 8192 comes from the last layer output of 512 multiplied by the batch size of 16, but I don’t see how that makes the matrix shape 8192 x 1.

1 Like
2

No, since the original model uses a custom forward method seen here while your nn.Sequential container will call all initialized modules in a sequential way and will thus miss the functional API calls, e.g. the x = torch.flatten(x, 1) line of code.

1 Like
3

Thanks! I actually just saw it looking it up. Is there a reason why it skips over those calls?

4

Yes, because these functional calls are not modules and are thus not returned by model.children().
PyTorch allows you to define the forward pass as you wish including loops, conditions, functional API calls etc.
nn.Sequential containers are used for simple models where the execution is strictly sequential and between nn.Modules.

1 Like