Automatic inputs calculation like in tensorflow

1

I just wanted to try PyTorch after several years of TensorFlow.

  1. But I wonder, why do I need to define layers in init and use them in forward?
    In TensorFlow, for example, I just build graph in one place, and then use it all over the code, like so:
        x = Conv2D(filters=16, kernel_size=(7, 7), strides=(1, 1), padding="valid")(x)
        x = ReLU()(x)
        x = MaxPooling2D((2, 2))(x)

        x = Conv2D(filters=32, kernel_size=(5, 5), strides=(1, 1), padding="valid")(x)
        x = ReLU()(x)
        x = MaxPooling2D((2, 2))(x)

        x = Flatten()(x)
        x = Dense(100)(x)
        x = ReLU()(x)

        x = Dense(self._num_classes)(x)

        return x
  1. It seems not so attractive for me to calculate inputs and outputs dims for each layer.
    Moreover, If I want to change the input dataset dim, I have to manually recalculate inputs outputs of layers (Linear at least).

So, is it possible to define models like in TensorFlow with automatically recalculation dims of inputs and outputs?

Thanks for attention.

2

For 1. you could use Sequential.
For 2. no. And that is because PyTorch won’t ask you to specify your input size ahead of time. This does come up on the forums every now and then, but for most people it is OK to compute the (usually just one in a typical CNN) size or just put data through the lower layers and then read the shape.

Best regards

Thomas

1 Like
3

BTW:
I found following approach very suitable

import torch
from torch import nn


class AlexNet(nn.Module):
    def __init__(self, dims: tuple, num_classes: int):
        super().__init__()
        batch_size, chanels, w, h = dims
        data = torch.ones([batch_size, chanels, w, h])
        self.conv_net = nn.Sequential(nn.Conv2d(in_channels=3, out_channels=96, kernel_size=(11, 11), stride=4),
                                      nn.ReLU(),
                                      nn.MaxPool2d(kernel_size=(3, 3), stride=2),

                                      nn.Conv2d(in_channels=96, out_channels=256, kernel_size=(5, 5), stride=1,
                                                padding=2),
                                      nn.ReLU(),
                                      nn.MaxPool2d(kernel_size=(3, 3), stride=2),

                                      nn.Conv2d(in_channels=256, out_channels=384, kernel_size=(3, 3), stride=1,
                                                padding=1),
                                      nn.ReLU(),

                                      nn.Conv2d(in_channels=384, out_channels=384, kernel_size=(3, 3), stride=1,
                                                padding=1),
                                      nn.ReLU(),

                                      nn.Conv2d(in_channels=384, out_channels=256, kernel_size=(3, 3), stride=1),
                                      nn.MaxPool2d(kernel_size=(3, 3), stride=2),

                                      nn.Dropout())
        batch_size = data.shape[0]
        out_shape = self.conv_net(data).view(batch_size, -1).shape[-1]

        fc_neurons = 4096
        self.fc = nn.Sequential(nn.Linear(in_features=out_shape, out_features=fc_neurons),
                                nn.ReLU(),
                                nn.Dropout(),
                                nn.Linear(in_features=fc_neurons, out_features=fc_neurons),
                                nn.ReLU(),
                                nn.Linear(in_features=fc_neurons, out_features=num_classes),
                                nn.Dropout()
                                )

        del data

    def forward(self, input):
        x = self.conv_net(input)
        x = x.view((x.shape[0], -1))
        x = self.fc(x)
        return x


if __name__ == '__main__':
    batch_size = 32
    chanels = 3
    w = 1024
    h = 1024
    num_classes = 10
    data = torch.ones([batch_size, chanels, w, h])
    model = AlexNet((batch_size, chanels, w, h), 10).forward(data)
    pass
2 Likes