New to PyTorch and struggling with tensor shapes

1

Hi all,

Im new to PyTorch and neural networks and am having some trouble figuring out tensor shaping.

I have a task to make a neural network that can distingush two intertwined rectangular spirals.

Running x.shape on the provided data gives [2592100, 2] which I am passing to a linear layer, so im not sure how to shape my tensors around this. I have used x = x.view(x.size(0), -1) but this doesnt seem to remedy the issue.

I have seen in another thread (ValueError: Expected input batch_size (324) to match target batch_size (4)) where the data has 4 variables so I wanted to know if theres a special way to solve this.
Thanks!

2

My code for reference:

class Network(torch.nn.Module):
    def __init__(self, layer, hid):
        super(Network, self).__init__()
        self.layer = layer
        
        self.short = nn.Sequential(
            nn.Linear(2, hid), # Hidden Layer 1
            nn.Tanh(),
            nn.Linear(hid, 2), # Output Layer
            nn.Sigmoid()
        )
        self.long = nn.Sequential(
            nn.Linear(1, hid), # Hidden Layer 1
            nn.Tanh(),
            nn.Linear(hid, hid), # Hidden Layer 2
            nn.Tanh(),
            nn.Linear(hid, 2), # Output Layer
            nn.Sigmoid()
        )
        

    def forward(self, input):
        x = input
        x = x.view(x.size(0), -1)
        
        if (self.layer == 1):
            output = self.short(x)
            return output
        else:
            output = self.long(x)
            return output
3

Your model works fine for the mentioned input shape, if layer=1 is used:

model = Network(layer=1, hid=16)
x = torch.randn(2592100, 2)
out = model(x)
print(out.shape)
> torch.Size([2592100, 2])

model = Network(layer=0, hid=16)
x = torch.randn(2592100, 1)
out = model(x)
print(out.shape)
> torch.Size([2592100, 2])

The self.long block expects a single input feature, so an input in the shape [batch_size, 1] would also work.
Since your input has only two dimensions, x.view(x.size(0), -1) won’t change the shape and you could remove it.

4

Thanks so much!, so in this case would that be written as,

 self.short = nn.Sequential(
            nn.Linear(2592100*2, hid), # Hidden Layer 1
            nn.Tanh(),

and the same input size for self.long?

5

No, 2592100 is the batch size and the dim1 size defines the input features in your example.
As given in the previous code snippet, your model works fine for an input in the shape [batch_size, 2] assuming the self.short module is used and [batch_size, 1] if the self.long module is used.

6

Im sorry if I’m not getting this, Im still encountering errors when trying to run the code, something along the lines of

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2592100x2 and 2592100x10)

When running self.long when layer = 2 for example, what exactly would I need to put in for

nn.Linear(_____, hid), # Hidden Layer 1

Again sorry if its a stupid question :sweat:

7

It seems you’ve now changed the in_features of the linear layer in self.long to 2592100, which is also wrong.
If your input has a shape of [batch_size, 2], use nn.Linear(2, hid) for the first linear layer in self.short and self.long.

8

That seems to have worked! Cant say how much youve helped me, thank you!