Did Conv2D shapes change between torch 1.4.0 and 1.6.0?

1

I have some simple code that’s behaving drastically differently with torch 1.4.0 and 1.6.0: Conv2D filters are producing different shaped outputs. I can’t see anything in the release notes that would suggest that 2D convolution layers should be constructed differently between those torch releases. The following simple (and possibly buggy) code reproduces the issue:

#!/usr/bin/env python3
import torch
from torch import nn

c = 'circular'
class EmbeddingNetwork(nn.Module):
    def __init__(self):
        super().__init__()
        ksize = 5
        self.embedding = nn.Sequential(
            nn.Conv2d(10, 10, ksize, stride=1, padding=ksize-1, padding_mode=c),
            nn.ReLU())

    def forward(self, x):
        print("x.shape:", x.shape)
        print("embedding model:", self.embedding)
        result =  self.embedding(x)
        print("embedding(x).shape:", result.shape)
        return result

x = torch.zeros(1, 10, 26, 26)
embedding = EmbeddingNetwork()
embedding(x)

Running this with PyTorch 1.6.0, I get:

x.shape: torch.Size([1, 10, 26, 26])
embedding model: Sequential(
  (0): Conv2d(10, 10, kernel_size=(5, 5), stride=(1, 1), padding=(4, 4), padding_mode=circular)
  (1): ReLU()
)
embedding(x).shape: torch.Size([1, 10, 30, 30])

Running with PyTorch 1.4.0, I get:

x.shape: torch.Size([1, 10, 26, 26])
embedding model: Sequential(
  (0): Conv2d(10, 10, kernel_size=(5, 5), stride=(1, 1), padding=(4, 4), padding_mode=circular)
  (1): ReLU()
)
embedding(x).shape: torch.Size([1, 10, 26, 26])
2

I cannot install 1.6.0 currently to reproduce this issue due to the conda server issue, but at least in 1.7.0.dev20200728, the output shape is [1, 10, 30, 30]. I’ll try to rerun it tomorrow.

3

Your 1.7.0dev result is as expected (torch 1.5.0 and above produce [1, 10, 30, 30]).

The following works for me for installing 1.4.0:

virtualenv -p python3 venv/
source venv/bin/activate
pip install torch==1.4.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
python /path/to/repro.py
4

just ran it with PyTorch 1.6.0
my output is:

x.shape: torch.Size([1, 10, 26, 26])
embedding model: Sequential(
  (0): Conv2d(10, 10, kernel_size=(5, 5), stride=(1, 1), padding=(4, 4), padding_mode=circular)
  (1): ReLU()
)
embedding(x).shape: torch.Size([1, 10, 30, 30])
1 Like
5

Ooops, my original post was backwards. I see output shape [1, 10, 26, 26] from torch 1.4.0 and [1, 10, 30, 30] from 1.6.0. Sorry for the confusion! Will edit the original post now.

6

Ahh ok, that changes things.
Mhhh seems like you are right.
On my pytorch 1.4.0 installation the above code also outputs:

x.shape: torch.Size([1, 10, 26, 26])
embedding model: Sequential(
  (0): Conv2d(10, 10, kernel_size=(5, 5), stride=(1, 1), padding=(4, 4), padding_mode=circular)
  (1): ReLU()
)
embedding(x).shape: torch.Size([1, 10, 26, 26])
7

There were a couple of issues (such as this one) for older PyTorch versions with circular padding. Good to hear it’s working properly with the latest version. :slight_smile: