# Same padding equivalent in Pytorch

**URL:** <https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121>\
**Category:** Uncategorized\
**Created:** [June 12, 2020, 12:20am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121 "2020-06-12T00:20:59Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![garthtrickett](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/garthtrickett/32/25419_2.png) [@garthtrickett](https://discuss.pytorch.org/u/garthtrickett)\
**Post date:** [June 12, 2020, 12:20am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/1 "2020-06-12T00:20:59Z")

</div>

I have a layer with an input of

```auto
torch.Size([64, 32, 100, 20])

```

In Keras I was using this

```auto
conv_first1 = Conv2D(32, (4, 1), padding="same")(conv_first1)

```

which lead to an output shape the same as an the input shape

If I use the below in pytorch I end up with a shape of 64,32,99,20

```auto
self.conv2 = nn.Conv2d(32, 32, (4, 1), padding=(1,0))

```

and If I instead use padding (2,0) it becomes 64,32,101,20

What should be used in order to end up with  
input\_shape == output\_shape  
64,32,100,20 = 64,32,100,20

---

<div class="post-metadata">

**Author:** ![Nikronic](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/32/71873_2.png) [@Nikronic](https://discuss.pytorch.org/u/Nikronic)\
**Post date:** [June 12, 2020, 1:10am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/2 "2020-06-12T01:10:53Z")

</div>

Hi,

PyTorch does not support `same` padding the way Keras does, but still you can manage it easily using explicit padding before passing the tensor to convolution layer. Here, symmetric padding is not possible so by padding only one side, in your case, top bottom of tensor, we can achieve `same` padding.

```auto
import torch
import torch.nn.functional as F

x = torch.randn(64, 32, 100, 20)
x = F.pad(x, (0, 0, 2, 1)) # [left, right, top, bot]
nn.Conv2d(32, 32, (4, 1))(x).shape

```

Please see these post about calculations [Converting tensorflow model to pytorch: issue with padding](https://discuss.pytorch.org/t/converting-tensorflow-model-to-pytorch-issue-with-padding/84224)

Bests

---

<div class="post-metadata">

**Author:** ![garthtrickett](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/garthtrickett/32/25419_2.png) [@garthtrickett](https://discuss.pytorch.org/u/garthtrickett)\
**Post date:** [June 15, 2020, 12:28am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/3 "2020-06-15T00:28:54Z")

</div>

> [@Nikronic](#):
>
> `x = F.pad(x, (0, 0, 2, 1)) # [left, right, top, bot]`

Is there some way to know whether the equivalent to keras is  
x = F.pad(x, (0, 0, 2, 1)) # [left, right, top, bot] or  
x = F.pad(x, (0, 0, 1, 2)) # [left, right, top, bot]

---

<div class="post-metadata">

**Author:** ![Nikronic](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/32/71873_2.png) [@Nikronic](https://discuss.pytorch.org/u/Nikronic)\
**Post date:** [June 15, 2020, 12:58am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/4 "2020-06-15T00:58:46Z")

</div>

Based on this [issue](https://github.com/tensorflow/tensorflow/issues/17654) (unfortunately links to docs are broken due to TF2 update), it seems it uses second approach.

```auto
x = F.pad(x, (0, 0, 1, 2)) # [left, right, top, bot]

```

code for TF `same` padding:

```auto
in_height, in_width = 100,20
filter_height, filter_width = 4,1
strides=(None,1,1)
out_height = np.ceil(float(in_height) / float(strides[1]))
out_width = np.ceil(float(in_width) / float(strides[2]))

#The total padding applied along the height and width is computed as:

if (in_height % strides[1] == 0):
  pad_along_height = max(filter_height - strides[1], 0)
else:
  pad_along_height = max(filter_height - (in_height % strides[1]), 0)
if (in_width % strides[2] == 0):
  pad_along_width = max(filter_width - strides[2], 0)
else:
  pad_along_width = max(filter_width - (in_width % strides[2]), 0)

print(pad_along_height, pad_along_width)
  
#Finally, the padding on the top, bottom, left and right are:

pad_top = pad_along_height // 2
pad_bottom = pad_along_height - pad_top
pad_left = pad_along_width // 2
pad_right = pad_along_width - pad_left

print(pad_left, pad_right, pad_top, pad_bottom)

```

---

<div class="post-metadata">

**Author:** ![skynet10100](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/ea5d25/32.png) [@skynet10100](https://discuss.pytorch.org/u/skynet10100)\
**Post date:** [January 4, 2021, 9:14pm UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/5 "2021-01-04T21:14:58Z")

</div>

@Nikronic saved my day. Thanks!

---

<div class="post-metadata">

**Author:** ![yuri](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/71c47a/32.png) [@yuri](https://discuss.pytorch.org/u/yuri)\
**Post date:** [March 28, 2021, 8:18am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/6 "2021-03-28T08:18:28Z")

</div>

it is compulsory to do padding before the conv layer or can we also feed the appropriate padding value to the conv layer  
thx

---

<div class="post-metadata">

**Author:** ![sumanmichael](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sumanmichael/32/39042_2.png) [@sumanmichael](https://discuss.pytorch.org/u/sumanmichael)\
**Post date:** [June 3, 2021, 1:16pm UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/7 "2021-06-03T13:16:11Z")

</div>

It worked for me…!

```auto
class Conv2dSamePadding(nn.Conv2d):
    def __init__ (self,*args,**kwargs):
        super(Conv2DTF, self). __init__ (*args, **kwargs)
        self.zero_pad_2d = nn.ZeroPad2d(functools.reduce(operator. __add__ ,
                  [(k // 2 + (k - 2 * (k // 2)) - 1, k // 2) for k in self.kernel_size[::-1]]))

    def forward(self, input):
        return self._conv_forward(self.zero_pad_2d(input), self.weight, self.bias)

```

Gist: [PyTorch Conv2d equivalent of Tensorflow tf.nn.conv2d(....,padding='SAME') · GitHub](https://gist.github.com/sumanmichael/4de9dee93f972d47c80c4ade8e149ea6)

---

<div class="post-metadata">

**Author:** ![yuri](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/71c47a/32.png) [@yuri](https://discuss.pytorch.org/u/yuri)\
**Post date:** [October 22, 2021, 11:25am UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/8 "2021-10-22T11:25:36Z")

</div>

this was solved since Pytorch 1.10.0  
“same” keyword is accepted as input for padding for conv2d

---

<div class="post-metadata">

**Author:** ![addisonklinke](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/addisonklinke/32/79137_2.png) [@addisonklinke](https://discuss.pytorch.org/u/addisonklinke)\
**Post date:** [November 19, 2021, 2:48pm UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/9 "2021-11-19T14:48:59Z")

</div>

@yuri One detail I’ve noticed is `padding='same'` does not export from PyTorch 1.10.0 to CoreML 5.1.0. For others interested, I came up with the following solution

```python
import collections
from itertools import repeat
import torch
from torch import nn
import torch.nn.functional as F

def _ntuple(n):
    """Copy from PyTorch since internal function is not importable

    See ``nn/modules/utils.py:6``
    """
    def parse(x):
        if isinstance(x, collections.abc.Iterable):
            return tuple(x)
        return tuple(repeat(x, n))

    return parse

_pair = _ntuple(2)

class Conv2dSame(nn.Module):
    """Manual convolution with same padding

    Although PyTorch >= 1.10.0 supports ``padding='same'`` as a keyword
    argument, this does not export to CoreML as of coremltools 5.1.0, 
    so we need to implement the internal torch logic manually. 

    Currently the ``RuntimeError`` is
    
    "PyTorch convert function for op '_convolution_mode' not implemented"
    """

    def __init__ (
            self,
            in_channels, 
            out_channels, 
            kernel_size,
            stride=1,
            dilation=1,
            **kwargs):
        """Wrap base convolution layer

        See official PyTorch documentation for parameter details
        https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html
        """
        super(). __init__ ()
        self.conv = nn.Conv2d(
            in_channels=in_channels,
            out_channels=out_channels,
            kernel_size=kernel_size,
            stride=stride,
            dilation=dilation,
            **kwargs)

        # Setup internal representations
        kernel_size_ = _pair(kernel_size)
        dilation_ = _pair(dilation)
        self._reversed_padding_repeated_twice = [0, 0]*len(kernel_size_)

        # Follow the logic from ``nn/modules/conv.py:_ConvNd``
        for d, k, i in zip(dilation_, kernel_size_, 
                                range(len(kernel_size_) - 1, -1, -1)):
            total_padding = d * (k - 1)
            left_pad = total_padding // 2
            self._reversed_padding_repeated_twice[2 * i] = left_pad
            self._reversed_padding_repeated_twice[2 * i + 1] = (
                    total_padding - left_pad)

    def forward(self, imgs):
        """Setup padding so same spatial dimensions are returned

        All shapes (input/output) are ``(N, C, W, H)`` convention

        :param torch.Tensor imgs:
        :return torch.Tensor:
        """
        padded = F.pad(imgs, self._reversed_padding_repeated_twice)
        return self.conv(padded)

```

---

<div class="post-metadata">

**Author:** ![Vishak\_Raj](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vishak_raj/32/29935_2.png) [@Vishak\_Raj](https://discuss.pytorch.org/u/Vishak_Raj)\
**Post date:** [April 27, 2022, 3:27pm UTC](https://discuss.pytorch.org/t/same-padding-equivalent-in-pytorch/85121/10 "2022-04-27T15:27:52Z")

</div>

I tried different methods for creating ‘same’ padding from basic, with same architecture and same data set with same pre-processing, this method works like as ‘same’ padding.

and, ‘same’ padding in cnn makes issue when converting from .pth to .onnx format, using this method, will solve that issue

for reference -

> <https://gist.github.com/sumanmichael/4de9dee93f972d47c80c4ade8e149ea6>