# DepthWise1dConv with causal padding. Need help

**URL:** <https://discuss.pytorch.org/t/depthwise1dconv-with-causal-padding-need-help/180172>\
**Category:** Uncategorized\
**Created:** [May 17, 2023, 4:18pm UTC](https://discuss.pytorch.org/t/depthwise1dconv-with-causal-padding-need-help/180172 "2023-05-17T16:18:14Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ergaster](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ergaster/32/56368_2.png) [@ergaster](https://discuss.pytorch.org/u/ergaster)\
**Post date:** [May 17, 2023, 4:18pm UTC](https://discuss.pytorch.org/t/depthwise1dconv-with-causal-padding-need-help/180172/1 "2023-05-17T16:18:14Z")

</div>

# Explanation

I want to implement a DepthWise1dConv with causal padding. That is, padding should be applied before the signal starts.

I followed this wonderful [resource](https://github.com/seungjunlee96/Depthwise-Separable-Convolution_Pytorch) to understand depth-wise convolutions, although explained for the 2D case, I assume it its directly applicable to the 1D case.

From the torch.nn.Conv1d [documentation](https://pytorch.org/docs/stable/generated/torch.nn.Conv1d.html), whenever padding is added, it is added to both sides of the input (and it can be observed in the module variable `_reversed_padding_repeated_twice`. But this is not the desired behavior.

However, whenever I try to set the padding manually by using `torch.nn.functional.pad` I get an error. What am I missing here?

# Minimal reproducible example

```auto
import torch

x = torch.rand(1, 64, 384) # (B, C, L)
conv1d = torch.nn.Conv1d(in_channels=64, out_channels=64, kernel_size=17, padding='same', groups=64)
print(f"{conv1d(x).shape=}, {conv1d._reversed_padding_repeated_twice=}")

p3d = (0, 0, 16, 0, 0, 0)
x = torch.nn.functional.pad(x, p3d)
print(f"{x.shape=}")
conv1d = torch.nn.Conv1d(in_channels=80, out_channels=64, kernel_size=17, padding='valid', groups=80)
conv1d(x)

```

## Output

```auto
conv1d(x).shape=torch.Size([1, 64, 384]), conv1d._reversed_padding_repeated_twice=[8, 8]
x.shape=torch.Size([1, 80, 384])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[98], line 10
      8 x = torch.nn.functional.pad(x, p3d)
      9 print(f"{x.shape=}")
---> 10 conv1d = torch.nn.Conv1d(in_channels=80, out_channels=64, kernel_size=17, padding='valid', groups=80)
     11 conv1d(x)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/conv.py:300, in Conv1d. __init__ (self, in_channels, out_channels, kernel_size, stride, padding, dilation, groups, bias, padding_mode, device, dtype)
    298 padding_ = padding if isinstance(padding, str) else _single(padding)
    299 dilation_ = _single(dilation)
--> 300 super(). __init__ (
    301 in_channels, out_channels, kernel_size_, stride_, padding_, dilation_,
    302 False, _single(0), groups, bias, padding_mode, **factory_kwargs)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/conv.py:92, in _ConvNd. __init__ (self, in_channels, out_channels, kernel_size, stride, padding, dilation, transposed, output_padding, groups, bias, padding_mode, device, dtype)
     90 raise ValueError('in_channels must be divisible by groups')
     91 if out_channels % groups != 0:
---> 92 raise ValueError('out_channels must be divisible by groups')
     93 valid_padding_strings = {'same', 'valid'}
     94 if isinstance(padding, str):

ValueError: out_channels must be divisible by groups

```

* * *

# In tensorflow

```auto
import tensorflow as tf

x = tf.random.uniform((1, 64, 384))
print(f"{x.shape=}")
causal_pad = tf.keras.layers.ZeroPadding1D((16,0))(x)
print(f"{causal_pad.shape=}")
dw_conv = tf.keras.layers.DepthwiseConv1D(
    kernel_size=17,
    strides=1,
    dilation_rate=1,
    padding='valid',
    use_bias=True,
    depthwise_initializer='glorot_uniform')(causal_pad)
print(f"{dw_conv.shape=}")

```

## Output

```auto
x.shape=TensorShape([1, 64, 384])
causal_pad.shape=TensorShape([1, 80, 384])
dw_conv.shape=TensorShape([1, 64, 384])

```

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 17, 2023, 7:17pm UTC](https://discuss.pytorch.org/t/depthwise1dconv-with-causal-padding-need-help/180172/2 "2023-05-17T19:17:57Z")

</div>

I’m not deeply familiar with Keras, but based on the shape I assume you are padding the temporal dimension.  
In this case this code should work:

```python
import torch

x = torch.rand(1, 64, 384) # (B, C, L)
conv1d = torch.nn.Conv1d(in_channels=64, out_channels=64, kernel_size=17, padding='same', groups=64)
print(f"{conv1d(x).shape=}, {conv1d._reversed_padding_repeated_twice=}")

p3d = (16, 0, 0, 0, 0, 0)
x = torch.nn.functional.pad(x, p3d)
print(f"{x.shape=}")
conv1d = torch.nn.Conv1d(in_channels=64, out_channels=64, kernel_size=17, padding='valid', groups=64)
out = conv1d(x)
print(out.shape)
# torch.Size([1, 64, 384])

```
