# Conv2d vs conv3d

**URL:** <https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109>\
**Category:** vision\
**Created:** [December 25, 2020, 4:26am UTC](https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109 "2020-12-25T04:26:48Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohit117](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mohit117/32/13447_2.png) [@mohit117](https://discuss.pytorch.org/u/mohit117)\
**Post date:** [December 25, 2020, 4:26am UTC](https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109/1 "2020-12-25T04:26:48Z")

</div>

Hello,

Suppose I am working with `n` RGB video frames with convolution kernels `k x k`. I can channelwise stack all the frames and use pytorch conv2d with kernel `3n x k x k` or can simply use 3d convolutions with kernels `n x 3 x k x k`. So which should be used for highest accuracy? Theoretically, in both cases, the neural network should find either configuration comfortable as the parameter complexity and receptive field remains the same. Or is it that conv3d will be advantageous?

Thankyou very much.

---

<div class="post-metadata">

**Author:** ![InnovArul](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/innovarul/32/5282_2.png) [@InnovArul](https://discuss.pytorch.org/u/InnovArul)\
**Post date:** [December 25, 2020, 4:43am UTC](https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109/2 "2020-12-25T04:43:43Z")

</div>

I can’t say for sure which is advantageous. It may depend on specific tasks.

Logically, if we talk about video-based tasks, `Conv2d` may not observe temporal context and each frame will be processed independently (except they might interact during Batchnorm).  
`Conv3d` would make the features from each frame interact and may learn temporal contexts. Empirically, `Conv3d` is shown to perform better. You may refer to `resnet3D` or I3D model design.

---

<div class="post-metadata">

**Author:** ![mohit117](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mohit117/32/13447_2.png) [@mohit117](https://discuss.pytorch.org/u/mohit117)\
**Post date:** [December 25, 2020, 5:01am UTC](https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109/3 "2020-12-25T05:01:52Z")

</div>

But as i said the frames are concatenated channel wise and thus the 2D kernel will be looking at all the frames and hence all frames interact as will be the case with conv3D.

---

<div class="post-metadata">

**Author:** ![InnovArul](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/innovarul/32/5282_2.png) [@InnovArul](https://discuss.pytorch.org/u/InnovArul)\
**Post date:** [December 25, 2020, 5:10am UTC](https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109/4 "2020-12-25T05:10:20Z")

</div>

I see. My bad that I didn’t understand it.

There is still a disadvantage with `Conv2d` in cases like below:

Let’s say we have 6 frames with RGB channels.  
With `Conv3d`, we can emulate applying a conv kernel for every 3 frames to learn short-range temporal features. i.e., with `in_channels=3 & kernel_size (3,5,5)` for example. In this way, there is a possibility to learn motion features in a hierarchical way.

With `Conv2d`, I am not sure if we can emulate it. As you said, we can apply `18 x 5 x 5` kernel. But it may not be efficient as learning hierarchically.

---

<div class="post-metadata">

**Author:** ![mohit117](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mohit117/32/13447_2.png) [@mohit117](https://discuss.pytorch.org/u/mohit117)\
**Post date:** [December 25, 2020, 5:12am UTC](https://discuss.pytorch.org/t/conv2d-vs-conv3d/107109/5 "2020-12-25T05:12:46Z")

</div>

Thanks for the understanding.