# Weights & Biases

**URL:** <https://discuss.pytorch.org/t/weights-biases/94718>\
**Category:** Uncategorized\
**Created:** [August 31, 2020, 4:44pm UTC](https://discuss.pytorch.org/t/weights-biases/94718 "2020-08-31T16:44:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![chaslie](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/9fc29f/32.png) [@chaslie](https://discuss.pytorch.org/u/chaslie)\
**Post date:** [August 31, 2020, 4:44pm UTC](https://discuss.pytorch.org/t/weights-biases/94718/1 "2020-08-31T16:44:01Z")

</div>

morning,

I am confused. When i store the weights and bias from a convolutional layer using:

```auto
kernels_int_in =conv2d.weight.detach().cpu()
conv2d.bias.detach().cpu()

```

the output shape for weight is [out\_channel,in\_channel,kernal\_height,Kernal\_width] and the size of the bias is a 1D vector of size [out\_channel].

when the weights are plotted using:

```auto
            kernels_t = kernels_int_in - kernels_int_in.min()
            kernels_t = kernels_t / kernels_t.max()
            ind=0
            fig, axarr = plt.subplots(4,32)
            plt.subplots_adjust(wspace=.02, hspace=.02)
            while ind !=128:
                for idx in range(4):
                    for idy in range(32):
                        axarr[idx,idy].imshow(kernels_t[ind,0,:,:].squeeze())
                        # axarr[idx,idy].imshow(act[ind])
                        ind=ind+1

```

the question is, why are the in\_channels stored and what are they telling me? Also over 30 epochs in my case the values of the out\_channels do not change significantly, is this suggesting that the kernels are not appropriate since the overall error stays quite high?

---

<div class="post-metadata">

**Author:** ![ayalaa2](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ayalaa2/32/21195_2.png) [@ayalaa2](https://discuss.pytorch.org/u/ayalaa2)\
**Post date:** [August 31, 2020, 5:06pm UTC](https://discuss.pytorch.org/t/weights-biases/94718/2 "2020-08-31T17:06:37Z")

</div>

> [@chaslie](#):
>
> why are the in\_channels stored and what are they telling me?

Can you expand on this? I’m not sure what you mean by the in\_channels being stored. Are you asking about that dimension in general? I see that in your plot making that you’re always indexing into in\_channels with 0 (so you might not be seeing all the data in your plots).

> [@chaslie](#):
>
> values of the out\_channels do not change significantly

Convolution kernel weights are typically pretty small in the first place. For example, in resnet18, some of their kernels weights, on average, are around 0.002 or even 0.0002 (depending on where you sample). I think it would be more appropriate to take a look at their gradients overtime. That will be a better indicator of seeing how those layers are doing.

---

<div class="post-metadata">

**Author:** ![chaslie](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/9fc29f/32.png) [@chaslie](https://discuss.pytorch.org/u/chaslie)\
**Post date:** [August 31, 2020, 5:23pm UTC](https://discuss.pytorch.org/t/weights-biases/94718/3 "2020-08-31T17:23:34Z")

</div>

what i am trying to plot is the actual kernals that are used in conv2D

---

<div class="post-metadata">

**Author:** ![ayalaa2](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ayalaa2/32/21195_2.png) [@ayalaa2](https://discuss.pytorch.org/u/ayalaa2)\
**Post date:** [August 31, 2020, 5:34pm UTC](https://discuss.pytorch.org/t/weights-biases/94718/4 "2020-08-31T17:34:38Z")

</div>

You need to iterate over your in channels to get all the information about your kernels. Conv2D kernels are 4 dimensional shapes that act upon every single input channel.