# Visualize feature map

**URL:** <https://discuss.pytorch.org/t/visualize-feature-map/29597>\
**Category:** vision\
**Created:** [November 14, 2018, 4:33pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597 "2018-11-14T16:33:18Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 20, 2020, 5:15am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/65 "2020-08-20T05:15:24Z")

</div>

I think we had a similar discussion before in [this topic](https://discuss.pytorch.org/t/visualize-feature-map/29597/45) and forward hooks should also work here. 😉

---

<div class="post-metadata">

**Author:** ![little\_adventurer](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/d6d6ee/32.png) [@little\_adventurer](https://discuss.pytorch.org/u/little_adventurer)\
**Post date:** [August 21, 2020, 9:07pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/67 "2020-08-21T21:07:22Z")

</div>

Wouldn’t that be the Class Activation Map?  
The output of a conv2d layer is the set of pixels that ‘activated’ the prediction.

And aren’t feature maps, the ‘kernel’ maps i.e. the filter applied (by each kernel) in order to obtain the CAMs ?

---

<div class="post-metadata">

**Author:** ![little\_adventurer](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/d6d6ee/32.png) [@little\_adventurer](https://discuss.pytorch.org/u/little_adventurer)\
**Post date:** [August 21, 2020, 9:27pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/68 "2020-08-21T21:27:44Z")

</div>

it’s an amazing package, does the job perfectly.

The only issue I had with it (as far as I remember) was the fact that it sort of “unfolds” the models.

E.g. resnet101 has 4 bottleneck layers, which are then “unfolded” and feature maps are extracted for the all of the conv2d layers. Which (if visualising all of them) renders them unreadable (to be fair I was placing them on the tensorboard rather than saving so I guess I shot myself in the foot there)

So one simply has to limit (every N or predefined indices) of layers they desire to visualise.

@Blueberry

> <https://github.com/mbary/feature_map_vis/blob/master/feature_maps_extractor.py>

  
made it a bit easier for myself to work with it.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 21, 2020, 10:47pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/69 "2020-08-21T22:47:08Z")

</div>

> [@little\_adventurer](#):
>
> And aren’t feature maps, the ‘kernel’ maps i.e. the filter applied (by each kernel) in order to obtain the CAMs ?

We might use different terminology, but I wouldn’t call the filters/weights features.  
Anyway, if you want to visualize the filters, you can directly access them via `model.layer.weight` and plot them using e.g. `matplotlib`.

---

<div class="post-metadata">

**Author:** ![little\_adventurer](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/d6d6ee/32.png) [@little\_adventurer](https://discuss.pytorch.org/u/little_adventurer)\
**Post date:** [August 21, 2020, 10:54pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/70 "2020-08-21T22:54:35Z")

</div>

You’re right, I looked it up and read around and feature maps and class activation maps seem to be used interchangeably.

I see it now, feature maps are the outputs of each conv2d layer,

> **[Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://arxiv.org/abs/1910.01279)**
>
> Recently, increasing attention has been drawn to the internal mechanisms of
> convolutional neural networks, and the reason why the network makes specific
> decisions. In this paper, we develop a novel post-hoc visual explanation method
> called Score-CAM...

  
is a combination of all these maps and weights!  
Hence the Score.

I got it slightly confused.  
Thanks for the answer!

---

<div class="post-metadata">

**Author:** ![Lewis\_Morris](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lewis_morris/32/22979_2.png) [@Lewis\_Morris](https://discuss.pytorch.org/u/Lewis_Morris)\
**Post date:** [August 22, 2020, 8:30am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/71 "2020-08-22T08:30:14Z")

</div>

@Blueberry @little_adventurer

I’ve just pushed a change to make this easier.

You can now specify the layer name (you could before but its now working better)

And also the recursion depth (how far into nested sequential blocks it will go)

I.e

`fe.set_image("pug.jpg" , allowed_modules = ["bottleneck"])`

will only output the bottleneck layers (in the pretraied pytorch model there are actually 30 odd)

or

`fe.set_image("pug.jpg", allowed_depth = 0)`

will only output these layers

`['Conv2d', 'BatchNorm2d', 'ReLU', 'MaxPool2d', 'Sequential (Block)', 'Sequential (Block)', 'Sequential (Block)', 'Sequential (Block)']`

You could use both parameters together and do this.

`fe.set_image("pug.jpg", allowed_modules=["conv"], allowed_depth = 2)`

Hope this helps you somewhat.

---

<div class="post-metadata">

**Author:** ![Pak\_Lau](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pak_lau/32/28099_2.png) [@Pak\_Lau](https://discuss.pytorch.org/u/Pak_Lau)\
**Post date:** [August 24, 2020, 10:17am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/72 "2020-08-24T10:17:05Z")

</div>

Hello,  
Has anyone done this (e.g. hook) with C++ and Libtorch?

> [@Visualize feature map](https://discuss.pytorch.org/t/visualize-feature-map/29597/2):
>
> You can just use a plot library like matplotlib to visualize the output. Sure! You could use some loss function like nn.BCELoss as your criterion to reconstruct the images. Forward hooks are a good choice to get the activation map for a certain input. Here is a small code example as a starter: import torch import torch.nn as nn import torch.optim as optim import torch.nn.functional as F from torch.utils.data import DataLoader import torchvision.transforms as transforms import torchv…

Thanks

---

<div class="post-metadata">

**Author:** ![maklachur](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/82dd89/32.png) [@maklachur](https://discuss.pytorch.org/u/maklachur)\
**Post date:** [October 7, 2020, 7:12am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/73 "2020-10-07T07:12:05Z")

</div>

Hope this snippet will help you to plot 16 outputs in the subplot:

```auto
fig, axarr = plt.subplots(4, 4)
k = 0
for idx in range(act.size(0)//4):
    for idy in range(act.size(0)//4):
        axarr[idx, idy].imshow(act[k])
        k += 1

```

For example, if we consider @ptrblck code snippet and change the conv2 layer as 16 feature maps for the visualization, the output could look like:

![feature_map](https://discuss.pytorch.org/uploads/default/original/3X/a/9/a97c1327c3ce64423bf6646a1959e655838a4a3c.png)

---

<div class="post-metadata">

**Author:** ![Hdk](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ec9cab/32.png) [@Hdk](https://discuss.pytorch.org/u/Hdk)\
**Post date:** [October 7, 2020, 8:44am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/74 "2020-10-07T08:44:59Z")

</div>

Hi, I have one doubt in addition to this.  
While calculating the loss i.e. nn.CrossEntopyLoss(output,labels) in this, we use output of last layer or final output of the network.

But I want to use feature maps of each convolutional layer in loss calculation as well in addition to above calculation.  
Can you please throw some light, how can it be done?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 7, 2020, 9:07am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/75 "2020-10-07T09:07:29Z")

</div>

It depends which target you would like to use for the intermediate activations.  
Since they are conv outputs you won’t be able to use e.g. `nn.CrossEntropyLoss` directly since these outputs don’t represent logits for the classification use case as they have a different shape.  
However, you could follow a similar approach as seen in Inception models, which create auxiliary outputs forked from intermediate outputs and use these aux. output layers to calculate the loss.

---

<div class="post-metadata">

**Author:** ![Hdk](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ec9cab/32.png) [@Hdk](https://discuss.pytorch.org/u/Hdk)\
**Post date:** [October 7, 2020, 1:22pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/76 "2020-10-07T13:22:10Z")

</div>

Actually I want to use discriminative loss in addition to cross entropy loss. The discriminative loss will be based on feature maps of each convolutional layers and cross entropy loss remains as usual. But I didn’t find any source so far, how to do it!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 7, 2020, 9:48pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/77 "2020-10-07T21:48:24Z")

</div>

You could return the intermediate activations in the `forward` as seen here:

```python
def forward(self, x):
    x1 = self.layer1(x)
    x2 = self.layer2(x1)
    out = self.layer3(x2)
    return out, x1, x2

```

and then calculate the losses separately:

```python
out, x1, x2 = model(input)
loss = criterion(out, target) + criterion1(x1, target1) + criterion2(x2, target2)
loss.backward()

```

or alternatively you could use forward hooks to store the intermediate activations and calculate the loss using them. [This post](https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/6) gives you an example on how to use forward hooks.

---

<div class="post-metadata">

**Author:** ![Hdk](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ec9cab/32.png) [@Hdk](https://discuss.pytorch.org/u/Hdk)\
**Post date:** [October 10, 2020, 11:54am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/78 "2020-10-10T11:54:28Z")

</div>

Thank you for the reply @ptrblck.

I actually created a new thread because the discussion deviates slightly and I have few more doubts. Can you please look at [this](https://discuss.pytorch.org/t/feature-maps-in-cnn-and-discriminative-loss/98847)?

---

<div class="post-metadata">

**Author:** ![Hdk](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ec9cab/32.png) [@Hdk](https://discuss.pytorch.org/u/Hdk)\
**Post date:** [October 11, 2020, 10:17pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/79 "2020-10-11T22:17:13Z")

</div>

If I do something like

Loss = my\_Loss (x1, original\_images)

for every layer the size of tensor will be different, thus it will give an error.  
i.e. after 1 convolutional layer, x1 has a size of [batch\_size, num\_features, h,w] = [50, 12, 28, 28]  
While that of original\_images = [50, 3, 32, 32].  
So, how to calculate loss then!

And one my more doubt is, can we do something to get all features (num\_features=12 in this case) to be as distinct as possible from one another?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 12, 2020, 12:34am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/80 "2020-10-12T00:34:29Z")

</div>

You could use additional conv/pooling/interpolation layers to create the same output size of the intermediate activations as the target (or input) tensor.  
Alternatively you could also change the model architecture such that the spatial size won’t be changed, but again it depends on your use case.

> [@Hdk](#):
>
> And one my more doubt is, can we do something to get all features (num\_features=12 in this case) to be as distinct as possible from one another?

I don’t know which approach would create activations meeting this requirement.

---

<div class="post-metadata">

**Author:** ![Hdk](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ec9cab/32.png) [@Hdk](https://discuss.pytorch.org/u/Hdk)\
**Post date:** [October 12, 2020, 9:28am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/81 "2020-10-12T09:28:22Z")

</div>

As far as we provide some kernel size, there will be a reduction in the size.

Size=( (W-K+2P)/ S )+1

I really don’t get your point.  
Can you please give dummy examples of both the scenarios you suggested?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 12, 2020, 8:23pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/82 "2020-10-12T20:23:15Z")

</div>

Yes, you cannot simply calculate a loss between arbitrarily sized tensors, so you would need to come up with a way to calculate the loss between intermediate activations (with different sizes) and a target.

To get the same spatial size you could e.g. use pooling/conv/interpolations while you would still have to make sure to create the same number of channels.  
This can be done via convs again (or via reductions), but it depends on your use case.

Here is a small example:

```python
act = torch.randn(1, 64, 24, 24)
target = torch.randn(1, 3, 28, 28)

# make sure same spatial size
act_interp = F.interpolate(act, (28, 28))

# create same channels
conv = nn.Conv2d(64, 3, 1)
act_interp_channels = conv(act_interp)

print(act_interp_channels.shape) # has same shape now
loss = F.mse_loss(act_interp_channels, target)

```

---

<div class="post-metadata">

**Author:** ![Hdk](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ec9cab/32.png) [@Hdk](https://discuss.pytorch.org/u/Hdk)\
**Post date:** [October 12, 2020, 8:57pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/83 "2020-10-12T20:57:40Z")

</div>

Thanks for the reply.  
This is something new, I will try it.

So, can we say like pooling and interpolate corresponds to downscaling and upscaling?

And in another approach, I have a output tensor(still intermediate activations after each convolutional layer) of say size (batch\_size= 100, channels= 32 , 1, 1) and target(which are labels now) of size (batch\_size= 100, 1) provided we have no. of classes = 10.

So, for loss calculation i.e. Loss = lossFunc(output, labels), Is there any loss function, which can reduce the dimensionality from 32 to 10 and then calculate the loss?  
Or I have to use a conv/linear layer for that and then calculate the loss? Because in this case it adds some more weights, which I want to avoid.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 13, 2020, 6:23am UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/84 "2020-10-13T06:23:29Z")

</div>

> [@Hdk](#):
>
> So, can we say like pooling and interpolate corresponds to downscaling and upscaling?

Yes, you could name it like that. Note that you could down- and upscale using an interpolation method.

> [@Hdk](#):
>
> and target(which are labels now) of size (batch\_size= 100, 1) provided we have no. of classes = 10.

I assume the shape should be `[100, 10]`?  
If so you could either use a trainable layer (as you’ve already described) or try to reduce the channels using e.g. `mean`, `sum` etc.  
However, a reduction might be a bit tricky in this use case, as the target size of 10 doesn’t fit nicely into the channel size of 32.  
Generally, all operations would be allowed which map 32 values to 10.

---

<div class="post-metadata">

**Author:** ![maxiuw](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/maxiuw/32/30259_2.png) [@maxiuw](https://discuss.pytorch.org/u/maxiuw)\
**Post date:** [October 25, 2020, 8:21pm UTC](https://discuss.pytorch.org/t/visualize-feature-map/29597/86 "2020-10-25T20:21:35Z")

</div>

ok that looks super cool 😃 can you give some tips, code, steps of how you did it :)?

[Previous page](https://discuss.pytorch.org/t/visualize-feature-map/29597.md?page=3)

[Next page](https://discuss.pytorch.org/t/visualize-feature-map/29597.md?page=5)