# Create feature map of pretrained resnet50 layers

**URL:** <https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793>\
**Category:** vision\
**Created:** [June 12, 2019, 9:15pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793 "2019-06-12T21:15:08Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![bramsurewaard](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/a3d4f5/32.png) [@bramsurewaard](https://discuss.pytorch.org/u/bramsurewaard)\
**Post date:** [June 12, 2019, 9:15pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/1 "2019-06-12T21:15:08Z")

</div>

Hey,

I am doing a project and I need to try and visualize what a certain layer of the resnet50 does. When doing research I found a relatively simple way to do say here: [https://machinelearningmastery.com/how-to-visualize-filters-and-feature-maps-in-convolutional-neural-networks/](https://machinelearningmastery.com/how-to-visualize-filters-and-feature-maps-in-convolutional-neural-networks/). But the problem is that I already have a pretrained network that is from PyTorch so the network doesn’t have the same attributres as the keras network. Anybody who can give me some tips on how to create a feature map of an already trained resnet? Thanks!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [June 12, 2019, 9:25pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/2 "2019-06-12T21:25:46Z")

</div>

To visualize the kernels, just get the `weight` parameter of the specific layer by addressing the attribute and try to plot it e.g. using `matplotlib`. E.g. if your model has 3 conv layers defined as `self.conv1 = nn.Conv2d(...)`, you could get the kernels via:

```python
model = MyModel()
kernel = model.conv2.weight.detach()

```

To get activations for a specific input, have a look at [this post](https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/6).

---

<div class="post-metadata">

**Author:** ![bramsurewaard](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/a3d4f5/32.png) [@bramsurewaard](https://discuss.pytorch.org/u/bramsurewaard)\
**Post date:** [June 16, 2019, 3:50pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/3 "2019-06-16T15:50:35Z")

</div>

Thanks for the tip!

But for me it is not clear how to get the weights of the different layers.  
When I use the model.eval() function this is what I get:

```auto
ResNet(
  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace)
  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (layer1): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
      (downsample): Sequential(
        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (2): Bottleneck(
      (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
  )
  (layer2): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
      (downsample): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (2): Bottleneck(
      (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (3): Bottleneck(
      (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
  )
  (layer3): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
      (downsample): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (2): Bottleneck(
      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (3): Bottleneck(
      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (4): Bottleneck(
      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (5): Bottleneck(
      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
  )
  (layer4): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
      (downsample): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): Bottleneck(
      (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
    (2): Bottleneck(
      (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
    )
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
  (fc): Linear(in_features=2048, out_features=1000, bias=True)
)

```

So I thought that when i would maybe do model.layer1.conv2.weight.detach() I would get the weights of the first layer. But this doesn’t work. I only get an error:

> AttributeError: ‘Sequential’ object has no attribute ‘conv2’

Could you maybe give me a tip to get to the feature map of any layer? Thanks!

---

<div class="post-metadata">

**Author:** ![sai\_tharun](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sai_tharun/32/12475_2.png) [@sai\_tharun](https://discuss.pytorch.org/u/sai_tharun)\
**Post date:** [June 16, 2019, 5:42pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/4 "2019-06-16T17:42:19Z")

</div>

It is not clear which conv2 weights you are trying to detach,as the layer1 has 3 bottleneck which has one conv2 each.  
if you want to detach conv2 weights from bottleneck 0,you could do this:

model.layer1[0].conv2.weight.detach()

---

<div class="post-metadata">

**Author:** ![bramsurewaard](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/a3d4f5/32.png) [@bramsurewaard](https://discuss.pytorch.org/u/bramsurewaard)\
**Post date:** [June 16, 2019, 5:53pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/5 "2019-06-16T17:53:02Z")

</div>

That works! Thanks!

Do you maybe know if it is possible to give an image as an input to the network en to then get the weights from each layer? And if so how would I do that?

---

<div class="post-metadata">

**Author:** ![sai\_tharun](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sai_tharun/32/12475_2.png) [@sai\_tharun](https://discuss.pytorch.org/u/sai_tharun)\
**Post date:** [June 16, 2019, 6:16pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/6 "2019-06-16T18:16:25Z")

</div>

```auto
inputs,labels = data
output = model(input)
for j,child in enumerate(model.children()):
    for param in child.parameters():
        print(param[j])

```

parameters will be same, even after you have given input to the model as you are not computing the loss and not doing backpropagation to update the parameters

---

<div class="post-metadata">

**Author:** ![bramsurewaard](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/a3d4f5/32.png) [@bramsurewaard](https://discuss.pytorch.org/u/bramsurewaard)\
**Post date:** [June 16, 2019, 6:25pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/7 "2019-06-16T18:25:58Z")

</div>

Thanks for the fast reply!

Unfortunately my knowledge of python is very low.

Could you maybe be more specific on how I would be able to input 1 image into the network and get the weights so that I can get a feature map?

---

<div class="post-metadata">

**Author:** ![sai\_tharun](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sai_tharun/32/12475_2.png) [@sai\_tharun](https://discuss.pytorch.org/u/sai_tharun)\
**Post date:** [June 16, 2019, 6:53pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/8 "2019-06-16T18:53:32Z")

</div>

loading an image using the image path

```auto
data_path = 'data/train/'
train_dataset = torchvision.datasets.ImageFolder(
      root=data_path,
      transform=torchvision.transforms.ToTensor()
    )

```

get the images into train\_loader using DataLoader, you can get single image by changing the batch\_size to 1, we use shuffle True because the model we train should be robust to avoid overfitting

```auto
train_loader = torch.utils.data.DataLoader(
        train_dataset,
        batch_size=64,
        shuffle=True
    )

```

---

<div class="post-metadata">

**Author:** ![sai\_tharun](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sai_tharun/32/12475_2.png) [@sai\_tharun](https://discuss.pytorch.org/u/sai_tharun)\
**Post date:** [June 16, 2019, 6:58pm UTC](https://discuss.pytorch.org/t/create-feature-map-of-pretrained-resnet50-layers/47793/9 "2019-06-16T18:58:34Z")

</div>

use iter to get only one image at a time

```auto
data = iter(train_loader)

```

next(data) loads the one image and then unpacks into the variables image and labels

```auto
image,labels = next(data)
output = model(images)

```