# Extracting speccific layers' output from a module list

**URL:** <https://discuss.pytorch.org/t/extracting-speccific-layers-output-from-a-module-list/158795>\
**Category:** Uncategorized\
**Created:** [August 10, 2022, 8:34am UTC](https://discuss.pytorch.org/t/extracting-speccific-layers-output-from-a-module-list/158795 "2022-08-10T08:34:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tm4](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/b38774/32.png) [@Tm4](https://discuss.pytorch.org/u/Tm4)\
**Post date:** [August 10, 2022, 8:34am UTC](https://discuss.pytorch.org/t/extracting-speccific-layers-output-from-a-module-list/158795/1 "2022-08-10T08:34:48Z")

</div>

I’m using a [pretrained model](https://github.com/facebookresearch/TimeSformer/blob/main/timesformer/models/vit.py#L291) in which there are several self\_attentions sequentially stacked each one after another and the number of them is 12. I need to extract the output of the fourth and 10th blocks of this sequential layers. In the following script, the `BLock` represents each self-attention layer:

```
dpr = [x.item() for x in torch.linspace(0, 0.1, 12)] # stochastic depth decay rule
    
    self.blocks = nn.ModuleList([
        Block(
            dim=embed_dim, num_heads=num_heads, mlp_ratio=mlp_ratio, qkv_bias=qkv_bias, qk_scale=qk_scale,
            drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[i], norm_layer=norm_layer, attention_type=self.attention_type)
        for i in range(12)])

```

The self-attention layers (stack of `Block`) are as follows:

```
## Attention blocks
    for blk in self.blocks:
        x = blk(x, B, T, W)

```

How can I extract the fourth and the 10th layers’ output?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 10, 2022, 6:54pm UTC](https://discuss.pytorch.org/t/extracting-speccific-layers-output-from-a-module-list/158795/2 "2022-08-10T18:54:59Z")

</div>

You can use forward hooks as described [here](https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/6) and register them to the 4th and 10th layers.

---

<div class="post-metadata">

**Author:** ![Tm4](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/b38774/32.png) [@Tm4](https://discuss.pytorch.org/u/Tm4)\
**Post date:** [August 12, 2022, 9:21pm UTC](https://discuss.pytorch.org/t/extracting-speccific-layers-output-from-a-module-list/158795/3 "2022-08-12T21:21:11Z")

</div>

@ptrblck ,Thanks a lot .