# Getting Intermediate Output of Self Created Sequential **URL:** \ **Category:** vision\ **Created:** [July 24, 2018, 7:52pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662 "2018-07-24T19:52:14Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tylerv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@tylerv](https://discuss.pytorch.org/u/tylerv)\ **Post date:** [July 24, 2018, 7:52pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/1 "2018-07-24T19:52:14Z")
I created my own sequential that has many modules in it. I was able to create a module PrintLayer and I placed it after the output i want to save. class PrintLayer(nn.Module): def **init** (self): super(PrintLayer, self). **init** () ``` def forward(self, x): print(x.shape) return x ``` It could print the shape and value of the output of a specific layer i want to observe during the forward pass, but how exactly do I save that value so I could retrieve it after the forward pass. Thanks! ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [July 24, 2018, 9:07pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/2 "2018-07-24T21:07:20Z")
You could store the activations in a `dict` using forward hooks. I’ve created a small example in [this thread](https://discuss.pytorch.org/t/how-can-l-load-my-best-model-as-a-feature-extractor-evaluator/17254/6?u=ptrblck). ---
**Author:** ![tylerv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@tylerv](https://discuss.pytorch.org/u/tylerv)\ **Post date:** [July 29, 2018, 10:32pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/3 "2018-07-29T22:32:06Z")
The problem is my model is embedded in a very complicated way like this. ![49%20PM](https://discuss.pytorch.org/uploads/default/original/2X/6/69abbf6b1802bc1ac067ac3066c99c9b09c0af38.png) ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [July 29, 2018, 10:47pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/4 "2018-07-29T22:47:38Z")
Would it be feasible to set the hook in the ` __init__ ` of some sub-module, e.g. `UnetSkipConnectionBlock`? Otherwise, you would need to address your layers from top to bottom. ---
**Author:** ![tylerv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@tylerv](https://discuss.pytorch.org/u/tylerv)\ **Post date:** [July 29, 2018, 11:09pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/5 "2018-07-29T23:09:30Z")
Yeah I can add stuff to the submodule, but I’m unsure how to add the hook there since each submodule is a Sequential and isn’t created like the example you have. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [July 29, 2018, 11:26pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/6 "2018-07-29T23:26:36Z")
You would have to index the modules like in this example: ```python model = nn.Sequential(nn.Sequential(nn.Sequential(nn.Linear(10, 2)))) model[0][0][0].register_forward_hook ``` ---
**Author:** ![tylerv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@tylerv](https://discuss.pytorch.org/u/tylerv)\ **Post date:** [July 29, 2018, 11:36pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/7 "2018-07-29T23:36:26Z")
Thanks! And then you would pass it a function like get\_activation that creates a hook and saves the value in a dict? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [July 29, 2018, 11:46pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/8 "2018-07-29T23:46:47Z")
Yes, then I would use the method I’ve linked in the other post. Note that you might want to remove the `.detach` in `hook`, if you want to backpropagate. ---
**Author:** ![tylerv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@tylerv](https://discuss.pytorch.org/u/tylerv)\ **Post date:** [July 31, 2018, 5:51pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/9 "2018-07-31T17:51:05Z")
I’m still having trouble accessing the modules. When i print out the children print(list(model.netG.children())), I get ![37%20PM](https://discuss.pytorch.org/uploads/default/original/2X/8/8dbaa972996acd512a5e9e31753cc49fc06e319a.png), and then when I print model.netG.module( I had to use .module to get rid of dataparallel error), I get ![16%20PM](https://discuss.pytorch.org/uploads/default/original/2X/b/b6bcff98283f7f6f6d0421e3e447178661433005.png). Now when I try to index that, it says UnetGenerator doesn’t support indexing. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [July 31, 2018, 6:07pm UTC](https://discuss.pytorch.org/t/getting-intermediate-output-of-self-created-sequential/21662/10 "2018-07-31T18:07:01Z")
It seems some sub-modules have the name “model”. Have a look at this small example, how to index the layers: ```python class MyModel(nn.Module): def __init__ (self): super(MyModel, self). __init__ () self.fc1 = nn.Sequential(nn.Linear(10, 10), nn.ReLU()) def forward(self, x): return self.fc1(x) model = MyModel() print(model) model.fc1[0] # get linear layer ```