OK, it starts to make sense.

I’ll post here some notes, for future reference (there is also a notebook, here).

One thing I’ve figured out, by inspecting the network graph, is that it is made of `Variable`

s (or `Parameter`

s, which are technically `Variable`

s too) and `_functions`

.

Given that `h_x = resnet_18(x)`

, I have that `h_x.creator`

is a `torch.nn._functions.linear.Linear`

object, which `previous_functions`

is a `tuple`

of `len`

`3`

, containing

- a
`torch.autograd._functions.tensor.View`

object,
- a weight matrix of size
`(1000, 512)`

- a bias vector of size
`(1000)`

Some of these `_functions`

object have cached values (as long as `volatile`

is `False`

for the input `Variable`

), but one cannot rely on them.

On the other side, by typing `print(resnet_18)`

, one can visualise the network’s `Module`

s’ name and respective `repr`

. (The output is quite lengthy, so I will avoid copying it here.)

Some of these `Module`

s have other `Module`

s inside, and the `repr`

makes sure to unroll them out, in insertion order (meaning, when they have been assigned to the respective `self`

super-object) and may not reflect the order with which they have been used, in the `forward()`

method.

Now, let’s have a look for *ResNet*-18.

```
>>> resnet_18._modules.keys()
odict_keys(['conv1', 'bn1', 'relu', 'maxpool', 'layer1', 'layer2', 'layer3', 'layer4', 'avgpool', 'fc'])
```

We can now register a forward hook to `avgpool`

, and have it return its output `Variable`

.

Let’s check first what this hook gets as parameters.

```
avgpool_layer = resnet_18._modules.get('avgpool')
h = avgpool_layer.register_forward_hook(
lambda m, i, o: \
print(
'm:', type(m),
'\ni:', type(i),
'\n len:', len(i),
'\n type:', type(i[0]),
'\n data size:', i[0].data.size(),
'\n data type:', i[0].data.type(),
'\no:', type(o),
'\n data size:', o.data.size(),
'\n data type:', o.data.type(),
)
)
h_x = resnet_18(x)
h.remove()
```

gives us

```
m: <class 'torch.nn.modules.pooling.AvgPool2d'>
i: <class 'tuple'>
len: 1
type: <class 'torch.autograd.variable.Variable'>
data size: torch.Size([1, 512, 7, 7])
data type: torch.FloatTensor
o: <class 'torch.autograd.variable.Variable'>
data size: torch.Size([1, 512, 1, 1])
data type: torch.FloatTensor
```

Sweet. We can now create a `Tensor`

of size `512`

and copy over the embedding.

```
my_embedding = torch.zeros(512)
def fun(m, i, o): my_embedding.copy_(o.data)
h = avgpool_layer.register_forward_hook(fun)
h_x = resnet_18(x)
h.remove()
```

Now the `Tensor`

`my_embedding`

will contain what we were looking for.

The only point for which I am still not that confident, is the connection between a `torch.autograd._functions.something.Something`

object and the `Module`

that did create it. Right now I’m just guessing the paternity.