Get initialization weights for pre-trained models

vision
1

For some experiments, it seems useful to have access to the initialization weights on pretrained models to do analysis.
For models in torchvision.models , like ResNet-50, is it possible to know what the initialization weights were? If not, could we know what the training recipe/code and seed were to reproduce the initialization weights in local experiments?

2

@nipunagarwala, Please see the following article for methods to get the initialization weights. https://www.askpython.com/python-modules/initialize-model-weights-pytorch

Summary:


`# Linear Dense Layer`
`layer_1 ``=` `nn.Linear(``5``, ``2``)`
`print``(``"Initial Weight of layer 1:"``)`
`print``(layer_1.weight)`

`# Initialization with uniform distribution`
`nn.init.uniform_(layer_1.weight, ``-``1``/``sqrt(``5``), ``1``/``sqrt(``5``))`
`print``(``"\nWeight after sampling from Uniform Distribution:\n"``)`
`print``(layer_1.weight)`
3

So we do know the function used for initialization, but thats not enough to get the exact weights, is it?
We need to know the seed used for the training process, and whether the seed actually works the same way on different platforms.

Not sure how to exactly replicate it, and if it is even possible.

4

@nipunagarwala, I would like to draw your attention to

`print``(layer_1.weight)`

in my previous response.

5

This is for new layers, not for pretrained models.
Maybe my question wasn’t clear