How to create a linear layer and initialize it with specific weight?

1

Hi all. I want to create a linear layer and then initialize it with specific weights(I have a .sav file). Can anybody tell me how to realize it with pytorch?
after doing this I should discard some layers of pretrained alexnet and add this linear layer to the alexnet. Can anybody tell me how to realize that with pytorch?

2

You can load parameters via:

with torch.no_grad():
    model.layer.weight.copy_(custom_weight_tensor)

I don’t know exactly what “discard” layers means, but in case you want to remove them you could write a custom model and reuse other layers or replace them with nn.Identity assuming that the shapes of the activations would still match.

3

Hi, thank you for your information. I trained a linear regression model and now I want to use these weights and bias from linear regression model to initialize the linear layer. I hope that the linear layer can behave like the linear regression model. how to realize that?

4

I don’t know how the architecture of the linear regression model looks like, but in case it’s using non-linear activation functions you wouldn’t be able to squeeze this model into a single linear layer.
Otherwise I would assume that training a single linear layer would also work for your use case.