Add a new layer in trained model

vision
1

As the title suggests I need to add a layer in the middle of a model which has already been trained and for which I only have a .pt file without any knowledge of the architecture. Since I don’t have the code used to build the model I cannot simply modify the structure by adding my layer and copying the new layer wherever I need it.

I tried the following code

model = torch.load('pretrained_model.pt')
new_layer = customLayer(...)
model.layer1 = nn.Sequential(model.layer1, new_layer)

But I get an error

AttributeError: 'collections.OrderedDict' object has no attribute 'layer1'

that I don’t know how to fix. Does anyone know the correct way of adding a layer between any two given layers of a trained model?

2

This sounds strange, since you would either store the state_dict only (recommended way) which would mean that you would also need to store the model definition explicitly to be able to even restore the model, or you would store the model directly (not recommended), which would then pickle the source files. Could you describe what pretrained_model.pt contains?

3

I know it sounds strange but the process I am working on starts from a ONNX file that I then convert to torch. For this reason I don’t have the code used to build the model as usual but only the saved model.
But now that you pointed it out it seems like my idea is not working since I can’t perform inference if I only have the .pt file. Unfortunate but I’ll find a different way, thanks

4

This might sound like a dumb question, but do you need to train the new model as a whole, or just the new layer? If you need to train the whole model, it sounds like you are too confined to do that, but if you need to train the new layer, you ought to be able to just use the output of the saved model as input into a new 1-layer model and train that. Depending on your latent output and final targets, this might be trivial, but given the situation you’re in, I’m not sure there’s much else you can do.