Transfer learning model.eval()

1

I have one pretrained model and I want to take the output of the model as the partial input of my model. In this way, how should I set the pretrained part to static? Does pre_trained.eval() help?

1 Like
2

That will help partially, but pytorch will still create the computation graph for pre_trained. To avoid this you should detach the output of pre_trained. Like this, for example.

pre_out = pre_trained(input).detach()
out = in_training(pre_out)
1 Like
3

Will this result in the error AttributeError: 'Variable' object has no attribute 'detech'?

4

How about converting the output pre_out from Variable to Tensor. Then you need Variable as input to the next model, so you can convert the Tensor data back to Variable again:

 pre_out = pre_trained(input).data 
 out = in_training(Variable(pre_out))

I am not sure if this is the best solution, but this is what I have done.

5

All Variables have a detach method, but it isn’t spelled detech.