Transfer learning model.eval()

I have one pretrained model and I want to take the output of the model as the partial input of my model. In this way, how should I set the pretrained part to static? Does pre_trained.eval() help?

1 Like

That will help partially, but pytorch will still create the computation graph for pre_trained. To avoid this you should detach the output of pre_trained. Like this, for example.

pre_out = pre_trained(input).detach()
out = in_training(pre_out)
1 Like

Will this result in the error AttributeError: 'Variable' object has no attribute 'detech'?

How about converting the output pre_out from Variable to Tensor. Then you need Variable as input to the next model, so you can convert the Tensor data back to Variable again:

 pre_out = pre_trained(input).data 
 out = in_training(Variable(pre_out))

I am not sure if this is the best solution, but this is what I have done.

All Variables have a detach method, but it isn’t spelled detech.