Does freezzing part of the network saves GPU memory?

Try
with torch.no_grad():
____feat = Net1(input)
output = Net2(feat)

This should work.

See this topic for explanation.