Backward extremely slow while fix major part of the network

autograd
1

Hello. I find that while training a network, loss.backward() takes extremely long if I fix most weights of the network. Why can this happen?