This is related to Discussion of practical speedup for pruning
I am trying to set the weights of one model to another like so:
mod = model.msd.msd_block # some module pruned_mod = pruned_model.msd.msd_block # some module it = 0 # this is because some layers can be removed, not important for question for l in req_layers: pruned_weight_name ="weight"+str(l) weight_name ="weight"+str(it) pruned_weights = getattr(pruned_mod, pruned_weight_name) weights = getattr(mod, weight_name) it2 = 0 for c in req_channels[it]: weights[0,it2] = pweights[0,c] it2 += 1 setattr(mod, weight_name, weights) it += 1
This throws the error:
ValueError: Cannot assign non-leaf Tensor to parameter 'weight0'. Model parameters must be created explicitly. To express 'weight0' as a function of another Tensor, compute the value in the forward() method.
I checked that the tensor
weights is the one I want but without
setattr(...) it does not update the weights in the model. Can anyone help me with a workaround?
For pruning speedup purposes, I have created a new class with fully pruned layers removed. In the remaining layers, the weight tensors are smaller than in the original. The new class keeps track of which non-pruned channels are inputs. The forward method etc. already works and there is a nice speed up! The only thing I need to do is set the kernel-weights to the original weights (which should be simple) but this is holding me back now.