Hey all, as part of ablation study, I had removed modules from the forward pass, but kept them in the model initialization. Is it possible to count only parameters that are actually used during the forward pass?
Thanks
Hey all, as part of ablation study, I had removed modules from the forward pass, but kept them in the model initialization. Is it possible to count only parameters that are actually used during the forward pass?
Thanks
I can’t think of a direct way to do this, but I guess if you’re training you could check which parameters actually got gradients and count only those.