Count only used parameters

1

Hey all, as part of ablation study, I had removed modules from the forward pass, but kept them in the model initialization. Is it possible to count only parameters that are actually used during the forward pass?

Thanks

2

I can’t think of a direct way to do this, but I guess if you’re training you could check which parameters actually got gradients and count only those.