This question is quite complicated. I give an example.
class model(nn.Module) def __init__() self.l1 = nn.linear(10,10) self.l2 = nn.linear(10,10) def forward(x) y1 = self.l1(x) y2 = self.l2(x) y3 = y1 + y2 return y2 + y2 + y3*y1
In this example, I could use forward_hook functions to trace two linear layers and their parameters.fn is hook function.\
y3 is not counted as a parameter and the macs of
y2 + y2 + y3*y1 is not counted in macs, too.
How can I solve this?
While “macs” is a way of measuring layers’ complexity. For example, y1 *(y2 + y3) is one macs, if y1, y2, y3 are floats.