This question is quite complicated. I give an example.
class model(nn.Module)
def __init__()
self.l1 = nn.linear(10,10)
self.l2 = nn.linear(10,10)
def forward(x)
y1 = self.l1(x)
y2 = self.l2(x)
y3 = y1 + y2
return y2 + y2 + y3*y1
In this example, I could use forward_hook functions to trace two linear layers and their parameters.fn is hook function.\
m.register_forward_hook(fn)
However, y3
is not counted as a parameter and the macs of y2 + y2 + y3*y1
is not counted in macs, too.
How can I solve this?
While “macs” is a way of measuring layers’ complexity. For example, y1 *(y2 + y3) is one macs, if y1, y2, y3 are floats.