Energy consumtion per layer

Hi all!

I would like to estimate the energy consumption of each layer in a model. Does anyone knows how to do it?

You would probably need to set up some model for what consumes energy.
Going after FLOPS is the most common metric, I would wonder whether memory accesses (which have a huge impact on the time something takes) also play a role.
After that, you would need to estimate the explanatory variables for each block you want to measure.
Finally you can take the overall power consumption and then allocate that to the layers using the above.

An alternative to all this could be to gradually “shorten” the model and measure the total power per 1 epoch or so. The difference between two runs could then be attributed to the part the shortened model leaves out.

Best regards

Thomas

Hello,
I understood the FLOPS argument but not what is EXPLANATORY variables and how to go about it.

I am only interested in total energy consumption of the model and do not require layerwise diagnostic.
Is there an easy way to do this using PyTorch. Some rough estimates will also do. Even external links to GITHUB repos doing this will help