Do ReLU and Dropout layers take up space on backward pass

Hi all,

I’m trying to get an estimate of my model size. Should intermediate layers that are the result of dropout or nonlinearities be included in the estimate?

Thank you,
Sean