Using same variable name in model calculation


I’m wondering if it makes a difference to reuse a variable name (from coder’s point of view) in model calculation.

For example, (1)

x = nn.module_1(input)
x = nn.module_2(x)
x = nn.module_3(x)

Versus (2)

x1 = nn.module_1(input)
x2 = nn.module_2(x1)
x3 = nn.module_3(x2)

Ignoring the case where one needs (I think) to use (2) (let’s say we need x2 but not x3 for another calculation), is (1) better?
I’ve been using (2) to improve readability (I use longer variable names that explain what’s going on)

Many Thanks!

(1) is better because with (2) you’re explicitly keeping all the intermediates alive. If you don’t want to reuse variable names, you’d need to equivalently do:

x1 = nn.module_1(input)
x2 = nn.module_2(x1)
del x1
x3 = nn.module_3(x2)
del x2

(how much more memory you end up using in practice depends on the specific module and whether they are already save the inputs for backward)

1 Like

Thanks very much for your help soulitzer! This cleared things up for me!