It seems that DataParallel.module returns initial version of the module passed to it, e.g in the following code:
import torch
import torch.nn as nn
from torch.autograd import Variable
from torch.nn import DataParallel
class Model(nn.Module):
def __init__(self):
super().__init__()
self.tmp = 0
def forward(self, x):
self.tmp = x
print("Actual:", self.tmp)
net = DataParallel(Model())
for i in range(3):
input = Variable(torch.rand(2, 1))
net(input)
print("Returned:", net.module.tmp)
print("============")
I get:
Actual: Actual: Variable containing:
0.1303
[torch.cuda.FloatTensor of size 1x1 (GPU 0)]
Variable containing:
0.9771
[torch.cuda.FloatTensor of size 1x1 (GPU 1)]
Returned: 0
============
Actual: Actual: Variable containing:
1.00000e-03 *
5.5766
[torch.cuda.FloatTensor of size 1x1 (GPU 0)]
Variable containing:
0.8851
[torch.cuda.FloatTensor of size 1x1 (GPU 1)]
Returned: 0
============
Actual: Actual: Variable containing:
0.6126
[torch.cuda.FloatTensor of size 1x1 (GPU 1)]
Variable containing:
0.4636
[torch.cuda.FloatTensor of size 1x1 (GPU 0)]
Returned: 0
============
How can I retrieve the updated values?