Hey everyone, I get some behavior for which I am not sure whether it’s a bug or not. I have the following piece of code
for dataset_no, (inputs, targets) in enumerate(trainloader):
inputs = Variable(inputs.cuda(), requires_grad=False)
targets = Variable(targets.cuda(), requires_grad=False)
# zero the parameter gradients
optimizer.zero_grad()
# forward + backward + optimize
outputs = model(inputs)
objective = criterion(outputs, targets) \
+ model.readout.l1() * gamma_readout \
+ model.group_sparsity() * gamma_hidden
objective.backward()
optimizer.step()
The first loop runs fast, but the second time, the lines
inputs = Variable(inputs.cuda(), requires_grad=False)
targets = Variable(targets.cuda(), requires_grad=False)
are execute it takes forever.
The network I use is
NetWork (
(core): Sequential (
(conv0): Conv3d(1, 32, kernel_size=(5, 5, 5), stride=(1, 1, 1))
(f0): CustomNonLinearity (
)
(conv1): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), dilation=(2, 2, 2))
(f1): CustomNonLinearity (
)
(conv2): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), dilation=(2, 2, 2))
)
(readout): FullyConnected (32 x 24 x 52 -> 452)
(nonlinearity): CustomNonLinearity (
)
)
Does anyone see an obvious mistake? Thanks a lot.