Hi,
I am training the model with input batch size of 16. After each batch iteration the parameter of model increase. why ?
here is my code
model = model.train()
for i in range(0, 335, 16 ):
y = 0
for k, x in enumerate(model.parameters()):
y += x.sum()
print (y)
input = data.cuda()
target = labels.cuda()
outputs = model(inputs)
loss = criterion(outputs, target)
optimizer.zero_grad()
loss.backward(retain_graph=True)
optimizer.step()
curr_err = loss.item()
err += curr_err
torch.cuda.empty_cache()
here is the increase parameter output
tensor(421.4539, device='cuda:0', grad_fn=<ThAddBackward>)
tensor(422.1019, device='cuda:0', grad_fn=<ThAddBackward>)----]16/335
tensor(423.4700, device='cuda:0', grad_fn=<ThAddBackward>)----]32/335
tensor(424.6810, device='cuda:0', grad_fn=<ThAddBackward>)----]48/335
tensor(425.5378, device='cuda:0', grad_fn=<ThAddBackward>)----]64/335
...................................... ...........................
................................ ..............................
tensor(434.1239, device='cuda:0', grad_fn=<ThAddBackward>)----]304/335
tensor(434.4677, device='cuda:0', grad_fn=<ThAddBackward>)==>-]320/335
Ideally it should be 421.4539
each time.
please help me out.