Hello,
I am training a 3D CNN and I set batch_size = 1 in the dataloader object since a batch_size =2 gives cuda out of memory. So I thought about another way to increase the batch_size to 2. The crux of the code looks something like (in the trainer loop of the program) :
for batch_idx, (subject) in enumerate(train_loader):
# Load the subject and its ground truth
itr+=1
image = subject['image']
mask = subject['gt']
# Loading images into the GPU and ignoring the affine
image, mask = image.float().cuda(), mask.float().cuda()
#Variable class is deprecated - parameteters to be given are the tensor, whether it requires grad and the function that created it
optimizer.zero_grad()
image, mask = Variable(image, requires_grad = True), Variable(mask, requires_grad = True)
# Forward Propagation to get the output from the models
output = model(image.float())
# Computing the loss
if itr%2==1 :
loss_odd = loss_fn(output.double(), mask.double(), n_classes)
if itr%2==0 :
loss_even = loss_fn(output.double(), mask.double(), n_classes)
loss = loss_odd + loss_even
loss.backward()
optimizer.step()
In the above dataloader I have set the batch_size =1 and I am trying to manually make the network “feel” as if the batch_size = 2 by accumulating the loss function and doing back prop after every 2 examples…
Am I doing this right?
Or am I missing something?
Thanks a lot in advance