I am training a 3D CNN and I set batch_size = 1 in the dataloader object since a batch_size =2 gives cuda out of memory. So I thought about another way to increase the batch_size to 2. The crux of the code looks something like (in the trainer loop of the program) :
for batch_idx, (subject) in enumerate(train_loader): # Load the subject and its ground truth itr+=1 image = subject['image'] mask = subject['gt'] # Loading images into the GPU and ignoring the affine image, mask = image.float().cuda(), mask.float().cuda() #Variable class is deprecated - parameteters to be given are the tensor, whether it requires grad and the function that created it optimizer.zero_grad() image, mask = Variable(image, requires_grad = True), Variable(mask, requires_grad = True) # Forward Propagation to get the output from the models output = model(image.float()) # Computing the loss if itr%2==1 : loss_odd = loss_fn(output.double(), mask.double(), n_classes) if itr%2==0 : loss_even = loss_fn(output.double(), mask.double(), n_classes) loss = loss_odd + loss_even loss.backward() optimizer.step()
In the above dataloader I have set the batch_size =1 and I am trying to manually make the network “feel” as if the batch_size = 2 by accumulating the loss function and doing back prop after every 2 examples…
Am I doing this right?
Or am I missing something?
Thanks a lot in advance