Hello K. Frank !
The last three operations are a LeakyReLU and 2 3D Convolutions with a kernel of size=5
such ass
nn.LeakyReLU(),
nn.Conv3d(in_channels=16, out_channels=16, kernel_size=5, padding=2),
nn.Conv3d(in_channels=16, out_channels=1, kernel_size=5, padding=2),
as for the code itself
lr=1e-3
optimizer = torch.optim.Adam(model.parameters(), lr=lr)
weights = torch.tensor([300.]).to(device)
loss_fn = torch.nn.BCEWithLogitsLoss(pos_weight=weights, reduction='mean')
def epoch_train_all(model: Baseline, optim, loss_fn, device, batch_size:int = 5, path: str = "./data/train/"):
optim.zero_grad()
avg_loss = 0.0
nb_files = 0
model.to(device)
for id, dm in get_files_it(path):
nb_files += 1
print(f"\r\tTRAIN file nb{nb_files:03d}, id:{id}", end=" ")
# transferring data from cache to device
data = dm.data.to(device)
pred = model(data)
del data
mask = dm.mask.to(device)
loss = loss_fn(pred, mask)
del pred
del mask
# once the data has been used, free it to free some space on the device
loss.backward()
avg_loss += float(loss)
del loss
if nb_files % batch_size == 0:
print("optim !", end='')
optim.step()
optim.zero_grad()
return
optim.step()
optim.zero_grad()
avg_loss /= nb_files
print(f"avg. train loss: {avg_loss}")
return avg_loss
I have isolated in the middle the part that you asked for.
I will add a few precisions:
- I use gradient accumulation, so my batch size can control the frequency of my backward pass
- there is here a
return after the 5 items in my batch to allow fitting on only the first 5 elements of my data (I have disabled the shuffling)
-
get_files_it(path): is an iterator I have made which returns my processed data in a class I have created
- I delete the elements once they are not used anymore to save some memory
As for the statistics:
data: input data
prediction: the output value of my network
pred w/ sig: the previous prediction passed into a sigmoid (not in the network because working bit BCEWithLogitsLoss)
ground truth: the expected mask
1st epoch:
data: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[0.0; 1.0]
prediction: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[-0.016314834356307983; 0.0024049831554293633]
pred w/ sig: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[0.49592143297195435; 0.5006012320518494]
ground truth: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[0.0; 0.0021929824724793434]
10th epoch
data: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[0.0; 1.0]
prediction: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[-269.3184814453125; -1.7151740789413452]
pred w/ sig: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[0.0; 0.15249381959438324]
ground truth: shape=torch.Size([1, 1, 248, 88, 176]), type: torch.cuda.FloatTensor, range:[0.0; 0.0021929824724793434]
Thank you very much for your time and help, I appreciate 
