Hi everyone.
I’m training a custom architecture for a specific problem. This architecture works perfectly. However, I’m trying to run an experiment to calculate the loss according to the attributes. In other words, I select the attribute and calculate the loss for the selected attribute.
The problem that is occurring is the increase in memory. I have tried to analyze what is happening, but I couldn’t figure it out.
Here is part of my code.
prev_size = 1
for attribute_item in combined_attributes:
NUM_FEATURES = len(attribute_item)
print(attribute_item)
if NUM_FEATURES != prev_size:
reset_model_optimizer()
prev_size = NUM_FEATURES
start_time = time.time()
for epoch in range(50):
# Create a generator object for data batches
data_gen = data_generator(trainloader)
for i, examples in enumerate(data_gen, 0):
(inputs, desired_output) = examples
# zero the parameter gradients
optimizer.zero_grad()
# forward
output_predictions = model(inputs)
# loss
desired_output = desired_output.to(output_predictions.device)
loss = torch.nn.MSELoss()(output_predictions, desired_output)
# backward + optimize
loss.backward()
optimizer.step()
# New
output_loss.append(loss.item())
output_attribute.append(attribute_item)
output_epoch.append(j for j in range(50))
end_time = time.time()
execution_time = end_time - start_time
minutes = int(execution_time // 60)
seconds = int(execution_time % 60)
time_format = "{:02d}:{:02d}".format(minutes, seconds)
print("Time elapsed:", time_format)
#df_tmp = pd.DataFrame()
#df_tmp['Attribute'] = output_attribute
#df_tmp['Epoch'] = output_epoch
#df_tmp['Loss'] = output_loss
#file_name = '__'.join(attribute_item) + ".xlsx"
#df_tmp.to_excel(file_name, index=False)
output_attribute = []
output_epoch = []
output_loss = []
del inputs
del desired_output
del output_predictions
del loss
If necessary, here is the complete code is here:
I would appreciate it if someone could help me, please.
Thank you very much.