Good afternoon everyone,
my neural network shows a behavior I can’t understand. I have already found out the source of the error, it is the SubsetRandomSample. Maybe you can help me to understand
When is create my Dataloader like this:
Dataset_val = Dataset(transform=Augmentation.transformation['val'])
Dataset_train = Dataset(transform=Augmentation.transformation['train'])
dataset_size = len(self.Dataset_train)
indices = list(range(dataset_size))
split = int(np.floor(test_size * dataset_size))
shuffle_dataset =True
if shuffle_dataset :
np.random.seed(random_seed)
np.random.shuffle(indices)
train_indices, val_indices = indices[split:], indices[:split]
train_sampler = SequentialSampler(train_indices)
valid_sampler = SequentialSampler(val_indices)
self.train_loader = torch.utils.data.DataLoader(Dataset_train,
batch_size= 64,
sampler=train_sampler,
)
self.validation_loader = torch.utils.data.DataLoader(self.Dataset_val,
batch_size=64,
sampler=valid_sampler,
)
Everything works fine.!!
Figure_2|668x500
But if I do:
Dataset_val = Dataset(transform=Augmentation.transformation['val'])
Dataset_train = Dataset(transform=Augmentation.transformation['train'])
dataset_size = len(self.Dataset_train)
indices = list(range(dataset_size))
split = int(np.floor(test_size * dataset_size))
train_indices, val_indices = indices[split:], indices[:split]
train_sampler = SubsetRandomSampler(train_indices)
valid_sampler = SubsetRandomSampler(val_indices)
self.train_loader = torch.utils.data.DataLoader(Dataset_train,
batch_size= 64,
sampler=train_sampler,
)
self.validation_loader = torch.utils.data.DataLoader(self.Dataset_val,
batch_size=64,
sampler=valid_sampler,
)
Figure_1|668x500
the training loss acts exactly like before, but the validation los stops decreasing almost immediately
I’ve tried to find any differences between the data, by looking at small Datasets of 5 to 10 datapoints, passed to the training or validation process, but in my opinion the same data, with the same type and size is passed
is there something wrong with my code, or do i missunderstand something completely wrong?
Best Greetings,
Filos92