I have a dataset that contains both the training and validation set. I am aware that I can use the SubsetRandomSampler
to split the dataset into the training and validation subsets. The dataset however, has an unbalanced class ratio. How can I also use the WeightedRandomSampler
together with the SubsetRandomSampler
? Below is what I currently have using only the SubsetRandomSampler
.
# build the dataset
dset = TrajectoryDataset(
path,
obs_len=args.obs_len,
skip=args.skip)
# get the class sample counts
class_sample_count = [dset.positive_sample_count(), dset.negative_sample_count())
# split to train val
validation_split = 0.8
dataset_size = len(dset)
indices = list(range(dataset_size))
split = int(np.floor(validation_split * dataset_size))
if 1 :
np.random.seed(1337)
np.random.shuffle(indices)
train_indices, valid_indices = indices[split:], indices[:split]
train_sampler = SubsetRandomSampler(train_indices)
valid_sampler = SubsetRandomSampler(valid_indices)
train_loader = DataLoader(
dset,
batch_size=32,#args.batch_size,
num_workers=2,#args.loader_num_workers,
collate_fn=seq_collate,
sampler=train_sampler)
valid_loader = DataLoader(
dset,
batch_size=32,#args.batch_size,
num_workers=2,#args.loader_num_workers,
collate_fn=seq_collate,
sampler=valid_sampler)