Yes, WeightedRandomSampler expects a weight value for each sample in the dataset.
Here is a complete example.
Alternatively, you could also use a class weight and pass it to your criterion, if it fits your use case.
E.g. nn.CrossEntropyLoss accepts a weight tensor, which assigns a specific weight to each class.