I am using a data loader to load batches of data into a DataParallel network. The issue is that my network uses batch normalization and batchnorm errors out when batch size is 1. But with 3 GPUs and a batch size of 15, I have an incomplete batch as the epoch change. This often leads at least one GPU with a batch size of 1. I have tried multiple variations of batch sizes, but the dataset size keeps giving me this problem. I don’t know how to resolve this using the automatic batching in the data loader. Is there an option to pad that last batch with all zeros or something?
marcman411 (Marc) #1
SimonW (Simon Wang) #2
you can manually pad the last batch or just drop it using