I am getting a weird error on the training phase on random epochs, depending on the batch size, saying ValueError: (‘Cannot warp empty image with dimensions’, (257, 0)). For example, when I have a batch of 5 and I am training the inception v3, the error only occurs on epoch 32, which doesn’t make sense since the other epoch until this point iterated over the whole dataset too, so I don’t understand this random crash. The error occurs when loading the new batch in the epoch:
#Now we iterate throught the batches of the dataloader
for batch in loader:
I guess this error might be raised by a scipy transformation, if an empty numpy array is passed to it.
You could add print statements to the Dataset.__getitem__ method and check, which index (and transformation) raises this error to narrow down, if the previous transformations might have created an empty array.
I actually don’t use scipy, I only use skimage in the method! I did that debug, I printed all the shaped and all the images of my dataset, however it doesn’t crash But I know remembered that I didn’t try to iterate the dataloader, I only iterated the datasets. I am going to try that