Corrupt JPEG data: 24 extraneous bytes before marker 0xd9

1

Hi,

When I check the folder of my images by below code, there is no corrupted images.
but when I use a custom dataset and start to train the network this error happens during training (it does not stop the training and it will go on after that, but I do not know if it affect the parameters being trained and I wonder how I can handle this).

Thanks,

Epoch: 0
Corrupt JPEG data: 24 extraneous bytes before marker 0xd9
[===========================================================>…] Step: 147ms | Tot: 11s720ms | Loss: 1.285 | Acc: 57.554% (640/111 12/12 2
[====================================================>…] Step: 883ms | Tot: 4s311ms | Loss: 0.665 | Acc: 60.587% (289/47 5/5 5
Saving…

annotations = pd.read_csv(datapath, sep=’\t’)
root_dir=’/home/ubuntu/files’
for index in range(len(annotations)):
img_path = os.path.join(root_dir, annotations.loc[index, ‘image_path’])
try:
img = Image.open(img_path)
img.verify()
except:
print(‘Bad file:’, index) # print out the names of corrupt files

2

I would try to track down the image file, which is raising this error and either remove it or load and save it again to hopefully get rid of the JPEG corruption.

3

Thanks. I did the same thing. Using mogrify command line tool (provided by ImageMagick), I could detect which image cause this error.

4

can you share the command line of mogrify for detection, thanks a lot

5

Thank you for the help, i had a similar problem whilst trying to send images through an api