Feeding JFIF vs EXIF standard jpg images to a model -- JFIF breaks with RuntimeError: stack expects a non-empty TensorList error

Do you know why the model doesn’t work for JFIF standard jpg images while it works for EXIF standard jpg images? and could you please guide how to fix it?

(phosa) [jalal@goku phosa]$ python demo.py --filename input/occluded_bat.jpg --class_name bat
2021-03-31 16:29:06,904 INFO     Calling with args: Namespace(class_name='bat', filename='input/occluded_bat.jpg', lw_collision=None, lw_depth=None, lw_inter=None, lw_inter_part=None, lw_scale=None, lw_scale_person=None, lw_sil=None, mesh_index=0, output_dir='output')
2021-03-31 16:29:09,696 INFO     Loading checkpoint from detectron2://PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl
2021-03-31 16:29:09,749 INFO     URL https://dl.fbaipublicfiles.com/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl cached in /home/grad3/jalal/.torch/fvcore_cache/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl
2021-03-31 16:29:12,002 INFO     Reading a file from 'Detectron2 Model Zoo'
WARNING: You are using a SMPL model, with only 10 shape coefficients.
class_name:  bat
Traceback (most recent call last):
  File "demo.py", line 145, in <module>
    main(get_args())
  File "demo.py", line 120, in main
    object_parameters = find_optimal_poses(
  File "/scratch3/research/code/phosa/phosa/pose_optimization.py", line 426, in find_optimal_poses
    object_parameters[k] = torch.stack(v)
RuntimeError: stack expects a non-empty TensorList

vs EXIF

(phosa) [jalal@goku phosa]$ python demo.py --filename input/bat_sidehold.jpg --class_name bat
2021-03-31 16:30:33,675 INFO     Calling with args: Namespace(class_name='bat', filename='input/bat_sidehold.jpg', lw_collision=None, lw_depth=None, lw_inter=None, lw_inter_part=None, lw_scale=None, lw_scale_person=None, lw_sil=None, mesh_index=0, output_dir='output')
2021-03-31 16:30:36,496 INFO     Loading checkpoint from detectron2://PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl
2021-03-31 16:30:36,503 INFO     URL https://dl.fbaipublicfiles.com/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl cached in /home/grad3/jalal/.torch/fvcore_cache/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl
2021-03-31 16:30:36,637 INFO     Reading a file from 'Detectron2 Model Zoo'
WARNING: You are using a SMPL model, with only 10 shape coefficients.
class_name:  bat
  0%|                                                                                                    | 0/200.0 [00:00<?, ?it/s]PoseOptimizer(
  (pool): MaxPool2d(kernel_size=7, stride=1, padding=3, dilation=1, ceil_mode=False)
  (renderer): Renderer()
)
loss: 1.74e+03:  25%|██████████████████▊                                                        | 50/200.0 [00:25<01:14,  2.02it/s]PoseOptimizer(
  (pool): MaxPool2d(kernel_size=7, stride=1, padding=3, dilation=1, ceil_mode=False)
  (renderer): Renderer()
)
loss: 1.67e+03:  50%|█████████████████████████████████████                                     | 100/200.0 [00:52<00:53,  1.86it/s]PoseOptimizer(
  (pool): MaxPool2d(kernel_size=7, stride=1, padding=3, dilation=1, ceil_mode=False)
  (renderer): Renderer()
)
loss: 1.67e+03:  75%|███████████████████████████████████████████████████████▌                  | 150/200.0 [01:19<00:25,  1.95it/s]PoseOptimizer(
  (pool): MaxPool2d(kernel_size=7, stride=1, padding=3, dilation=1, ceil_mode=False)
  (renderer): Renderer()
)
loss: 1.67e+03: 100%|██████████████████████████████████████████████████████████████████████████| 200/200.0 [01:45<00:00,  1.90it/s]
Loss 17.7909: 100%|██████████████████████████████████████████████████████████████████████████████| 400/400 [02:32<00:00,  2.62it/s]
2021-03-31 16:34:55,988 INFO     Saved rendered image to output/bat_sidehold.jpg.
2021-03-31 16:34:56,010 INFO     Saved top-down image to output/bat_sidehold_top.jpg.

image metadata:

(phosa) [jalal@goku phosa]$ file input/occluded_bat.jpg 
input/occluded_bat.jpg: JPEG image data, JFIF standard 1.01
(phosa) [jalal@goku phosa]$ file input/bat_sidehold.jpg 
input/bat_sidehold.jpg: JPEG image data, EXIF standard
(phosa) [jalal@goku phosa]$ identify input/occluded_bat.jpg
input/occluded_bat.jpg JPEG 1440x810 1440x810+0+0 8-bit sRGB 242255B 0.000u 0:00.000
(phosa) [jalal@goku phosa]$ identify input/bat_sidehold.jpg 
input/bat_sidehold.jpg JPEG 1156x1600 1156x1600+0+0 8-bit sRGB 96559B 0.000u 0:00.007

As a followup, do you know how I can convert JFIF standard jpg images to EXIF standard jpg images in Linux? Somehow the majority of images I download from the Web end up being JFIF standard jpg.

This error doesn’t seem to be PyTorch-specific, as it seems the image loading library fails to load an image based on your description.
Similar issue posted here with debug suggestion.

1 Like