What are the main reasons for receiving RuntimeError: stack expects a non-empty TensorList error for torch.stack?

Mona_Jalal · April 1, 2021, 12:36am

I am not sure why sometimes this error is shown when using images. Initially, I thought it is happening because of using JFIF standard jpg images vs EXIF standard jpg images but now I found a case that also happens for EXIF standard jpg images. How can I debug this problem and how can that be solved? I am using PyTorch 1.8.1.

(phosa) [jalal@goku phosa]$ file input/input/bat* | grep -i exif
input/input/bat5.jpg:  JPEG image data, EXIF standard
(phosa) [jalal@goku phosa]$ python demo.py --filename input/input/bat5.jpg --class_name bat
2021-03-31 20:26:59,384 INFO     Calling with args: Namespace(class_name='bat', filename='input/input/bat5.jpg', lw_collision=None, lw_depth=None, lw_inter=None, lw_inter_part=None, lw_scale=None, lw_scale_person=None, lw_sil=None, mesh_index=0, output_dir='output')
2021-03-31 20:27:02,203 INFO     Loading checkpoint from detectron2://PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl
2021-03-31 20:27:02,213 INFO     URL https://dl.fbaipublicfiles.com/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl cached in /home/grad3/jalal/.torch/fvcore_cache/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_3x_coco/164955410/model_final_3c3198.pkl
2021-03-31 20:27:02,300 INFO     Reading a file from 'Detectron2 Model Zoo'
WARNING: You are using a SMPL model, with only 10 shape coefficients.
class_name:  bat
Traceback (most recent call last):
  File "demo.py", line 145, in <module>
    main(get_args())
  File "demo.py", line 120, in main
    object_parameters = find_optimal_poses(
  File "/scratch3/research/code/phosa/phosa/pose_optimization.py", line 426, in find_optimal_poses
    object_parameters[k] = torch.stack(v)
RuntimeError: stack expects a non-empty TensorList

an image that breaks the code:

(phosa) [jalal@goku phosa]$ identify -verbose input/input/bat5.jpg
Image: input/input/bat5.jpg
  Format: JPEG (Joint Photographic Experts Group JFIF format)
  Mime type: image/jpeg
  Class: DirectClass
  Geometry: 1417x957+0+0
  Resolution: 72x72
  Print size: 19.6806x13.2917
  Units: PixelsPerInch
  Colorspace: sRGB
  Type: TrueColor
  Base type: Undefined
  Endianess: Undefined
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
  Channel statistics:
    Pixels: 1356069
    Red:
      min: 4  (0.0156863)
      max: 246 (0.964706)
      mean: 121.121 (0.474984)
      standard deviation: 65.4112 (0.256514)
      kurtosis: -1.42305
      skewness: -0.279998
      entropy: 0.952832
    Green:
      min: 4  (0.0156863)
      max: 248 (0.972549)
      mean: 123.048 (0.482542)
      standard deviation: 63.9564 (0.25081)
      kurtosis: -1.37041
      skewness: -0.31156
      entropy: 0.955039
    Blue:
      min: 4  (0.0156863)
      max: 247 (0.968627)
      mean: 125.684 (0.492879)
      standard deviation: 64.8382 (0.254268)
      kurtosis: -1.35544
      skewness: -0.330874
      entropy: 0.957843
  Image statistics:
    Overall:
      min: 4  (0.0156863)
      max: 248 (0.972549)
      mean: 123.284 (0.483468)
      standard deviation: 64.7353 (0.253864)
      kurtosis: -1.38286
      skewness: -0.307323
      entropy: 0.955238
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 1417x957+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: JPEG
  Quality: 94
  Orientation: Undefined
  Properties:
    date:create: 2021-04-01T00:02:56+00:00
    date:modify: 2021-04-01T00:01:01+00:00
    exif:DateTime: 2019-05-08T22:34:50+02:00
    exif:ExifOffset: 156
    exif:ExifVersion: 48, 50, 50, 48
    exif:PixelXDimension: 1417
    exif:PixelYDimension: 957
    exif:ResolutionUnit: 2
    exif:Software: Adobe Photoshop CS6 Windows
    exif:XResolution: 72/1
    exif:YResolution: 72/1
    jpeg:colorspace: 2
    jpeg:sampling-factor: 1x1,1x1,1x1
    signature: b933305114f74f66bd8c0430f6689843f3287008be2e0663974091322a869040
  Profiles:
    Profile-app12: 15 bytes
    Profile-exif: 210 bytes
    Profile-xmp: 960 bytes
  Artifacts:
    filename: input/input/bat5.jpg
    verbose: true
  Tainted: False
  Filesize: 256664B
  Number pixels: 1.35607M
  Pixels per second: 39.5923MB
  User time: 0.030u
  Elapsed time: 0:01.034
  Version: ImageMagick 6.9.10-68 Q16 x86_64 2021-02-24 https://imagemagick.org

an image that will work:

(phosa) [jalal@goku phosa]$ identify -verbose input/bat_sidehold.jpg 
Image: input/bat_sidehold.jpg
  Format: JPEG (Joint Photographic Experts Group JFIF format)
  Mime type: image/jpeg
  Class: DirectClass
  Geometry: 1156x1600+0+0
  Resolution: 72x72
  Print size: 16.0556x22.2222
  Units: PixelsPerInch
  Colorspace: sRGB
  Type: TrueColor
  Base type: Undefined
  Endianess: Undefined
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
  Channel statistics:
    Pixels: 1849600
    Red:
      min: 0  (0)
      max: 255 (1)
      mean: 226.177 (0.886968)
      standard deviation: 64.3149 (0.252215)
      kurtosis: 5.41792
      skewness: -2.58614
      entropy: 0.409396
    Green:
      min: 0  (0)
      max: 255 (1)
      mean: 219.381 (0.860319)
      standard deviation: 57.0324 (0.223657)
      kurtosis: 0.586459
      skewness: -1.36916
      entropy: 0.443656
    Blue:
      min: 0  (0)
      max: 255 (1)
      mean: 220.606 (0.86512)
      standard deviation: 54.885 (0.215235)
      kurtosis: 0.68018
      skewness: -1.36917
      entropy: 0.442032
  Image statistics:
    Overall:
      min: 0  (0)
      max: 255 (1)
      mean: 222.055 (0.870803)
      standard deviation: 58.7441 (0.230369)
      kurtosis: 2.75662
      skewness: -1.88089
      entropy: 0.431695
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Interlace: JPEG
  Intensity: Undefined
  Compose: Over
  Page geometry: 1156x1600+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: JPEG
  Quality: 65
  Orientation: Undefined
  Properties:
    date:create: 2021-03-26T05:48:14+00:00
    date:modify: 2021-03-26T05:48:14+00:00
    exif:Copyright: (c) Zhagunov | Dreamstime.com
    exif:ResolutionUnit: 2
    exif:XResolution: 72/1
    exif:YCbCrPositioning: 1
    exif:YResolution: 72/1
    jpeg:colorspace: 2
    jpeg:sampling-factor: 2x2,1x1,1x1
    signature: c1017a4d64ee13044fd81942a442cbaf6c28e42c61282ed3ab376d3dd92fd3c1
    xmpRights:WebStatement: https://www.dreamstime.com/about-stock-image-licenses
  Profiles:
    Profile-8bim: 54 bytes
    Profile-exif: 126 bytes
    Profile-iptc: 41 bytes
      Copyright String[2,116]: (c) Zhagunov | Dreamstime.com
      unknown[2,0]: 
    Profile-xmp: 3158 bytes
  Artifacts:
    filename: input/bat_sidehold.jpg
    verbose: true
  Tainted: False
  Filesize: 96559B
  Number pixels: 1849600
  Pixels per second: 35.8514MB
  User time: 0.050u
  Elapsed time: 0:01.051
  Version: ImageMagick 6.9.10-68 Q16 x86_64 2021-02-24 https://imagemagick.org

ptrblck · April 1, 2021, 6:27am

I’m not sure how you are loading the images, but it seems that the used library has trouble loading these problematic image files (e.g. PIL or OpenCV).
Based on the error message a None object will be returned which then fails in the torch.stack operation.
Could you try to load the mentioned images using the image lib and check what the output would be?

Mona_Jalal · April 2, 2021, 6:19am

thanks for the pointers. yeah the problem happens because the bat class is not detected at all in certain classes (please let me know if you might have any suggestions)

the photo of the girl where bat is detected is the only one phosa doesn’t break.

ptrblck · April 2, 2021, 6:33am

Is the problem that the model isn’t able to detect this class and you would like to improve the model itself or would you rather like to fix the None error in case no valid classes were detected by the model?
I would personally try to fix the second issue first, as you might face the same error for other classes in case the model is also unable to detect them.