1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [1, 3, 375, 1242]

Sure, here is one sample of my RGB mask:
[[[107 142 35]
[107 142 35]
[107 142 35]

[ 70 130 180]
[ 70 130 180]
[ 70 130 180]]

[[107 142 35]
[107 142 35]
[107 142 35]

[ 70 130 180]
[ 70 130 180]
[ 70 130 180]]

[[107 142 35]
[107 142 35]
[107 142 35]

[ 70 130 180]
[ 70 130 180]
[ 70 130 180]]

[[128 64 128]
[128 64 128]
[128 64 128]

[128 64 128]
[128 64 128]
[128 64 128]]

[[128 64 128]
[128 64 128]
[128 64 128]

[128 64 128]
[128 64 128]
[128 64 128]]

[[128 64 128]
[128 64 128]
[128 64 128]

[128 64 128]
[128 64 128]
[128 64 128]]]

The shape is (375, 1242, 3)


This is one example of my mask

Thanks for the example.
Are you using these RGB values directly for your classes?
If so, I would recommend to use a mapping such that each pixel contains only a class index in the range [0, nb_classes-1].

Sorry I’m new to pytorch, what do you mean by mapping?
I put my code link below if you are interested.

I mean something like a key value pair between your color codes in RGB and the corresponding class index.
E.g. [128, 64, 128] would map to class0.
Have a look at this post for another example using grayscale images.
Could you post the classes for each separate color in your segmentation mask?

2 Likes

Thank you for your explanation, I’m using the KITTI semantic segmentation datasets. which are conform with The Cityscapes Dataset, it has 30 classes. But I didn’t find the key value pair between my colour codes and the corresponding class index.
I have seen the post you mentioned, and I don’t know the class mapping value neither, could you help?

I’m not sure where to find the mapping. Here it seems a mapping is given for 11 classes.
However, if you can’t find the right mapping, you could also just get all unique color codes and create your own mapping.

Thanks a lot for the effort you put. I really appreciate it. At the moment I am getting CUDA error: out of memory. I will sort out memory issue and will update you.

Sorry to bother again, I have more questions:

  1. In the unet example, if I do the image mapping, for example, if I have 10 classes for the labels, the last layer of the network is log_softmax, does it mean that the output of the network is the probability map of of every pixel.
  2. If so, after using the NLLLoss and the Adam optimisation, the weight in the network is optimized, now I feed the network with a random training image, the output is a probability map, if I want to visualize it, I need to simply imshow(output) or I have to remap it back?
  1. Yes, you will get the log probabilities with each channel corresponding to the class index.

  2. plt.imshow should work. I would try to transform the output using torch.exp to get the probabilities in the range [0, 1], since the colormap might look more “natural”.

Hi, I’m wondering how to create the mapping, I saw that your mapping code in the post, how did you find the mapping relationship?
And how to create my own mapping? Do you mean that I should check all color in my mask image and then find the html code :open_mouth:

If the mapping is not given in the dataset (or the hosting website) somewhere, you could just create your own mapping like I did for the medical segmentation dataset.
Which dataset are you using? Maybe the mapping was already created somewhere.

Thank you for your reply, could you give me the link of your example for the medical segmentation dataset?
Here is the dataset I used:

Sure! @Neda used the TEM dataset from here. I just downloaded it and created a pseudo-mapping using all unique values in the images, since I couldn’t find an official color-to-label mapping.

I’m just wondering how you create your pseudo-mapping ? You print all values in the images and observe the difference? When I try to print the image, It only shows me very small part of the data, and normally for one image, the data is very large, so could you tell me how you can observe all different unique values in one image and then create the mapping.

You could try the following code using torch.unique and the dim argument:

image = Image.open(path)
im_arr = np.array(image)
im_tensor = torch.from_numpy(im_arr)
im_tensor = im_tensor.permute(2, 0, 1)
unique_colors = torch.unique(im_tensor.view(im_tensor.size(0), -1), dim=1)

Let me know, if that works for you!

1 Like

Thank you very much :smile:, this works!
Here is my code:

colors = torch.tensor([])
for i in range(len(trainmask)):
    im_str = trainimage[i]
    im_arr = io.imread(os.path.join(train_mask, im_str))
    im_tensor = torch.from_numpy(im_arr)
    im_tensor = im_tensor.permute(2,0,1)
    unique_colors = torch.unique(im_tensor.view(im_tensor.size(0), -1)).type(torch.FloatTensor)
    colors = torch.cat((colors, unique_colors))
    colors = torch.unique(colors)
print(colors)

and here is the result:

tensor([  64.,  150.,  107.,  110.,   20.,  220.,  119.,  180.,   11.,
         152.,    0.,  160.,   60.,  102.,  230.,   81.,  255.,  120.,
          30.,  100.,   80.,  140.,  251.,  142.,   74.,  111.,   90.,
          35.,  156.,  190.,  170.,  153.,   32.,  250.,  244.,  165.,
         128.,   70.,  130.,  232.])

There are 40 unique colour in my mask image. It means that I have 40 unique class in my dataset.

I’m now wondering how to find the relationship between these colour value and the class type.
Do you have some suggestions?

I’m not that sure about the last unique call.
unique_colors should be a [number_of_colors, 3] tensor so that each unique color contains the RGB values. Currently you are calling unique again on it, so that the RGB color is not usable anymore.

Anyway, if you’ve fixed this and can’t find any official mapping, I would just create a dict with your own mapping and stick to it. In the end you just need class indices for the different colors. It doesn’t matter if car has the class index 0 or 40 as long as you don’t mix it up. :wink:

1 Like

Do you mean I should specify the dimension of the unique to get [number_of_colors, 3]?
I try to specify the dimension of the unique, when I add dim=1, I get this error:

unique() got an unexpected keyword argument 'dim'

my torch version is 0.4.1
Do you know what’s wrong?:smiley:

The dim argument was added after the 0.4.1 release, so I would recommend to update to the latest stable release (1.0) to get this and other useful features. :wink:

1 Like