Output of nn.MaxUnpool2d is not unique resp. gives different results after each run

1

Hi all,

I am trying to implement a U-NET. I stumbled upon a problem using MaxUnpool2d. The solution is not unique, hence, I get different solutions each time I run it.

I reproduced the error here:

import torch
import torch.nn as nn

x = torch.load("x.pt")
indices = torch.load("indices.pt")

unpool = nn.MaxUnpool2d(kernel_size=(3, 3), stride=(1, 1), padding=(0, 0))


###############################################

output_1 = unpool(x,indices)
output_2 = unpool(x,indices)


############################################
print("Sum of Output_1:")
print(sum(sum(sum(sum(output_1)))))
print("---------------------")
print("Sum of Output_2:")
print(sum(sum(sum(sum(output_2)))))

GitHub repository:
https://github.com/fridolinvii/MaxUnpool2d.git

I’d be thankful for anyone, who has an idea, what might be wrong.

Edit: My problem is, that output_1 is different from output_2

2

What do you mean by “the solution is not unique” ?
Multiple indices point to the same output?

3

I mean that if I do the following:

output_1 is different to output_2

4

How did you get the indices? Are they overlapping by any chance?

5

Thank you for your reply.
I apply

  1. Conv2D
  2. maxpool2D
  3. Cov2D
  4. [x,indices] = maxpool2D <-got these indices
  5. multiple FC layers
  6. y = RNN (convert the output to the size of x)
  7. output_1 = unpool(y,indices)

How can I check, If I have overlapping indices?
If you want, I can give an exemplary script.

6

Hi,

In the case where the maxpool2d kernels in the forward overlap, the inpooling can be ambiguous as two values need to be written at the same place.
This function is non-deterministic in this case. You can read more about this here.

7

Thank you for the link.

I tried to add:

torch.manual_seed(0)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

But I still get two different outputs.

If I understand it correctly, the problem might be

floating-point addition is not perfectly associative for floating-point operands

Hence, the non-deterministic function?

8

Hi,

Yes the seeding won’t change anything for the determinism.
Unfortunately, the only options I can see here is to change the pooling to avoid overlapping kernel.

9

Hi,
Thank you for your help.
So, when I understand you correctly, The output maxpool2d has overlapping indices, and when I do unpool, these lead to different results.

10

Yes it can.
When during the pooling, a value was used twice (because kernels overlap).
In the unpooling layer, you want to write two different values back at that spot.
And for speed reasons, we don’t check all the Tensors for overlaps and so you might end up with one value or the other depending on the runs.