The size of images working with Unet

1

Hi

I have a question about Unet. I have a 3D dataset, in which each volume is of size 112x40x40. For my convenient I resized the volumes (using interpolation) to the size 128x128x128 and trained a Unet for segmentation.

Now, for some reasons I want to preserve the original size of volumes, is there any way that I can train Unet with the original dimensions 112x40x40?

My thought is that the spatial dimension of images should be of power 2 when we are working with Unet. Is that correct?

If it is not possible that train Unet with the original size of volumes, is there any other way than interpolation and zero padding that I can resize volumes to size 128x128x128?

Thank you in advance.

Best

2

It depends on the implementation and how flexible the model architecture is to arbitrary input shapes.
While powers of two are often easier to handle, you could certainly write a more “robust” model accepting more input shapes e.g. by padding/slicing the activations if needed.

3

Thank you. Do you know any GitHub repository that has such flexible implementations of Unet ?

4

No, I don’t know a specific UNet implementation with a focus on variable input shapes.
However, once you’ve found a “good” implementation you could check which shapes would work, which would fail, and adapt the code if needed.
E.g. transposed conv layers give you the option to specify the output_size in case multiple output sizes would be possible and you could of course add padding etc. based on shape conditions.
Generally I see that models are written for specific use cases and not “universal” workloads. E.g. if the author of a model expects to work with input images using a static shape of 224x224, I don’t think much work will be spent on making the model flexible for other inputs.

1 Like
5

I have implemented a 2d UNET which retains image dim:

have a look the differences between a standard one and the preserve_dim model. Can be easily converted to 3d by changing layers to conv3d…

2 Likes
6

Thank you for your answer. Just one more question, if we use zero padding instead of interpolation to resize the images (from 112 to 128) dose it affect the performance of Unet in terms of Dice score for example ?

In my experiments I just resized the images using zero padding instead of interpolation and I trained the Unet. The dice with zero padding was 0.6807 while with interpolation it was 0.8304. I am wondering if this drop in performance is natural or I am doing something wrong. Couldn’t find anything in net.

By resizing with zero padding, I mean resizing the volumes of size (112x40x40) to (128x128x128). Just one point is that. the dataset that I am working with it is very small and imbalanced. I am trying to detect some signals from noise where in each volume we have less than 0.1 percent signal. Could it be a reason that model dose not work with zero padding?

Thank you.

7

Thank you so much.

I would take a look.

8

The drop in performance sounds reasonable, since your inputs would only contain:

(112*40*40) / (128*128*128) * 100
~ 8.54%

valid data while the rest of the input values doesn’t contain any information.

1 Like
9

Ok, thank you. So if I use a Unet with the original size of images I mean the size (112x40x40) then I should expect the same performance as zero padding in terms of Dice score, am I right?

Thank you for all your inputs.

10

I looked at your Github, I quickly tried to check it on the example that you gave. Pytorch gave me an error about padding =same, it expects an integer or tuple not str. It shouldn’t be like that but I have this error.

By the way, dose your model also support the dimensions that are not equal?

I mean : hight is different from width.

11

Should work for dimensions that are not equal, dont really see why it might fail. However i made the model according to dimensions in the paper.

With the error on padding = 'same' not sure why that’s there since conv3d also supports padding=same. Can you share the error message

12

I tried the 2d version of your code that I get the error of the padding='same'. I posted the error here.
Can be the version of pytorch be a problem?

`TypeError Traceback (most recent call last)
in
1 # testing
2 x = torch.randn((1,1,572, 572))
----> 3 pred = model(x)
4 pred.shape

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
→ 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),

in forward(self, x)
25 # CONTRACTION
26 # down 1
—> 27 x = self.down1(x)
28 skip_connections.append(x)
29 x = self.pool(x)

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
→ 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),

in forward(self, x)
10
11 def forward(self, x):
—> 12 return self.process(x)

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
→ 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/container.py in forward(self, input)
115 def forward(self, input):
116 for module in self:
→ 117 input = module(input)
118 return input
119

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
→ 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/conv.py in forward(self, input)
421
422 def forward(self, input: Tensor) → Tensor:
→ 423 return self._conv_forward(input, self.weight)
424
425 class Conv3d(_ConvNd):

~/anaconda3/envs/vir-env/lib/python3.8/site-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
417 weight, self.bias, self.stride,
418 _pair(0), self.dilation, self.groups)
→ 419 return F.conv2d(input, weight, self.bias, self.stride,
420 self.padding, self.dilation, self.groups)
421

TypeError: conv2d(): argument ‘padding’ (position 5) must be tuple of ints, not str

`

13

You might need to update your PyTorch version, as this padding option is relatively new.

14

Ok, thank you very much.