How to Normalize 4d Tensor?

Hi Folks.

I have a tensor of size:
[B, C, dimA, dimB, h, w]

The above tensor is supposed to be a batch of B images, with C channels (currently 1 grayscale channel). dimA and dimB are the result of unfolding each images into smaller blocks and then we have the pixels in each block, h and w.

Using convNd I have applied a 4D convolution. I would now like to apply 4D batch normalization across this tensor. Is this possible? Is there a more efficient alternative?

Thanks guys!

You can try two things

  1. Use view to change the size of tensor to that of 1d and then apply BatchNorm1d. After that again use view.
  2. Define BatchNorm4d. batch norm internally uses torch.batch_norm and it can work on input of any size. But you might not get CUDNN speedup (I am not sure about this).

Thanks for the response! I’ll implement the view method for now. Do I need to use contiguous in conjunction with view() to keep my tensor in the correct order?

I looked up the definition of batch_norm and it seems to have two different functions, one for CPU and one for GPU as you mentioned. They’re also written in C++ which I haven’t worked with in a while!

I’ll report back later.

I was also wondering about contiguous. I don’t think you have to. But you might have to double check this, if you get an error without contiguous.

The 1D normalize solution seems to work for me. Thank you!