# Simple way to inverse transform ? Normalization

**URL:** <https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821>\
**Category:** vision\
**Created:** [July 12, 2017, 4:12pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821 "2017-07-12T16:12:35Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![tsterin](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/c6cbf5/32.png) [@tsterin](https://discuss.pytorch.org/u/tsterin)\
**Post date:** [July 12, 2017, 4:12pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/1 "2017-07-12T16:12:35Z")

</div>

Hi all!  
I’m using torchvision.transforms to normalize my images before sending them to a pre trained vgg19.  
Therefore I have the following:

```
normalize = transforms.Normalize(mean = [0.485, 0.456, 0.406],
                      std = [0.229, 0.224, 0.225])

```

My process is generative and I get an image back from it but, in order to visualize, I’d like to “un-normalize” it.  
Is there a simple way, in the API, to inverse the normalize transform ?  
Or should it be coded by hand ?

Also I’m a bit surprise that the process works really fine without any normalization step.  
The whole thing is about style transfer, from this paper: [https://arxiv.org/abs/1508.06576](https://arxiv.org/abs/1508.06576), and there’s a nice pytorch implementation outhere (not mine) here: [https://github.com/alexis-jacq/Pytorch-Tutorials](https://github.com/alexis-jacq/Pytorch-Tutorials).

That implementation doesn’t normalize anything before feeding images to vgg19 and the results are OK.  
Basically vgg19 is used to extract features from feeded images.  
Your thoughts on why it stills works ?

---

<div class="post-metadata">

**Author:** ![peter\_fox](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/d78d45/32.png) [@peter\_fox](https://discuss.pytorch.org/u/peter_fox)\
**Post date:** [August 31, 2017, 11:51am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/2 "2017-08-31T11:51:05Z")

</div>

did you find any solution for your problem yet? I also normalize before training, to get better loss values, but the generated images look very dark and in terms of colors strange.

---

<div class="post-metadata">

**Author:** ![joels](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/c68b51/32.png) [@joels](https://discuss.pytorch.org/u/joels)\
**Post date:** [September 12, 2017, 2:24am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/3 "2017-09-12T02:24:35Z")

</div>

Hey  
A way to reverse the normalization does not seem to exist. However it is pretty straightforward to create a simple class that does so.

```
class UnNormalize(object):
    def __init__ (self, mean, std):
        self.mean = mean
        self.std = std

    def __call__ (self, tensor):
        """
        Args:
            tensor (Tensor): Tensor image of size (C, H, W) to be normalized.
        Returns:
            Tensor: Normalized image.
        """
        for t, m, s in zip(tensor, self.mean, self.std):
            t.mul_(s).add_(m)
            # The normalize code -> t.sub_(m).div_(s)
        return tensor

```

You instantiate it with the same arguments used for the normalize. and then use it the same way

```
unorm = UnNormalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))
unorm(tensor)
```

---

<div class="post-metadata">

**Author:** ![svd3](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/eb8c5e/32.png) [@svd3](https://discuss.pytorch.org/u/svd3)\
**Post date:** [March 22, 2018, 9:33pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/4 "2018-03-22T21:33:37Z")

</div>

Most easiest way would be:

```auto
invTrans = transforms.Compose([transforms.Normalize(mean = [ 0., 0., 0.],
                                                     std = [1/0.229, 1/0.224, 1/0.225]),
                                transforms.Normalize(mean = [-0.485, -0.456, -0.406],
                                                     std = [1., 1., 1.]),
                               ])

inv_tensor = invTrans(inp_tensor)

```

---

<div class="post-metadata">

**Author:** ![mjust.lkc](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mjust.lkc/32/38185_2.png) [@mjust.lkc](https://discuss.pytorch.org/u/mjust.lkc)\
**Post date:** [May 19, 2018, 9:20am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/5 "2018-05-19T09:20:12Z")

</div>

A more concise approach based on Saurabh’s answer:

```auto
inv_normalize = transforms.Normalize(
    mean=[-0.485/0.229, -0.456/0.224, -0.406/0.255],
    std=[1/0.229, 1/0.224, 1/0.255]
)
inv_tensor = inv_normalize(tensor)

```

---

<div class="post-metadata">

**Author:** ![svd3](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/eb8c5e/32.png) [@svd3](https://discuss.pytorch.org/u/svd3)\
**Post date:** [August 8, 2018, 1:25am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/6 "2018-08-08T01:25:04Z")

</div>

You’ve adjusted the mean wrongly  
the mean should be  
`mean=[-0.485 * 0.229, -0.456 * 0.224, -0.406 * 0.255]`

because the renormalization needs the answer to be (X/sigma - mu) = (X - mu\*sigma)/sigma

---

<div class="post-metadata">

**Author:** ![mjust.lkc](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mjust.lkc/32/38185_2.png) [@mjust.lkc](https://discuss.pytorch.org/u/mjust.lkc)\
**Post date:** [August 8, 2018, 5:59pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/7 "2018-08-08T17:59:53Z")

</div>

The inverse normalization should be  
x = z\*sigma + mean  
= (z + mean/sigma) \* sigma  
= (z - (-mean/sigma)) / (1/sigma),  
since the normalization process is actually z = (x - mean) / sigma if you look carefully at the documentation of `transforms.Normalize`. I tested my original code and it worked fine.

---

<div class="post-metadata">

**Author:** ![dizcza](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dizcza/32/14730_2.png) [@dizcza](https://discuss.pytorch.org/u/dizcza)\
**Post date:** [October 10, 2018, 4:48pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/8 "2018-10-10T16:48:34Z")

</div>

Keep in mind, `torchvision.transforms.Normalize`'s operations are _in-place_. If you want immutable implementation,

```auto
class NormalizeInverse(torchvision.transforms.Normalize):
    """
    Undoes the normalization and returns the reconstructed images in the input domain.
    """

    def __init__ (self, mean, std):
        mean = torch.as_tensor(mean)
        std = torch.as_tensor(std)
        std_inv = 1 / (std + 1e-7)
        mean_inv = -mean * std_inv
        super(). __init__ (mean=mean_inv, std=std_inv)

    def __call__ (self, tensor):
        return super(). __call__ (tensor.clone())

```

---

<div class="post-metadata">

**Author:** ![hongsukchoi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hongsukchoi/32/9461_2.png) [@hongsukchoi](https://discuss.pytorch.org/u/hongsukchoi)\
**Post date:** [March 18, 2019, 12:57am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/9 "2019-03-18T00:57:07Z")

</div>

Thank you.  
But there is a typo in the third element of mean.  
It’s 0.225 not 0.255

```auto
inv_normalize = transforms.Normalize(
    mean=[-0.485/0.229, -0.456/0.224, -0.406/0.225],
    std=[1/0.229, 1/0.224, 1/0.255]
)
inv_tensor = inv_normalize(tensor)

```

---

<div class="post-metadata">

**Author:** ![nofreewill](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nofreewill/32/37853_2.png) [@nofreewill](https://discuss.pytorch.org/u/nofreewill)\
**Post date:** [February 5, 2020, 7:52pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/11 "2020-02-05T19:52:26Z")

</div>

There is still a typo in the std 😃

```auto
inv_normalize = transforms.Normalize(
   mean=[-0.485/0.229, -0.456/0.224, -0.406/0.225],
   std=[1/0.229, 1/0.224, 1/0.225]
)
inv_tensor = inv_normalize(tensor)

```

---

<div class="post-metadata">

**Author:** ![edgarriba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/edgarriba/32/57_2.png) [@edgarriba](https://discuss.pytorch.org/u/edgarriba)\
**Post date:** [February 5, 2020, 8:18pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/12 "2020-02-05T20:18:34Z")

</div>

you can also use `kornia.denormalise`  
[https://kornia.readthedocs.io/en/latest/color.html#kornia.color.Denormalize](https://kornia.readthedocs.io/en/latest/color.html#kornia.color.Denormalize)

---

<div class="post-metadata">

**Author:** ![Ksalomon](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ksalomon/32/23396_2.png) [@Ksalomon](https://discuss.pytorch.org/u/Ksalomon)\
**Post date:** [April 29, 2020, 12:44am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/13 "2020-04-29T00:44:21Z")

</div>

> [@tsterin](#):
>
> normalize = transforms.Normalize(mean = [0.485, 0.456, 0.406], std = [0.229, 0.224, 0.225])

A slightly modified version of the above code

This suppose a defined mean and std.

```auto
inv_normalize = transforms.Normalize(
   mean= [-m/s for m, s in zip(mean, std)],
   std= [1/s for s in std]
)

inv_tensor = inv_normalize(tensor)

```

---

<div class="post-metadata">

**Author:** ![shwinshaker](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shwinshaker/32/50475_2.png) [@shwinshaker](https://discuss.pytorch.org/u/shwinshaker)\
**Post date:** [July 31, 2020, 6:34am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/14 "2020-07-31T06:34:31Z")

</div>

Sorry to bother, but this doesn’t seem to work for me. As `inv_normalize(tensor)` will raise a `TypeError` complaining `tensor is not a torch image.`

Is this a version issue? Thanks in advance!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 1, 2020, 7:10am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/15 "2020-08-01T07:10:23Z")

</div>

Could you check the shape and type of the input you are passing to this method?  
I guess the number of dimensions of type might be unexpected and thus `Normalize` might raise this issue.

---

<div class="post-metadata">

**Author:** ![yptheangel](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yptheangel/32/20966_2.png) [@yptheangel](https://discuss.pytorch.org/u/yptheangel)\
**Post date:** [October 16, 2020, 1:45pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/16 "2020-10-16T13:45:44Z")

</div>

following @joels method, it worked for me.

```auto
def inverse_normalize(tensor, mean, std):
    for t, m, s in zip(tensor, mean, std):
        t.mul_(s).add_(m)
    return tensor

input_tensor= inverse_normalize(tensor=input_tensor, mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))

```

---

<div class="post-metadata">

**Author:** ![yptheangel](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yptheangel/32/20966_2.png) [@yptheangel](https://discuss.pytorch.org/u/yptheangel)\
**Post date:** [November 7, 2020, 6:36am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/17 "2020-11-07T06:36:33Z")

</div>

I found out that even after performing the inverse normalization, the **inverse normalized opencv imag is not equivalent to an original opencv via `cv2.imread`**. I am sticking to getting a mask from pytorch model output and multiplying it with original frame via `cv2.imread`

---

<div class="post-metadata">

**Author:** ![hustzwen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hustzwen/32/36411_2.png) [@hustzwen](https://discuss.pytorch.org/u/hustzwen)\
**Post date:** [July 2, 2021, 4:43am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/18 "2021-07-02T04:43:13Z")

</div>

like this,

```auto
def inverse_normalize(tensor, mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)):
    mean = torch.as_tensor(mean, dtype=tensor.dtype, device=tensor.device)
    std = torch.as_tensor(std, dtype=tensor.dtype, device=tensor.device)
    if mean.ndim == 1:
        mean = mean.view(-1, 1, 1)
    if std.ndim == 1:
        std = std.view(-1, 1, 1)
    tensor.mul_(std).add_(mean)
    return tensor

```

---

<div class="post-metadata">

**Author:** ![Apogentus](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/apogentus/32/15014_2.png) [@Apogentus](https://discuss.pytorch.org/u/Apogentus)\
**Post date:** [July 5, 2021, 10:55am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/19 "2021-07-05T10:55:27Z")

</div>

Works like charm. Normalization+unnormalization give original image.

---

<div class="post-metadata">

**Author:** ![Giorgio](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/cab0a1/32.png) [@Giorgio](https://discuss.pytorch.org/u/Giorgio)\
**Post date:** [September 30, 2021, 11:04am UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/20 "2021-09-30T11:04:30Z")

</div>

this code seems simpler

If you want to reverse the normalization, all you need to do is to use a new normalization, with slight modifications:

```auto
mean = torch.tensor([0.4915, 0.4823, 0.4468])
std = torch.tensor([0.2470, 0.2435, 0.2616])

normalize = transforms.Normalize(mean.tolist(), std.tolist()) 

unnormalize = transforms.Normalize((-mean / std).tolist(), (1.0 / std).tolist())
img_unn = unnormalize(img)

plt.imshow(img_unn.permute(1, 2, 0))
plt.show()

```

Anyway I hate the fact that we don’t have un-normalize (de-normalize), it would be quite helpful for regression neural networks too.

> <https://github.com/pytorch/vision/issues/528>
>
> Basically the inverse of \`transforms.Normalize\` as this will allow us to visuali…ze tensors during training more easily.

---

<div class="post-metadata">

**Author:** ![guillermogotre](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/guillermogotre/32/47737_2.png) [@guillermogotre](https://discuss.pytorch.org/u/guillermogotre)\
**Post date:** [March 20, 2022, 3:48pm UTC](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821/21 "2022-03-20T15:48:28Z")

</div>

Here is some working code as this is still one of the first Google results in inverse normalization. It follows torchvision.transforms convention so you can compose it.

[Code Gist](https://gist.github.com/guillermogotre/844024ac37d35c8a00a6133887cbd18b)

```auto
class UnNormalize(torchvision.transforms.Normalize):
    def __init__ (self,mean,std,*args,**kwargs):
        new_mean = [-m/s for m,s in zip(mean,std)]
        new_std = [1/s for s in std]
        super(). __init__ (new_mean, new_std, *args, **kwargs)

```

[Next page](https://discuss.pytorch.org/t/simple-way-to-inverse-transform-normalization/4821.md?page=2)