# Understanding transform.Normalize( )

**URL:** <https://discuss.pytorch.org/t/understanding-transform-normalize/21730>\
**Category:** vision\
**Created:** [July 25, 2018, 11:09pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730 "2018-07-25T23:09:50Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![Rui\_Li](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rui_li/32/13938_2.png) [@Rui\_Li](https://discuss.pytorch.org/u/Rui_Li)\
**Post date:** [October 17, 2019, 3:17am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/21 "2019-10-17T03:17:06Z")

</div>

It really helps, thanks~

---

<div class="post-metadata">

**Author:** ![snip3r77](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/65b543/32.png) [@snip3r77](https://discuss.pytorch.org/u/snip3r77)\
**Post date:** [January 2, 2020, 7:29am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/22 "2020-01-02T07:29:55Z")

</div>

So if we encounter grayscale images, we will use

```auto
transforms.Normalize([0.5], [0.5])

```

and if we encounter RGB( 3 channels ) , we will use the following

```auto
transforms.Normalize(mean=[0.485, 0.456, 0.406],
                     std=[0.229, 0.224, 0.225]) 

```

---

<div class="post-metadata">

**Author:** ![Hong\_Cheng](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hong_cheng/32/10639_2.png) [@Hong\_Cheng](https://discuss.pytorch.org/u/Hong_Cheng)\
**Post date:** [January 6, 2020, 2:29am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/23 "2020-01-06T02:29:37Z")

</div>

here is my implementation of the custom `ToTensor()` and `Normlization()` method, which is called `ToTensor_Norm()` .

```auto
class ToTensor_Norm(object):
    """Convert ndarrays in sample to Tensors."""
    def __init__ (self, mean, std):
        self.mean = mean
        self.std = std

    def __call__ (self, sample):
        image, pls, tr_angles, positions = \
            sample['image'], sample['pls'], sample['tr_angles'], sample['positions']
        image = image[np.newaxis, :, :]
        # swap color axis because
        # numpy image: H x W x C
        # torch image: C X H X W
        # image = image.transpose((2, 0, 1)) # single channel
        image = torch.from_numpy(image)
        for t, m, s in zip(image, self.mean, self.std):
            t.sub_(m).div_(s)

        return {'image': image,
                'pls': torch.from_numpy(pls),
                'tr_angles': torch.from_numpy(tr_angles),
                'positions': torch.from_numpy(positions)
                }

transformed_dataset = PLDataset(csv_file=csv_file,
                                    root_dir=root_dir,  
                                    transform = transforms.Compose([ToTensor_Norm([5.50180734], [8.27773514])]),
                                    # transform=transforms.Compose([ToTensor()])
                                    )

```

---

<div class="post-metadata">

**Author:** ![rajanala](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rajanala/32/18947_2.png) [@rajanala](https://discuss.pytorch.org/u/rajanala)\
**Post date:** [April 2, 2020, 3:37am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/24 "2020-04-02T03:37:34Z")

</div>

I was also having same doubt…i am learning pytorch . Normalise depends on the number of channels. if MNIST its grey scale with only one channel . so you can do …transforms.Normalize((0.5,), (0.5,))… If three channel, you may need to specify for all channels for example : CIFAR10.

---

<div class="post-metadata">

**Author:** ![achyut-srivastava](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/achyut-srivastava/32/29600_2.png) [@achyut-srivastava](https://discuss.pytorch.org/u/achyut-srivastava)\
**Post date:** [June 16, 2020, 6:09pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/25 "2020-06-16T18:09:08Z")

</div>

You have to divide by 255 and then you can proceed with transforms.Normalize()

---

<div class="post-metadata">

**Author:** ![Nikronic](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/32/71873_2.png) [@Nikronic](https://discuss.pytorch.org/u/Nikronic)\
**Post date:** [July 8, 2020, 2:21pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/26 "2020-07-08T14:21:32Z")

</div>

Just note that you need to use your own mean and std if your dataset is not similar to ImageNet. In the 3-channel case you have mentioned, you are using mean and std from ImageNet which works for most of the datasets that are similar but if you are using datasets such as medical image processing, then you need to obtain proper mean and std regarding your own dataset.

---

<div class="post-metadata">

**Author:** ![SiddharthSingi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/siddharthsingi/32/26161_2.png) [@SiddharthSingi](https://discuss.pytorch.org/u/SiddharthSingi)\
**Post date:** [July 17, 2020, 1:11pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/27 "2020-07-17T13:11:26Z")

</div>

@bhushans23 , @InnovArul  
In this case we are transforming from [0,1] to [-1,1] using normalization.

Normalization usually however means to subtract each data point with the dataset mean, and then divide by the datasets standard deviation. In our case if you were to consider the dataset to be 11 numbers from [0,1], i.e `(0.0, 0.1, 0.2, ....0.9, 1.0)` its mean=0.5 but its stddev=0.316

We use `transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])`. That is mean=0.5, stddev=0.5 for all three channels.

Can someone please explain as to how exactly did we get to these numbers and what would one do if the image is in the range(0,255)?

---

<div class="post-metadata">

**Author:** ![Nikronic](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/32/71873_2.png) [@Nikronic](https://discuss.pytorch.org/u/Nikronic)\
**Post date:** [July 17, 2020, 7:54pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/28 "2020-07-17T19:54:49Z")

</div>

Hi,

There is main difference here. If you use `mean=0.5` and `std=0.5`, your output value will be between `[-1, 1]` based on the normalization formula : `(x - mean) / std` which also called z-score. This score can be out of [-1, 1] when used mean and std of dataset, for instance ImageNet.  
The definition says that we need to use population mean and std but it is usually unavailable, sample mean/std can be used as an estimation.

> **[Standard score](https://en.wikipedia.org/wiki/Standard_score)**
>
> In statistics, the standard score is the number of standard deviations by which the value of a raw score (i.e., an observed value or data point) is above or below the mean value of what is being observed or measured. Raw scores above the mean have positive standard scores, while those below the mean have negative standard scores. 
> It is calculated by subtracting the population mean from an individual raw score and then dividing the difference by the population standard deviation. This process ...

---

<div class="post-metadata">

**Author:** ![SiddharthSingi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/siddharthsingi/32/26161_2.png) [@SiddharthSingi](https://discuss.pytorch.org/u/SiddharthSingi)\
**Post date:** [July 20, 2020, 1:21pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/29 "2020-07-20T13:21:28Z")

</div>

thats helpful, thanks

---

<div class="post-metadata">

**Author:** ![nshrimali](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nshrimali/32/27038_2.png) [@nshrimali](https://discuss.pytorch.org/u/nshrimali)\
**Post date:** [July 23, 2020, 12:44pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/31 "2020-07-23T12:44:27Z")

</div>

I have seen that in training the MNIST dataset, we use transforms.Normalize((0.1307,), (0.3081,)),  
My understanding is we calculate mean of dataset and subtract it from each image,

People directly use the values in their codes, but there is no calculation how these are derived.

Also is there is a way that we can automate this, instead of hard-coding, it calculates the mean of dataset and subtracts it.

Does using batch norm as the first layer of our network would have a similar effect?

---

<div class="post-metadata">

**Author:** ![Rishi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rishi/32/27193_2.png) [@Rishi](https://discuss.pytorch.org/u/Rishi)\
**Post date:** [July 28, 2020, 1:52am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/32 "2020-07-28T01:52:01Z")

</div>

See here the answer by Sowmith himself. I hope thats what your are looking for [Normalization in the mnist example](https://discuss.pytorch.org/t/normalization-in-the-mnist-example/457/7)

---

<div class="post-metadata">

**Author:** ![monster](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/monster/32/24299_2.png) [@monster](https://discuss.pytorch.org/u/monster)\
**Post date:** [August 2, 2020, 7:55pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/33 "2020-08-02T19:55:40Z")

</div>

Is it correct or all values should be positive ?  
mean=[-0.16160992, -0.09780669, 0.44261688]  
std = [1.3066291, 1.3798972, 1.4423286]

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 3, 2020, 2:40am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/34 "2020-08-03T02:40:50Z")

</div>

It depends on your dataset, and if the mean of all samples is negative (which might be the case), then these values look alright.

EDIT: Just to avoid confusion: if you are working with images, which are using `uint8` pixel values, the mean should be positive, since these values cannot get negative values. However, for any other dataset the mean might be whatever makes sense. 😉

---

<div class="post-metadata">

**Author:** ![nik1806](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nik1806/32/27772_2.png) [@nik1806](https://discuss.pytorch.org/u/nik1806)\
**Post date:** [August 14, 2020, 8:00am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/35 "2020-08-14T08:00:26Z")

</div>

The transformation to [-1,1] is performed to keep values center around 0. This helps in faster convergence.

---

<div class="post-metadata">

**Author:** ![said\_rasidin](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/said_rasidin/32/26749_2.png) [@said\_rasidin](https://discuss.pytorch.org/u/said_rasidin)\
**Post date:** [August 16, 2020, 5:47pm UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/36 "2020-08-16T17:47:14Z")

</div>

How about if the values not within range [-1, 1] after normalize, I check the max and min value and out of that range. My transformation:

`transforms.Compose([transforms.ToTensor(), transforms.Normalize(mean=[0.35, 0.35, 0.35], std=0.12, 0.12, 0.12)])`

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 18, 2020, 8:09am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/37 "2020-08-18T08:09:35Z")

</div>

`Normalize` does not necessarily create output values in the range `[-1, 1]`, but the “standard score” as explained by @Nikronic in [this post](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/28).

Do you need the output values to be in a specific range?

---

<div class="post-metadata">

**Author:** ![Hieu\_Nguyen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hieu_nguyen/32/28098_2.png) [@Hieu\_Nguyen](https://discuss.pytorch.org/u/Hieu_Nguyen)\
**Post date:** [August 24, 2020, 9:14am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/38 "2020-08-24T09:14:11Z")

</div>

Hi, I am a newbie in Machine learning. I still wonder why maximum is 1 and minimum is 0?

---

<div class="post-metadata">

**Author:** ![yongen9696](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/e79b87/32.png) [@yongen9696](https://discuss.pytorch.org/u/yongen9696)\
**Post date:** [September 17, 2020, 8:13am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/39 "2020-09-17T08:13:57Z")

</div>

The whole dataset has divided by 255.

---

<div class="post-metadata">

**Author:** ![Mona\_Jalal](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mona_jalal/32/25715_2.png) [@Mona\_Jalal](https://discuss.pytorch.org/u/Mona_Jalal)\
**Post date:** [October 14, 2020, 1:03am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/40 "2020-10-14T01:03:31Z")

</div>

> [@Meltemology](#):
>
> ```auto
> transforms.Normalize(mean=[0.485, 0.456, 0.406],
> std=[0.229, 0.224, 0.225]) 
> 
> ```

Is there a script or piece of code to run on my own images to get the corresponding values similar to below to pass to Normalize method?

```auto
transforms.Normalize(mean=[0.485, 0.456, 0.406],
                     std=[0.229, 0.224, 0.225]) 

```

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 14, 2020, 6:52am UTC](https://discuss.pytorch.org/t/understanding-transform-normalize/21730/41 "2020-10-14T06:52:36Z")

</div>

You could use an approach posted e.g. in [this thread](https://discuss.pytorch.org/t/computing-the-mean-and-std-of-dataset/34949/20).

[Previous page](https://discuss.pytorch.org/t/understanding-transform-normalize/21730.md?page=1)

[Next page](https://discuss.pytorch.org/t/understanding-transform-normalize/21730.md?page=3)