Generating random tensors according to the uniform distribution pytorch?

1

I saw:

and thought that was a strange way to do it.

So there isn’t something default like torch.uniform?

their code:

If U is a random variable uniformly distributed on [0, 1], then (r1 - r2) * U + r2 is uniformly distributed on [r1, r2].

Thus, you just need:

(r1 - r2) * torch.rand(a, b) + r2
Alternatively, you can simply use:

torch.FloatTensor(a, b).uniform_(r1, r2)
2 Likes
2

toch.rand returns a tensor samples uniformly in [0, 1). Scaling it as shown in your example should work.
Also, the second approach is fine.

If you would like to have a standalone torch.uniform method, I would recommend to create an issue in github and start the discussion there. :wink:

2 Likes
3

ok! My first time doing this, hopefully this is the right way to do it:

1 Like
4

Hello,

yes this is true. According to the PyTorch documentation:
torch.rand returns a tensor filled with random numbers from a uniform distribution on the interval [0,1)
https://pytorch.org/docs/stable/torch.html#torch.rand
I would appreciate if you can explain the following. I am trying to figure out how the authors of the papers generate normal and uniform distribution:
“Gaussian Noise. The synthetic Gaussian noise dataset consists of 10,000 random 2D Gaussian
noise images, where each RGB value of every pixel is sampled from an i.i.d Gaussian distribution
with mean 0.5 and unit variance. We further clip each pixel value into the range [0, 1].
Uniform Noise. The synthetic uniform noise dataset consists of 10,000 images where each RGB
value of every pixel is independently and identically sampled from a uniform distribution on [0, 1].”

In the code they seemed to use this for generating Gaussian dataset:

        images = torch.randn(1, 3, 32, 32) + 0.5
        images = torch.clamp(images, 0, 1)

and this for generating Uniform dataset:

       images = torch.rand(1, 3, 32, 32)

According to your explanation the generation of the Uniform dataset is being correct.
What I would like to know is how they used the rand for the Gaussian dataset, although this discussion aligns with what they did:


A simple option is to use the randn function from the base module. It creates a random sample from the standard Gaussian distribution. To change the mean and the standard deviation you just use addition and multiplication. Below I create sample of size 5 from your requested distribution. 
import torch
torch.randn(5) * 0.5 + 4 # tensor([4.1029, 4.5351, 2.8797, 3.1883, 4.3868])

I am still little confused about how they could use “rand” function for generating Gaussian images?
Thank you

5

They are not using rand but randn, which is the normal (gaussian) distribution.

1 Like
6

I find this very confusing naming. Why have rand and randn? It’s not very descriptive names of what they are.

For example, I assumed the default way to sample was normal for a long time…

7

Anyway, do you mind providing a good example of how to use the torch.distributions.Uniform interface?

I saw the docs and the example in the linked issue but wasn’t very helpful. In particular what passing tensors of size larger than 1 do?

my specific question can be found here: Implement torch.uniform · Issue #24162 · pytorch/pytorch · GitHub

copy pasted:

I think this already exists?

a = torch.tensor([0., 10.])
b = torch.tensor([1., 11.])
torch.distributions.Uniform(a, b).sample()
>>> tensor([ 0.8583, 10.0226])


This example is confusing me. What does uniform do when receiving tensors of size larger than 1x1 mean? Whats the behavior when they are greater than 1 dimension?

I saw the docs but they weren't helpful nor any example or discussion of the case you provides is given: https://pytorch.org/docs/stable/distributions.html#torch.distributions.uniform.Uniform

What I want is to replicate this numpy code:

xn = np.random.uniform(low=-1, high=1, size=(num_samples,Din))

Also what is the use of creating a sampler object first?

8

This is the way I found works:

# generating uniform variables

import numpy as np

num_samples = 3
Din = 1
lb, ub = -1, 1

xn = np.random.uniform(low=lb, high=ub, size=(num_samples,Din))
print(xn)

import torch

sampler = torch.distributions.Uniform(low=lb, high=ub)
r = sampler.sample((num_samples,Din))

print(r)

r2 = torch.torch.distributions.Uniform(low=lb, high=ub).sample((num_samples,Din))

print(r2)

# process input
f = nn.Sequential(OrderedDict([
    ('f1', nn.Linear(Din,Dout)),
    ('out', nn.SELU())
]))
Y = f(r2)
print(Y)

but I have to admit I don’t know what the point of generating sampler is and why not just call it directly as I do in the one liner (last line of code).

Comments:

Reference:

9

These function names are taken from numpy, which adapted them from MATLAB, if I’m not mistaken, so they are commonly used in scientific software.

Good to hear your Uniform distribution works. :slight_smile:

10

Thanks for the reply!

I think what puts me off is that the interface isn’t that similar to numpy. e.g. even if those names are the same there is a .random to call them in numpy but there is not in pytorch which is where my confusion and me searching for uniform came about.

For numpy it’s numpy.random.rand or randn while in pytorch you can just do torch.randn or rand and for uniform one has to do torch.distribution.uniform.Uniform or distribution.Distribution.

Anyway, thanks for clarifying. :slight_smile:

Just curious, why doesn’t it match numpy more closely?

11

I think these were early design decisions, which were already introduced in Torch7 (Lua).
Once these functions are commonly used, it’s hard to change them due to backwards compatibility. Especially once you reach a non-beta version.

12

Thanks for taking the time to explain. :brain:

Cheers!