Creating ANY vectors/tensors to gpu directly

I need to create a random vector with a specific distribution each iteration and run it in gpu/cuda. I know one can do x.cuda() on it but that seems rather slow based on me playing around with it. Is there a better way to do this so that it starts of in GPU or something of that sort? Especially cuz Im doing this every iteration.

Obviously if it generalizes it such that any vector/tensor I create are directly set to CUDA that would be very nice.

Note I want it to make sure it does not screw up my dataloader. For example I am aware that:


might screw up my data loader.

depends on the distribution. what’s the distribution you need?

MultiVariateGaussian. Thats not the same as torch.normal right?

# sample b dim n multivariate gaussian rvs
L = sigma.potrf(upper=False)
z = torch.cuda.DoubleTensor(b, n).normal_()
samples = mu + L.matmul(z.unsqueeze(-1)).squeeze(-1)

just sample or rsample from a MultivariateNormal should also just work…

why is the answer to my question dependent on which type of distribution I use? Why doesn’t pytorch just put it in GPU inside by itself or something like that? Im confused.

You can generally create tensors on GPUs. I just asked so I can write you the code you need to sample it directly on GPU.

1 Like

Sorry for being dense, but how do you generalize the code you gave me to any distribution? Thanks so much for your patience and help SimonW, your awesome! :smiley:

For now use this pattern torch.cuda.*Tensor(*shape).inplace_sampling_method_here_(). Here are the available inplace sampling methods: Notice that these are the basic building block distributions. You can use the results to generate samples of more complex distributions, e.g. multivariate Gaussian.

In the next version we will have dtype in tensor factory methods, so you can just do things like torch.randn(3, 4, dtype=torch.cuda.double).

1 Like

link to torch.distributions.multivariate_normal.MultivariateNormal

(somehow its not easy to find via Google :confused:)

The search function on the doc website is better.

I know this is just adding noise to this answer but I’ll post anyway…I of course tried it, if I recall correctly I couldn’t find it either that easily.

I just tried before I wrote the replay :slight_smile: Maybe you can try again.

still not working unless I have a really dum typo:

Oh I was searching the master doc :slight_smile: I guess 0.3.1 didn’t have the best doc for distributions then.

whats the master doc?

Simon sorry to bug you but this multivariate normal thing still doesn’t work:

AttributeError: module 'torch.distributions' has no attribute 'MultivariateNormal'

note the other error:

AttributeError: module 'torch.distributions' has no attribute 'multivariate_normal'

Yes, it doesn’t work in 0.3.1. I just checked the doc. It is only on master, but not in 0.3.1.