If the criterion is not set to cuda but the network is set to cuda, does the operation work in cuda?

1

If the criterion is not set to cuda but the network is set to cuda, does the operation work in cuda?

1 Like
2

It depends (if there is no buffer in your loss)
Usually, it works fine.
Here is an example:

import torch as t

# Crossentropy with weight
criterion = t.nn.CrossEntropyLoss(weight=t.Tensor([1, 3]))
input = t.autograd.Variable(t.randn(4, 2)).cuda()
target = t.autograd.Variable(t.Tensor([1, 0, 0, 1])).long().cuda()

# Error, since weight is still in cpu
# loss = criterion(input, target)

# it's ok
criterion.cuda()
loss = criterion(input, target)

print(criterion._buffers)

See celll around In[45] for more detail (it’s tutorials written in Chinese )

2 Likes
3

Anyway, I should write .cuda. Thank you! It helped me a lot. That your pytorch page looks very good. I hope the article is in English. If that writing were English, it would be very good …

4

Thank you. I’ll try it if I have free time. But it would be really busy seeking for interns and jobs this year.

1 Like