Suppose I have a tensor
t that resides on GPU and I would like to apply some math operations on it with constants:
t = torch.FloatTensor(...).cuda() t = 0.1 * t + 0.2
I wonder how Torch/Python handles the constants
0.2. Are they transferred from CPU to GPU memory each time this operation is performed, or is the code optimized automatically and the constants are moved to GPU?
I am concerned about the efficiency.
Should I define constant CUDA tensors and use them instead (less readable code)?