Difference between torch::Tensor and at::Tensor

C++
1

I dont quite get the difference between those two tensor entities. I roughly remember to have read somewhere that at::Tensor should best be avoided for some reason I cannot recall, but what is the general advise here?
thanks in advance

2 Likes
2

at::Tensor is not differentiable while torch::Tensor is. It was similar to the difference between Variables and pure tensors in Python pre 0.4.0.
As far as I know torch::Tensors won’t have any overhead in using them even if you don’t need to differentiate them, so that might be the reason to prefer the torch namespace for creating tensors.

9 Likes
3

Hi,

I feel like even in “official” code those two are used interchangeably (inconsitently?). Lets take for example cannonical tutorial https://pytorch.org/tutorials/advanced/cpp_extension.html#writing-the-c-op

Header of forward pass returns std::vector<at::Tensor> (and where is the casting inside?):

std::vector<at::Tensor> lltm_forward(
    torch::Tensor input,
    torch::Tensor weights,
    torch::Tensor bias,
    torch::Tensor old_h,
    torch::Tensor old_cell)

and then later header of backward pass returns std::vector<torch::Tensor>

std::vector<torch::Tensor> lltm_backward(
    torch::Tensor grad_h,
    torch::Tensor grad_cell,
    torch::Tensor new_cell,
    torch::Tensor input_gate,
    torch::Tensor output_gate,
    torch::Tensor candidate_cell,
    torch::Tensor X,
    torch::Tensor gate_weights,
    torch::Tensor weights)

Regards,

1 Like
4

Another example are brand new cpu ops in torchvision:


written purely using at::Tensor (actually I even see also TH/TH.h headers here, but no torch::Tensor)

5

Ok guys, I still do not understand about this torch::Tensor vs at::Tensor stuff, Will bad things happen if I use at::Tensor instead of torch::Tensor ? d

2 Likes
6

In case someone stumble on this thread, see also comment

2 Likes