I was looking that there is are Variables and Tensors. Though I don’t quite understand why we need two structures that do essentially what looks to me, the same thing. The docs even say:
Variable API is nearly the same as regular Tensor API (with the exception of a couple in-place methods, that would overwrite inputs required for gradient computation). In most cases Tensors can be safely replaced with Variables and the code will remain to work just fine.
As far as I can tell from reading the docs is that Variables are part of the autograd package thus they can perform gradient computations and stuff. The only clear distinction to me seems to have torch tensors as “placeholders” for data. But variables can already sort of do that by setting the requires_grad=False to false. Thus it just seems odd. Why not only have Tensors and thats it nothing else and avoid making the API more convoluted and confusing than it needs to be (even the docs for Variable aren’t complete cuz they say to go read Tensors API). I am just giving my guesses, I assume there is probably some reason the code is organized as it is and would like to understand it a bit more so that I wrap my head around pytorch (coming from a matlab/tensorflow background).
variables wrap tensors, and construct a chain of operations between teh tensors, so taht the gradietns can flow back.
so, eg you create variable a, and then add 1 to it to get b. Theres now a link stored between a and b, in the creator property of b. Then when you call .backward() on b, the gradient backpropagates, via the function in b.creator into a.
tensors dont have the concept of gradients, creator etc. they purely store data, and handle operations on that data, like adding, multiplying nad stuff.
The Variable class is a wrapper over torch Tensors (nd arrays in torch) that supports nearly all operations defined on tensors. PyTorch requires that the input tensor to be forward propagated has to be wrapped in a Variable. This facilitates automatic back propagation by simply calling the method backward() in the Variable class.
