In the first example a is the result of the cuda() method call on a user-created Variable. i.e. Variable(torch.randn(2,10), requires_grad=True) is a leaf, but Variable(torch.randn(2,10), requires_grad=True).cuda() is a different variable, and is not.