Copy_() operation not copying values

Hi, at some part of my project, copy_() operation struggled me a bit,
thus I tried to make a toy example as below but could not find a solution for it.

import torch

class A():
    def __init__(self):
        self.tensor = torch.zeros((5, 6), dtype=torch.float)

    def check(self, b):
        self.tensor[[2, 4]].copy_(b.tensor)
        print(self.tensor)

class B():
    def __init__(self, tensor):
        self.tensor = tensor

m = torch.ones((2, 6), dtype=torch.float)

a = A()
b = B(m)

a.check(b)

As written, this simple code is just to copy 1.0s to 0.0 at specific locations,
and what I get is only full of zeros, which means copy_() operation did not work.

tensor([[0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.]])

below code which uses assigning operation = instead of copy_() works

def check(self, b):
    self.tensor[[2, 4]] = b.tensor
    print(self.tensor)
tensor([[0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.]])

and moreover, below code which is not involved in the use of classes works fine

c = torch.zeros((5, 6), dtype=torch.float)
d = torch.ones((2, 6), dtype=torch.float)

c[[2, 4]] = d
print(c)
tensor([[0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.]])

I am hoping to fully understand the reason of not copying the values at the first toy example
(I tried on PyTorch 1.6.0 and 1.7.0)

Hi,

The problem is that advanced indexing like [[2, 4]] does not always return a view into the original Tensor. It sometimes returns a different Tensor. And so when you do tensor[[2, 4]].copy_(b.tensor), you actually do the copy into a Tensor that is completely independent of tensor.

For this to work, you can do tensor[[2, 4]] = b.tensor so that the advanced indexing will be used in “assignment mode” and pytorch will do the right thing.

1 Like

Thank you for the great answer!

I am still a bit concerned with using the copy operations as I don’t know what the conditions for returning a different tensor and would like to seek for it later on

Thank you :slight_smile: