Copy_() operation not copying values

#1

Hi, at some part of my project, copy_() operation struggled me a bit,
thus I tried to make a toy example as below but could not find a solution for it.

import torch

class A():
    def __init__(self):
        self.tensor = torch.zeros((5, 6), dtype=torch.float)

    def check(self, b):
        self.tensor[[2, 4]].copy_(b.tensor)
        print(self.tensor)

class B():
    def __init__(self, tensor):
        self.tensor = tensor

m = torch.ones((2, 6), dtype=torch.float)

a = A()
b = B(m)

a.check(b)

As written, this simple code is just to copy 1.0s to 0.0 at specific locations,
and what I get is only full of zeros, which means copy_() operation did not work.

tensor([[0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.]])

below code which uses assigning operation = instead of copy_() works

def check(self, b):
    self.tensor[[2, 4]] = b.tensor
    print(self.tensor)

tensor([[0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.]])

and moreover, below code which is not involved in the use of classes works fine

c = torch.zeros((5, 6), dtype=torch.float)
d = torch.ones((2, 6), dtype=torch.float)

c[[2, 4]] = d
print(c)

tensor([[0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.],
        [0., 0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1., 1.]])

I am hoping to fully understand the reason of not copying the values at the first toy example
(I tried on PyTorch 1.6.0 and 1.7.0)

#2

Hi,

The problem is that advanced indexing like [[2, 4]] does not always return a view into the original Tensor. It sometimes returns a different Tensor. And so when you do tensor[[2, 4]].copy_(b.tensor), you actually do the copy into a Tensor that is completely independent of tensor.

For this to work, you can do tensor[[2, 4]] = b.tensor so that the advanced indexing will be used in “assignment mode” and pytorch will do the right thing.

1 Like
#3

Thank you for the great answer!

I am still a bit concerned with using the copy operations as I don’t know what the conditions for returning a different tensor and would like to seek for it later on

Thank you :slight_smile: