[solved] RuntimeError: in-place operations can be only used on variables that don't share storage with any other variables

I am getting the following error in a custom layer.

RuntimeError: in-place operations can be only used on variables that don't share storage with any other variables, but detected that there are 2 objects sharing it

Here is the forward function of the nn layer.

def forward(self, batch_sentence1, batch_sentence2):
""""Defines the forward computation of the matching layer."""
sequence_length = batch_sentence1.size(1)
output_variable = Variable(
	torch.zeros(self.config.batch_size, sequence_length, self.num_directions, self.length))

for word_idx in range(sequence_length):
	for batch_idx in range(self.config.batch_size):
		v1 = batch_sentence1[batch_idx][word_idx]
		v2 = batch_sentence2[batch_idx][-1]
		for matching_idx in range(self.length):
			weighted_v1 = torch.mul(self.weight_forward[matching_idx], v1)
			weighted_v2 = torch.mul(self.weight_forward[matching_idx], v2)

			cosine = weighted_v1.dot(weighted_v2)
			cosine = cosine / (torch.norm(weighted_v1, 2) * torch.norm(weighted_v2, 2))
			output_variable[batch_idx][word_idx][0][matching_idx] = cosine

Getting the error in the last line. I have checked if the output_variable shares storage with other object but couldn’t find any.

Can anyone point me to the problem in my code?

1 Like

I tried the following after noticing this post - What's the difference between a[0][1] and a[0 , 1] and it worked for me.

output_variable[batch_idx, word_idx, 0, matching_idx] = cosine

So, I am just curious to know why pytorch interprets x[i, j] and x[i][j] differently? As soumith mentioned in that post,

When you index with x[i][j], then an intermediate Tensor x[i] is created first, and the operation [j] is applied on it. If you index with x[i, j] then there’s no intermediate operation.

I want to know when we should use x[i][j] and when x[i, j]?

1 Like

always use x[i, j], it’s cheaper too.

1 Like

Wow dude, your finding has supported me a vital solution to my own solution, much thx

hello smth,
I just wonder why there are still a lot of code write in a[i][j] form, rather than a[i,j], though they might know it would waste a batch of room.

@Zichun_Zhang i think that’s just legacy. if you find such instances, please help highlight or fix them