torch.cat will be tracked by Autograd and the backward pass will pass through this operation into the incoming tensors. This is useful in case you want to e.g. concatenate outputs of two different models, calculate the loss, and compute the gradients for both models.
If you don’t want to track this operation, you could .detach() the input tensors.
1 Like