RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [7616, 3, 3]], which is output 0 of AsStridedBackward0, is at version 12; expected version 11 instead

autograd
1

I’m running into issues during backprop on a function that calculates skeleton joint positions in 3D space from rotation matrices. This function computes the global position of one joint at a time.
Implementation:

def torch_mat_to_pos(mat_rotations: torch.Tensor, parents: List[int], offsets: torch.Tensor) -> torch.Tensor:
    """
    Convert rotation matrices to joint positions.

    Args:
        mat_rotations: Rotation matrices as tensor of shape (frames, joints, 3, 3).
        parents: Parents of each joint as tensor of shape (joints,).
        offsets: Offsets of each joint as tensor of shape (joints, 3).

    Returns:
        Joint positions as tensor of shape (frames, joints, 3).
    """
    global_rot = torch.clone(mat_rotations)
    global_pos = torch.zeros((mat_rotations.shape[0], mat_rotations.shape[1], 3),
                             dtype=mat_rotations.dtype, device=mat_rotations.device)
    for i, parent in enumerate(parents):
        if parent == -1:
            continue

        # multiply this joint's rotmat to the rotmat of its parent
        global_rot[:, i] = mat_rotations[:, i] @ global_rot[:, parent]

        k = offsets[i].repeat((mat_rotations.shape[0], 1, 1))
        # multiply the offsets by the parent's rotmat
        q = k @ global_rot[:, parent]
        # add the offsets to the parent's position
        global_pos[:, i] = global_pos[:, i] + torch.squeeze(q, 1)
    return global_pos

I’ve enabled anomaly detection, and it seems like the issue is in this line:
global_rot[:, i] = mat_rotations[:, i] @ global_rot[:, parent]

I tried creating global_rot with torch.zero_like and ended up with the same result.
Also, note that I’m getting this error already at index 1 , right after this line (when testing backprop on global_rot[:, I]).

Any suggestions / ideas on how to solve this? Thanks in advance!

2

I think I found a solution that doesn’t require assignment.
I had to create a separate function that calculates a joint’s global rotation matrix and call it each time I need a parent’s global rotation.
calculating q would look like this:

parent_rot = _get_joint_global_rot_mat(parent, mat_rotations, parents)
q = k @ parent_rot

This way I’m not getting the in place modification error I was getting earlier. The caveat is that I’m recumputing the same multiplications multiple times, so it’s less efficient. Since I’m running this with torchscript it’s less of an issue.

I guess I didn’t realize how much more limited I am with the options I have to implement anything that runs through backprop.