On PyTorch 2, I did the following:
target["boxes"] = torch.as_tensor(np.array(np.zeros((0, 4)), dtype=float))
target["labels"] = torch.as_tensor(np.array([], dtype=int), dtype=torch.int64)
make sure after the transform(img, target) the shape and dtype of both arrays are right.
target["boxes"].shape = torch.Size([0, 4])
target["labels"].shape = torch.Size([0])
target["boxes"].dtype = torch.float64
target["labels"].dtype = torch.int64