[Strange GPU memory behavior] Strange memory consumption and out of memory error

[D-X-Y (XY-D)]

I meet a similar problem.
Which is caused by torch.backends.cudnn.benchmark = True, when change to torch.backends.cudnn.enabled = True, everything is ok.