How to do gradient clipping in C++?


(Thomas Johnson) #1

Can someone explain how to do gradient clipping in C++? As far as I can see there is no equivalent to torch.nn.utils.clip_grad_norm_ and you have to use clamp instead. Exactly how and where to call clamp in my training loop isn’t clear to me though.