I’m looking at a line of code from the chatbot tutorial. The tutorial is here:
The code is for clipping the gradient. How does the author come up with the clip value of 50? Below is some code that mimics the code in the tutorial
clip = 50.0 _ = torch.nn.utils.clip_grad_norm_(self.model_0_wra.parameters(), clip)
Thanks for any help.