I have problem with calculating torch.nn.functional.kl_div because I regularly got negative output values. I’ve thought that problem is in some numerical instability. I found that such problems were already addressed in the following issue Numerical problems with torch.nn.functional.kl_div · Issue #32520 · pytorch/pytorch · GitHub by adding log_target flag to the kl_div function. Unfortunately such solution didn’t resolve problems in my particular case. I will be very appreciate if someone could look at the code and give some advice.
Link to the data and code sample kl_div_problem - Google Drive