I am trying to compare the empirical distributions of elements of two tensors by computing a coarse histogram of the two tensors (
torch.histc is not differentiable). I want to compute the gradients of a particular loss function measuring the dissimilarity between the two distributions w.r.t the input tensor which is a leaf variable. However the returned grad is always None. Below is a minimal example of what I’m trying to do. Any suggestions?
import torch def hist_loss(X, Y, bins: int=10): X = X.view(-1) Y = Y.view(-1) xmin, xmax = X.min(), X.max() ymin, ymax = Y.min(), Y.max() xstep = (xmax - xmin)/bins ystep = (ymax - ymin)/bins hist_loss = torch.zeros(1, requires_grad=True).to(X) zero = torch.zeros(1, requires_grad=True).to(X) one = torch.ones(1, requires_grad=True).to(X) n = len(X) for i in range(bins): # x is the empirical distribution of the X # y is the empirical distribution of the Y x = torch.sum(torch.where(((X>xmin+i*xstep) & (X<xmin+(i+1)*xstep)), one, zero)) y = torch.sum(torch.where(((Y>ymin+i*ystep) & (Y<ymin+(i+1)*ystep)), one, zero)) hist_loss = hist_loss + ((x-y)/n)**2 hist_loss = torch.sqrt(hist_loss) return hist_loss x = torch.randn(1000, requires_grad=True) y = torch.randn(1000) z = hist_loss(x, y) z.backward() x.grad is None