Hi there, I am trying to implement a loss function that depends on pairwise distances of output data. Specifically I have neural network f which take X as input and the output is $Y = {y_i}_{i=1}^N$. My error function has following form:
$E = | d_{ij} - c_{ij} | ^2$
where d_{ij} is distance between y_i and y_j, and c_{ij} is the target value for this distance.
I know how to write “vectorized” loss function like MSE, softmax which would take a complete vector to compute the loss. But in my case, it seems that I have to do “atomistic” operations on each entry of the output vector, does anyone know what would be a good way to do it?
Lets’s say the vectors that you want to take pairwise distances are in a tensor A of shape (N, D), where N is number of vectors and D is the dim. Then we can create two tensors, of shape (N, N, D). For the first tensor B, B[i][j] = A[i] for 0 <= i < N, and for the second tensor C, C[j][i] = A[i]. Then we can view these two tensors as (NxN, D) tensors and do use the pairwise distance module. This probably isn’t the most efficient way, but doesn’t require much addition coding on your part.