Hi all! Started today using PyTorch and it seems to me more natural than Tensorflow. However, I would need to write a customized loss function. While it would be nice to be able to write any loss function, my loss function is a bit specific.So, I am giving it (written on torch)
X = np.asarray([[0.6946, 0.1328], [0.6563, 0.6873], [0.8184, 0.8047], [0.8177, 0.4517],
[0.1673, 0.2775], [0.6919, 0.0439], [0.4659, 0.3032], [0.3481, 0.1996]], dtype=np.float32)
X = torch.from_numpy(X)
y = np.asarray((1,3,2,2,3,1,2,3), dtype=np.float32)
y = torch.from_numpy(y)
def similarity(i, j):
''' This function defines the similarity between vectors i and j
inputs: i, j - vectors of the same length
sigma - the denumerator parameter
output: sim - similarity value (real number from 0 to 1) '''
dist = torch.norm(i - j)
''' This function creates the similarity matrix of a dataset
input: mat - dataset in matrix format
sigma - a paramter which defines similarity
output: simMatrix - the similarity matrix '''
a = mat.size()
a = a
simMatrix = torch.zeros((a,a))
for i in xrange(a):
for j in xrange(a):
simMatrix[i][j] = similarity(mat[i], mat[j])
n = y.size()
n = n
converted_y = torch.zeros((n, n))
for i in xrange(n):
for j in xrange(n):
if y[i] == y[j]:
converted_y[i, j] = 1
def customized_loss(X, y):
X_similarity = similarity_matrix(X)
association = convert_y(y)
loss_num = torch.sum(torch.mul(X_similarity, association))
loss_all = torch.sum(X_similarity)
loss_denum = loss_all - loss_num
loss = loss_num/loss_denum
loss = customized_loss(X, y)
Now, of course, considering that I am going to use it as the final layer, of the neural net, I would need to compute the gradients of it and then use them in the backpropagation.
Explaining the function a bit:
I first transform the input data space into a kind of similarity matrix (0 it means the data being the same, the higher the number in ij-th entry, the higher is the dissimilarity). Then in order to find the intra-cluster loss, I multiply this matrix with a 0/1 matrix, where the ij-th entry is 1 if the element i and j are in the same cluster, 0 otherwise. The intra-cluster loss is find similarity, and finally, we just divide the two losses.
My questions are:
- Can this be done in PyTorch, without writing Lua code?
- Can the gradients of this be computed in an automatic way (torch autograd)?
- Can such a loss function be given as input in optim.SGD? (optim.X in general case where X is the optimization algorithm)
Thanks for any answer, or possible hint.