I wanna solve user-item prediction issue. For example, dataset contains 2 users and 5 items. So,
userID: [0, 1]
itemID: [2, 3, 4, 5, 6]
I embed the nodes (both users and items nodes) and get the nodes embeddings after my model. For prediction, I was wondering if I should minus num_users (2 here) in the CrossEntropyLoss?
The code snippet is as following:
nodes_embs = nn.Embedding(2 + 5, emb_dim)
users_embs = nodes_embs[:2, :]
items_embs = nodes_embs[2:, :]
scores = users_embs.matmul(items_embs.t()) # [2, 5]
# the labels like
labels = torch.tensor([5, 4])
# user1 would choose itemID=5 for next-item
# user2 would choose itemID=4 for next-item
# I was wondering if I should minus the itemID in labels tensor?
# like [5-2, 4-2] ?
loss = cross_entropy_loss(scores, labels)
Or how to get cross-entropy loss for the itemID starts from num_users issue? Thanks in advance.
The labels you pass into CrossEntropyLoss should be integer class
labels that run from 0 to nClasses - 1. So if your raw labels are in (2, 3, 4, 5, 6) you need to map them to (0, 1, 2, 3, 4) by
subtracting 2.
I have another question about updating gradients of some specific items. As described above, I have 7 nodes, 2 for users and 5 for items. In one part of my model, I only need updating items embeddings. The code is as following:
nodes_embs = nn.Embedding(2 + 5, emb_dim)
nodes_weights = nodes_embs.weight
new_items_embs = nn.Embedding(5, emb_dim)
new_items_weights = new_items_embs.weight
# If I wanna update nodes_weights[2:] with new_items_weights, I try
nodes_weights[2:, :] = new_items_weights
but it raises RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.
I was wondering how to solve such errors. Thanks in advance.
I don’t understand your use case, but I will make a few comments:
First, is nodes_weights = nodes_embs.weight something that you
update entirely manually, something you update solely by training with
backpropagation / optimization, or do you do both?
If you are not using optimization, just set requires_grad = False
for the original weight tensor that you refer to as nodes_embs.weight.
(if you are only using optimization, you shouldn’t be manually overwriting
elements of nodes_embs.weight.)
If you do want to overwrite some weight elements that are also being
optimized, use a with torch.no_grad(): block:
with torch.no_grad():
nodes_weights[2:, :] = new_items_weights