CrossEntropyLoss for Next-Item Prediction (itemID starts from NUM_USERS)

I wanna solve user-item prediction issue. For example, dataset contains 2 users and 5 items. So,

userID: [0, 1]
itemID: [2, 3, 4, 5, 6]

I embed the nodes (both users and items nodes) and get the nodes embeddings after my model. For prediction, I was wondering if I should minus num_users (2 here) in the CrossEntropyLoss?

The code snippet is as following:

nodes_embs = nn.Embedding(2 + 5, emb_dim)
users_embs = nodes_embs[:2, :]  
items_embs = nodes_embs[2:, :]

scores = users_embs.matmul(items_embs.t())    # [2, 5]

# the labels like 
labels = torch.tensor([5, 4])    
# user1 would choose itemID=5 for next-item
# user2 would choose itemID=4 for next-item

# I was wondering if I should minus the itemID in labels tensor?
# like [5-2, 4-2] ?
loss = cross_entropy_loss(scores, labels)

Or how to get cross-entropy loss for the itemID starts from num_users issue? Thanks in advance.

Hi icmp!

Yes, this is correct.

The labels you pass into CrossEntropyLoss should be integer class
labels that run from 0 to nClasses - 1. So if your raw labels are in
(2, 3, 4, 5, 6) you need to map them to (0, 1, 2, 3, 4) by
subtracting 2.


K. Frank

Hi Frank @KFrank ,

Thank you for your reply.

I have another question about updating gradients of some specific items. As described above, I have 7 nodes, 2 for users and 5 for items. In one part of my model, I only need updating items embeddings. The code is as following:

nodes_embs = nn.Embedding(2 + 5, emb_dim)
nodes_weights = nodes_embs.weight

new_items_embs = nn.Embedding(5, emb_dim)
new_items_weights = new_items_embs.weight

# If I wanna update nodes_weights[2:] with new_items_weights, I try
nodes_weights[2:, :] = new_items_weights

but it raises RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

I was wondering how to solve such errors. Thanks in advance.

Hi icmp!

I don’t understand your use case, but I will make a few comments:

First, is nodes_weights = nodes_embs.weight something that you
update entirely manually, something you update solely by training with
backpropagation / optimization, or do you do both?

If you are not using optimization, just set requires_grad = False
for the original weight tensor that you refer to as nodes_embs.weight.

(if you are only using optimization, you shouldn’t be manually overwriting
elements of nodes_embs.weight.)

If you do want to overwrite some weight elements that are also being
optimized, use a with torch.no_grad(): block:

with torch.no_grad():
    nodes_weights[2:, :] = new_items_weights


K. Frank